22-12-2025, 10:05 PM
(22-12-2025, 08:41 PM)Rafal wrote: You are not allowed to view links. Register or Login to view.Personally I am still struggling with understanding statistical research of VM. These guys often claim that Voynichese behaves similarly to real languages. ... Because if it is only clusters then it doesn't prove similarity to real languages.
Indeed, there is no convincing proof yet that it *IS* natural language. We only have many statistical properties that it has and that natural languages also have, but which some forms of gibberish would not have.
For example, imagine an algorithm that made up each word by combining a a prefix, core, and suffix from three given sets (as in one of the many proposed "word paradigms") of sizes p,q,r, with each component drawn at random. This would generate a text where each word type would appear with the same frequency 1/(p q r), hence with a flat Zipf plot (of frequency x rank). But Voynichese has Zipf plot that is similar to that of other languages.
One could have a more complicated algorithm where each of the three components is randomly chosen from its set according to unequal frequencies. Like, by stuffing the "prefix" bag with four copies of qo, two copies of cho, etc. This would generate words with unequal frequencies, and the result could approximate the Zipf law better. But to get a good fit, as we see, would require careful tuning of the contents of each bag -- and awareness that such tuning was needed...
Or imagine an algorithm that generates random words with a Zipf-like distribution, but generates each word independently from the previous ones. Then the frequency of a word pair "W1 W2" would be the just product of the overall frequencies of W1 and W2. But that is not the case of natural languages: there are always forbidden pairs and pairs that occur more often than expected from their individual freqs. And that is also the case of Voynichese...
People have noticed that the first word on a line, or after an obstruction like a plant stem, is somewhat longer than average; while the last word or two are somewhat shorter. This has been taken as a sign that Voynichese is not a natural language and/or that the encoding algorithm is sensitive to line breaks -- the "LAAFU" hypothesis. However, the same deviations in word length have been seen to occur when text in any language is written within fixed margins with the banal line breaking algorithm ("write the next word on the same line if it fits, otherwise break the line before it"). This result not only could explain away the LAAFU data, but, if so, it would be evidence that the text was copied from a draft, with the Scribe ignoring the line breaks of the latter and inserting new breaks as needed.
And so on. AFAIK, no one has found a statistical property of Voynichese that does not occur also in some natural language.
All the best, --stolfi