[split] The Zipf law and the Voynich Manuscript

[Jorge_Stolfi]

This assertion that the Voynich script follows Zipf's law is starting to seem like dogma to me.

No, it is an empirical observation (that is in fact a problem for many theories).

Many natural languages deviate from Zipf's law in the first most common words, for various reasons.  For instance, the most common words in Romance languages include the definite articles, In most of these languages there are four articles (masc and fem, sing and plural). Their frequencies are affected by the frequencies of genders and numbers in the language's lexicon.  But different Romance languages, even closely related ones, often assign different genders for the same noun.  Moreover, in Italian there are more than four articles, because one has different forms depending on the initial of the next word.  This alone means that Italian and Spanish cannot both follow Zipf´'s law.

Here are some actual Zipf plots that I prepared years ago for my 2012 presentation at Mondragone:  (Roadside Picnic is a sci-fi novel in Russian):

    [                                                         

English prose  texts in fact are exceptional because they do follow Zipf's law with remarkable accuracy, even at the high-frequency end.  This is due to Englsh having lost the notion of noun gender and the need to inflect articles and adjectives for number,so there is only one definite article.   Medieval English however already deviates from the ideal law;  I did not check why exactly.

   

But a complicated cypher that maps the same word type to different word types, like the Vigenère ciphers, can completely unzipfy the plot:

   

On the other hand, the East Asian languages with monosyllabic "words" deviate significantly from the ideal at both ends, high and low frequency:

                 

And Voynichese deviates from the ideal only as much as some of other natural languages: