[split] The Zipf law and the Voynich Manuscript

[Gabriel L]

I'm writing in this old thread because it directly addresses Zipf's law. It's often said that the main text of the Voynch follows Zipf's law, but not the labels. This has been assumed, along with other things, in manuscript research, but I haven't seen a reasoned and convincing explanation.

Greetings,
I have been somewhat detached from the vms for several years, but every so often I take a look in the internet. René mentioned to me this site some time ago, so 'hello everybody'.
Yes I think labels should be best excluded (I am assuming labels to be names, or verbs or... ? which is not what Zipf's law is about (i.e. an observation that seems to happen in linguistic corpora, not words lists).
For the rest of the argument, I suggest reading the paper in Cryptology, Oct 2001, Vol xxv, 4, where it is described (among other things) that the vms "approximately follows" Zipf's law of word frequencies and also Zipf's law of word lengths (not in terms of phonemes as originally described by Zipf, but in number of characters).
To do this is relatively simple, compute the slope of the curve of log(rank) vs log(frequency). Since we are talking about an empirical law, we should expect data points deviating from the ideal slope of -1.
In addition one should be careful that in some places it is not completely clear if the spacing is a marker of different words, so such departures are expected in particular in the vms. Also, for low frequency words, the plot becomes 'stepped', so the rank-frequency relation cannot be strictly a straight line.
Claiming the second most common word is seen more often than what Zipf's law predicts cannot be taken as evidence that the text is not meaningful. Such departures happen in all plots. What is more important is the main trend in the data.

Regards
G.