How multi-character substitution might explain the voynich’s strange entropy

[Mauro]

But I also agree with @Jorge_Stolfi here: n-grams entropies do not necessarily mean much because they are heavily dependent on the transcription. Word entropies are more reliable (they only depend on 'space' being actually a 'space', and on differently written words being actually different words).

I don't agree with Stolfi's dismissal of letter-based statistics. We don't know if Voynichese is a phonetic representation at all. Whatever it is, is very much dependent on its writing system. So to get a clue about what's going on, we must compare different ways of transliterating Voynichese to different writing systems. Including codes and ciphers, which are essentially also writing systems.

I don't think letter based statistic should be dismissed altogether, but only that characters-based entropies are less reliable than word-based entropies (I'm not sure also @Jorge_Stolfi thinks exactly this, I interpreted him this way).

Ie: the bigram entropy (of the Voynichese.com transcription in this case, I don't have the data ready at hand for RF1a-n) is 6.02 for pure EVA; replacing 'ch' = 'C', 'sh' = 'S', 'qo' = Q, and the gallows 'cth' 'ckh' etc. with T,K,P,F raises the entropy to 6.19 [it's a perfectly reversible transformation]; replacing also  'iii' = 'J', ii='I', 'eee' = 'U' and 'ee' = 'E' raises the entropy to 6.38; and so on (*). While in all these cases the 1-word entropy remains exactly constant at 10.454 (**) (and will remain constant under any possible substitution of any EVA string, provided the transformation is reversible)

What I do vehemently agree with is that people must be more aware that Voynichese =/= EVA. When running letter-based tests, there must always be awareness of the choices made and their effects.

I vehemently agree too!!!!!


(*) In the most extreme case, replacing each word type with a different symbol (a rather weird, but perfectly legit 'transcription') will bring the 1-gram entropy to the same value of the 1-word entropy (10.454)

(**) which, by the way, I just noticed is rather high. Chanson de Roland gets 9.686, the complete works of Shakespeare get 9.974, Boccaccio's Decameron gets 10.006, Caesar's De Bello Gallico has the highest 1-word entropy (as expected for a language as Latin) at 11.211. Classic Greek (Aeschilus, Prometheus Bound) gets 10.457, but with the caveat I removed all the diacritics (entropy should be higher with the diacritics, taht is to say, the tones, in place).