Character entropy of Voynichese

[Sam G]

Basically the low entropy rules out the possibility that the VMS is a cipher, because very few kinds of ciphers can lower entropy, and these can all be excluded for other reasons.

Here's an interesting bit from the list archives.  Jacques Guy says:

You are not allowed to view links. Register or Login to view.

There aren't many weapons in the arsenal. If the VMs is a cipher, then
it is a cipher which _lowers_ the entropy of the text. The only cipher
that can do that (Jim Gillogly, please correct me) is a cipher that
encodes single characters of the plain texts as sequences of characters,
or whole words and sentences. E.g. cat -> cloakarmtower (c > cloak, etc.)
Even so, the cipher has to be a bijection: c becomes cloak only and not other
words. There is another possibility: lots of nulls. It is much like
cat -> cloakarmtower, except that you are allowed any number of words
to encipher each letter _provided that_ they are built on strict, narrow
patterns. I'll take French "javanais" to illustrate this. The rule is:
insert "av" before the first vowel of every syllable. Thus: "bonjour"
-> "bavonjavour". Modify the rule to: insert "av", or "ov", or "ugl"
and you get a cipher text with a lower entropy than the plain text.
In that case, the VMs is very short, and the labels must be ignored.

Jim Gillogly's response:

Besides these cases, some ambiguous ciphers such as the Keyphrase
can lower the entropy.  As an English example, you can have:


plaintext:  abcdefghijklmnopqrstuvwxyz
ciphertext: THEPRESIDENTSPEAKSNONSENSE


In this case "my hovercraft is full of eels"
becomes      "SS IESRSESTEO DN ENTT EE RRTN"


Note the excess of S's.  The Keyphrase will frequently produce
runs of three or four of the same letter, and occasionally
adjacent ciphertext words which represent different plaintext
words.  However, the Keyphrase and other entropy-reducing
ciphers I know (except those that use lots of nulls, as Jacques
points out) also reduce the size of the alphabet.


Unless one is unlucky, the recipient will be able to get most
of the words right given the mapping, as will the cryptanalyst
given enough material.

In the case of the VMS, the keyphrase thing would've been cracked already as it is basically a form of simple substitution cipher (and probably wouldn't lower the entropy enough anyway), and verbose ciphering (or the addition of verbose nulls) can be ruled out because the words in the VMS simply aren't long enough, and there aren't enough repeated 2+ word phrases to consider single plaintext words enciphered into multiple ciphertext words.  The presence of single-word labels also poses a major problem for any scenario that involves treating the VMS words as anything other than words, or at least stand alone "units" capable of bearing meaning (as opposed to merely parts of verbosely enciphered words).

Really, there is no way of escaping the conclusion that the low entropy is simply an intrinsic part of the content, not the result of some "cipher mechanism".  The most likely explanation for that is that the VMS is written in a language which has low entropy, i.e. that has a relatively rigid phonotactic structure.  From that point of view, there is nothing "anomalously low" about the entropy at all, since there exist languages with lower entropy, i.e. with a more rigid phonotactic structure than we find in the VMS text.