Spaces and Entropy

[Koen G]

Brought in here from another thread:

But if applying transformations - at the beginning and then - removing spaces?

This is something I was planning to look into back when I was writing about entropy, but for some reason I didn't fully pursue this line of investigation. I did do a little bit of it in the opening post of this thread, but didn't take it all the way yet. This approach may have some potential.

To run a quick test, I referred back to my original You are not allowed to view links. Register or Login to view. where apparently I had come up with a series of transformations for Herbal A: [qok, chol, chor, che, chy, ol, cho, or, qot, ar, eey, al, qo]. Note that this is after eliminating the possible EVA-effects [ch, sh, ain, aiin].

With spaces conserved, I got the following entropy values for HA, which I still considered a bit too low:

h1 = 4.20
h2 = 2.94

Now, if we assume the VM somehow messed with spaces as well, we can just take spaces out of the equation. So if I remove spaces from the transformed file and compare it to a no-space corpus, this is what we get:

   

The closest dot to the red one is a Slavic text with the following values (h0, h1, h2) 5.044394119, 4.556408709, 3.634152131
The values for the transformed VM file are 5.209453366, 4.570687902, 3.650064015

Note that at this point I have been messing a lot with the file and all of this should be approached with caution. What I would say is the following:
If we assume a verbose cipher and omit spaces as a variable, it is possible to reach normal entropy values for Voynichese. Whether this leads to a viable "text" is a whole different question.

Also, so far I have only been performing transformations based on numerical data. A real solution would ideally look for a more systematic solution (for example, [o] is a null, [o] modifies the preceding glyph, stuff like that).