Measuring Long-Range Structure in the Voynich Manuscript

[quimqu]

Even if entropy is not predictability, they are very linked.

For a well-known measure of predictability of characters separated by distance d, I could modify slightly my very simple code (Java or JavaScript) that calculates the conditional character entropy (h2), to calculate h2(d), using the conditional probability of the character at index + d (modulo the size of the text to avoid border effects), knowing the character at a given index, instead of d = 1 for h2.

Isn't MI(d) = h1 - h2(d) ? Not sure...

h1 = First order character entropy = Shannon's entropy
h2 = Second order character entropy = Conditional entropy

I would be surprised if h1 - h2(d) doesn't drop to near zero (asymptote at zero) very quickly (approximately when d > 25) and even more quickly (approximately when d > 10) when the text is word-shuffled.

Yes, formally it is, but in real texts H2(d) does not approach H1 as quickly as intuition suggests because long-range statistical correlations keep a small but measurable dependence even at larger distances.