Measuring Long-Range Structure in the Voynich Manuscript

[nablator]

This could suggest that, in these cases, a significant part of the structure is carried by the sequential chaining of words, meaning that each word may depend more directly on the previous one than in typical natural language texts.

We already know that there are very significant non-random patterns across word breaks in the VMS and also in Torsten's generator, and I see now he has a setting "method.canFollow=statistic" for selecting the follow-up word. It could be interesting to play with conf.properties to activate/deactivate settings that have an effect on word ordering.

So the loss of predictability in a word-shuffled version of the Voynich text accounts, understandably, for the gap between the two curves at medium range (at a distance consistent with the selection of the 1st letter of the bigram in a word and the 2nd in the next word or next few word because the selection of the next word also impacts the words after it), but I don't understand why the gap remains at very long range.

Distance 1-300:
   

Distance 1-3000 (dotted black curves are the moving average of the 50 previous values) :
   

Note: I am using the "circularized" version of the conditional entropy calculation so the number of bigrams in the calculation is the same for any distance.

We know for sure that Torsten text is from a generative algorithm... If Voynich has the same pattern... can we say it is also artificially generated?

If only it were that simple to test artificiality.