Aanother statistical take on Voynichese

[Jorge_Stolfi]

So here's my analysisn You are not allowed to view links. Register or Login to view.
Results strongly indicate that the text was generated via simple methodology (order-3 markov chain).
Also this shows that there is zero measurable correlation between text and the imagery.  Does it prove anything? Guess not, however result seem to leave little to imagination.

your generated sample, which you describe as "indistinguishable from Voynichese to the eye" appears a bit odd to me:
qoeeal shd shos lsheodain oteedy dchdar ol ar okold qokal qopchdy or olkchy qokeey yteey otair deeol shar chedy chor ykodaiin
It's reasonably good, which is of little surprise given the order-3 Markov chain

Indeed.  An order-3 Markov can probably generate also text that will be "indistinguishable from English to the eye" (of someone who cannot read English).  Here is the output of Claude Shannon's classic experiment: "IN NO IST LAT WHEY CRATICT FROURE BIRS GROCID. PONDENOME OF DEMONSTURES OF THE REPTAGIN IS. REGOACTIONA OF CRE"

An order-3 Markov will reproduce the frequency distributions of characters, bigrams, and trigrams of the training set.  Thus it will reproduce the character entropies H0, H1, and H2, and any correlations of characters across word gaps.  It will also approximate correlations over longer distances, because of chained correlations (the 4th letter depends on letters 2 amd 3, but these depend on letter 1, so letter 4 will partly depend on letter 1 too.)  IIRC, there is a good chance that it will produce output with a Zipf-like word frequency distribution.

An order-3 Markov may fail to capture some subtler properties of the training set, such as the distribution of word type lengths or a rule like "a word can have at most one letter 't'  or 'k', but not both".  But the "core-mantle-crust" structure of Voynichese words makes it possible for an order-3 Markov to "learn" the rule "a word can have at most one gallows". That's because the last 3 glyphs will almost always tell the generator whether it is in the "prefix" part ("uphill", before the gallows) or in the "suffix"  part ("downhill", after the gallows); and thus it can avoid generating a second gallows in the latter.  

On the other hand, an order-3 Markov may miss the rules "at most three benches" (Ch, Sh, ee; counting platform gallows like CTh) and "at most three dealers" (d, l, r, s) that seem to hold for Voynichese words.  Do we see exceptions to these rules in the output of your generator?

All the best, --stolfi