Speculative fraud hypothesis

[Jorge_Stolfi]

various Zipfian laws as well as frequency distributions were also compared in the article.

Still, the parameters of the algorithm can be tuned to make the output be similar to any meaningful text by all those criteria.  The fact that the text they created matched those features of Voynichese only means that the parameters were properly tuned to achieve that result.

Note that every output word will be a copy of one of the seed words, modified by some random number M of iterations of Mutate. The expected value Mavg of M is approximately Q log(N/S), where Q is the mutation probability, N is the number of words generated before the word in question (including the seed text), and S is the number of words in the seed text. 

In particular, for N = 5'000 (middle of the first 10'000 words) and S = 12 (as in their recent post), log(N/S) is ~7; that is, each output word will be a seed word copied ~7 times and mutated ~7 Q times, on average.  If we let the algorithm run for 1'000'000 iterations and take only the last 10'000 words, letting N be 995'000 (middle of the taken words) gives log(N/S) as ~12, meaning that each output word will be a seed word copied ~12 times and mutated ~12 Q times.

To get a reasonable match of the FxR plot, the mutation probability Q must be large enough for most output words to be mutated at least once. (Otherwise the output would be mostly copies of the S seed words; other words would be few and would have much smaller frequencies.) Namely, for N = 10'000 and S = 12, Q must be on the order of 0.05 (5%) or more.  Then at least 30% of the first 10'000 words will be mutated, which should result in ~3000 distinct words. 

Digraph frequencies

However, if Q is large enough to produce a reasonable FxR plot, the letter and digraph distributions of the generated text will be determined largely by the Mutate procedure.  These statistics will be "correct" only if the Mutate procedure is tuned to produce them.  A simple "untuned" mutate procedure, that blindly inserted, deleted, or replaced a letter at random at a random point of the word -- even with the proper letter frequencies -- would ultimately produce a flat random x random distribution of digraphs. 

For example, suppose that the Mutate procedure were simplistic and just added q at the beginning of the word with some probability P1 and deleted the first letter with some probability P2.  By adjusting P1 and P2 one could get the "correct" frequency of q in the output.  However, that procedure would create many qCh, qt, and qq pairs, which do not occur in Voynichese.  In order to avoid this problem, the Mutate procedure must add q only as a pair qo, and only in front of words that do not start with q, with suitable probability.  And a similar observation applies to every glyph pair.

Therefore, if the algorithm produces a large enough number of new words, and reproduces the letter and digraph distributions of Voynichese, it means only that that the Mutate procedure was well-tuned to preserve those letter and letter pair distributions.  And indeed their Mutate procedure, IIUC, parses the word to be mutated according to a Voynichese word structure model, and changes each part in specific ways with carefully chosen probabilities.

FxR plot

It is known that the output of very simple random word generators, such as a character-based Markov chain of order zero, can have a frequency x rank plot (FxR plot) that follows Zipf's law.  Thus it is not surprising that the output of the T&T algorithm can do the same, if the mutation probability is properly chosen.

As noted above, if Q is too low, most output words will be copies of the seed words.  The number L of distinct words will be only a little bigger than the seed size S. Then the FxR plot will be roughly flat at about 1/S for ranks 1 to S, and then nearly zero from S to L. 

On the other hand, if Q is too high, most output words will have been affected by multiple rounds of Mutate, and therefore will likely be unique in the output sample, or nearly so.  Among the first 10'000 output words, the number L of distinct words will be several thousand, and each of these word types will occur only a few times. The FxR plot will then be nearly flat for ranks 1 to L.

Between those two extremes there will be a sweet spot where the output text will have the right mix of seed words with varied number of Mutate rounds.  Words that have been through only one or two rounds of Mutate will often be equal.  Therefore there will be word types with quite varied frequencies, and the FxR plot will be close to the ideal Zipf plot.

To illustrate this point, here are three runs of my implementation of the T&T algorithm,
with Q = 0.01, Q = 0.20, and Q = 0.99.  (The reset probability was set to 0.20, so that the source index was reset after copying  5 words on average.  The reference text "WoW" is 10'000 words extracted from Well's novel War of the Worlds.  The seed text is the first 12 words of that sample, namely "like that of the thing called a siren in our manufacturing towns". My Mutate procedure is the "dumb" one I described previously, that is insensitive to word structure except that when inserting a letter it  uses the approximate letter frequencies of English, irrespective of the insertion point; and when replacing a letter it tries to preserve its class -- vowel, stop, or sibilant.)

---

Q = 0.01   
    txout-200-010-2.png
  called a siren in our manufacturing a siren our manufacturing towns
  called a siren in our called called called manufacturing a our thing
  called a siren in our ... called manufacturing called our a in towns
  called a a called a a siren in called manuacturing a called
  manufacturing called our manufacturing called called

  9497 valid words,  503 invalid,  85 word types

---

Q = 0.20
    txout-200-200-1.png 
  manufacturing towns siren manufacturing in our in our in caled in our
  manufacturing towns manufacturinge tons siren manufacturing our
  manufacturing owns manufacturinge ... manufturinge ol mamufacturng
  is owr wn fih ouor ol tynz siren manufactring usiren in trat thak
  owjs manufacturinge iz zide that ohmh like that ien the e o oz thop

  3915 valid words, 6085 invalid, 1149 word types

---

Q = 0.99
    txout-200-990-4.png
  lie that manufactuing otowns phat ef the thiing calfled na ssiren in eur
  siren ih eur sireen ir eyr sireer er eys if our manufakturing townz liy
  that manfactuing manufactugring ... ni mnoaqueg lhiwwh thiifvc xafs
  adifi nfw mrtwgjingh ea uhplzq uz ese yazo iwtya ucy qhynp ecegr hsiwnh
  qlu kog h khianlx cyso geq odumo bsfineine wvf enl

  324 valid words,  9676 invalid, 7469 word types   

---

For reference, the English 10'000-word sample has 2360 word types.  As one can see, with Q = 0.2 the FxR plot matches very closely that of the English sample.  Even with that crude Mutate procedure, that almost always produces an invalid English word, almost 40% of the first 10'000 words are valid (meaning that they occur in the full text of the novel). 

The "founder effect"

The T&T algorithm has another problem that is more apparent when the seed text is very short (like a single line, as stated in that post).  Because of the random nature of the sampling, it often happens that, among the first couple dozen output words, some seed words are replicated several times more than others, just by chance.  Any such initial imbalance will soon become "frozen" as those initial words get copied and re-copied.

For example, suppose that the seed is "from the summit of these pyramids forty centuries look down upon you' (S = 12).  At this point, each of these words has frequency 1/12.  Now let's say that the algorithm starts copying at the word 'upon', and keeps copying for five iterations.  Now the text will have 17 words, and will end with "down upon you upon you upon you upon".  It will have four occurences of "upon", three of "you", and one each for all the other words.  (Assume there are no mutations so far.)  Even if the source pointer is reset at this moment, new words will be about four times as likely to descend from "upon" than from "pyramids"; and this bias will persist for a very long time.

Because of this "founder effect", the frequencies of letters and digraphs in the output text may be significanly different than those in the seed text, and the FxR plot may deviate from the Zipf law at the high-frequency end.  Moreover, these distortions will be different in different runs of the program, even if all parameters are the same.

Here are four runs of my implementation, all with Q = 0.20 and all other parameters fixe as above:

    txout-200-200-1.png
    txout-200-200-2.png
    txout-200-200-3.png
    txout-200-200-4.png