Discussion of "A possible generating algorithm of the Voynich manuscript"

[MarcoP]

Hello. I did some statistical analysis on the most frequent repeated word-sequences, and some analysis on the transition relationships between characters with principal component analysis (PCA) on the same auto-generated text that Timm & Schinner used in the article, compared to an excerpt of the Voynich manuscript (the 'recipes' section, f103r-116v, the same excerpt they were aiming to simulate with their algorithm in the article). I used my routines written in C++ to extract and assemble the information.


Comments on the results text files (attached)

The sizes of both texts, measured in number of words (tokens), is similar: 10832 vs 10681. However, Timm's and Schinner's algorithm under-predicts the number of unique words somewhat (2228 vs 3103 in the Voynich text). The frequency distribution of the most frequent words seems similar, except for the three most frequent words, which are over-predicted by approximately twice the amounts. 

Hi Jonas,
the graphs I attach compare Timm and Schinner's generated text with each individual section in the VMS (Takahashi transcription). It is possible that I have made errors along the way, but I think the variability illustrated by the graphs is actually there.
The top graph measures Moving Average Type Token Ratio (a technique that Koen discussed extensively on the forum and for which Nablator provided a Java implementation). I used a 200 words window. This is closely related with the number of unique words, but the moving average allows comparing texts of different lengths. Timm and Schinner's generated text has a MATTR value that is lower than all the VMS sections, but it is quite close to the value for Quire 13 (aka the Biological or Balneological section).

Frequencies of the most common words are somehow anti-correlated with respect to MATTR: if you have more word types (higher MATTR) the most common words tend to be less frequent. Here the differences between Voynich sections are striking: the most common word in Quire 20 (aiin, 1.8%) is less than half as frequent as the most common word in Herbal A (daiin, 5.0%). As discussed by Timm and Schinner, the fact that the most common words are different in the different sections is even more puzzling. BTW, frequent words in Q20 have very similar frequencies (non-Zipf-like): this results in different trancriptions pointing out different words has the most frequent.

As you say, the most common words in the generated text are much more frequent than those in Quire20, but (as for MATTR 200) the distribution is quite similar to that observed in Quire 13.