Mapping Voynich connections through rare tokens

[Torsten]

...

Hello qumiqu!

The raw numbers need context. 197 repeated trigrams in 37,000 words is less than expected for natural languages — moreover two questions matter more than the count:

Do they repeat in fixed order? In Timm 2014 (p. 3) I found only 35 word sequences of three or more words appearing at least three times. Of those 35, only 5 have fixed word order. The other 30 have variable word order — the same words appearing together but in different arrangements. For instance "chol," "shol," and "cthol" occur together three times, each time in different order:

f1v: "chol cthol shol"
f4r: "chol shol cthol"
f42r: "cthol chol shol"

In a natural language, phrases have fixed word order because grammar constrains the sequence. In the VMS, word order varies — which is expected if similar words co-occur because they belong to the same word family being copied in the same production session.

Do they repeat because of meaning or because of similarity? In 24 out of those 35 cases, the repeated sequences use at least two words that are either spelled the same or very similarly (Timm 2014, p. 3). The repetition reflects word-family clustering, not grammatical phrases.

Reddy & Knight (2011, p. 82) quantified this directly by measuring how much the previous word predicts the next word:

Text	Unigram	Bigram	Improvement
VMS B	2.30%	2.50%	8.85%
English	4.72%	11.9%	151%
Arabic	3.81%	14.2%	252%
Chinese	16.5%	19.8%	19.7%
Hungarian	5.84%	13.0%	123%

In any meaningful text, knowing the previous word dramatically constrains what comes next — 151% improvement for English, 252% for Arabic, 123% for Hungarian. Even Chinese with its famously weak word order gets 19.7%. The VMS gets 8.85%. Knowing the previous word tells you almost nothing about the next word.

Repeated bigrams and trigrams exist in the VMS — but they behave differently from phrases in meaningful text. They use variable word order, they cluster by word similarity rather than by grammatical function, and they provide almost no predictive power for the following word.

This is this reason Tiltman and D'Imperio wrote about the VMS:
"Languages simply do not behave in this way." ... "And yet I am not aware of any long repetitions of more than 2 or 3 words in succession, as might be expected for instance in the text under the botanical drawings". (You are not allowed to view links. Register or Login to view. p.9)

"The short words, the many sequential repetitions, the rarity of one- or two letter words, the rarity of doublets (doubled letters), all militate against simple substitution. So also does the strange lack for parallel context surrounding different occurrences of the same word as shown by words indexes. In the words of several researchers 'the text just doesn't act like natural language'." (You are not allowed to view links. Register or Login to view. p. 30)