GPT Models Fail to Find Language Structure in the Voynich Manuscript

[quimqu]

Hmm. Your range for regular language bigrams spans two orders of magnitude, from 24.6 to 3.4. is Voynichese's 1.9 then really an outlier?

Yes, that’s a fair question — and in fact I already hint at it in the paragraph about the Latin religious text:

> “The Latin religious text In Psalmum David CXVIII had pretty low bigram and trigram scores — not too far from the Voynich numbers. This could be because of its complex sentence structure or how rarely some word combinations repeat. But even then, it still produced some consistent word sequences, which the GPT picked up.”

I was actually surprised myself by how low the score was for the Psalmum text. It made me wonder whether Voynichese might somehow resemble this kind of structure. Still, there are a few things to consider:

The Psalmum GPT did manage to reproduce some consistent multi-word patterns, even if rarely.

The Voynich GPT didn’t manage to regenerate any trigrams at all — not even once.

And perhaps most striking: the other Latin text I used, De Docta Ignorantia, gave scores much more in line with natural language.


So while Psalmum and Voynichese are relatively close in absolute numbers, there still seems to be a meaningful difference in behavior — especially in terms of sequential co-occurrence.

That said, this is definitely one of the grey areas that calls for further digging — maybe through more controlled training setups or different tokenization strategies (e.g., syllables vs. words). I’d love to explore this further.

---

I wonder if there is any correlation with the number of repeating bi/trigrams in each corpus text

My assumption was that training on 11k tokens would give a good sense of the model's ability to learn and reproduce frequent patterns, but you're right — if some texts naturally have fewer internal repetitions, then we should expect lower match rates in generation, even if the language is "natural."

It would be interesting to calculate how many unique bigrams/trigrams appear more than once in each corpus — and then compare that to the generation match rate. If there's a strong correlation, it could help distinguish between data sparsity vs. structural absence of co-occurrence.

I’ll look into it!