How LLM models try to understand Voynichese

[quimqu]

Am I reading this correctly? It looks like GPT cannot successfully predict the fifth word knowing the previous 4 words. Correct predictions seem very rare and are the most common words like "daiin" or "chol", so this looks like pure chance to me.

Yes, that's absolutely right — this also happens when I train a GPT model from scratch using just a single English book. It's expected: the corpus is extremely small, and unfortunately, that's all we have for the Voynich Manuscript.
This project was never intended to produce a model that "understands" Voynichese — training a language model on a single book is not enough to learn a true grammar or semantics. However, I believe that even the limited learning the model does achieve may offer insights into the internal structure or patterns of the language.
That was the sole aim of this work: not to prove comprehension, but to explore which elements the model finds predictable, and how those predictions are distributed across the text.

But for example, it's interesting to see that at the beginning of the Voynich, the model (although mostly wrong) tends to predict common tokens like daiin, chol, etc. Later in the manuscript, it shifts toward predicting more qokeedy, qokain, and similar forms. So even though the predictions are mostly incorrect, the model’s behavior seems to depend on the region of the Voynich it is processing.