How LLM models try to understand Voynichese

[Jorge_Stolfi]

To train any word predictor that uses the last K words, you need a corpus in which all possible sequences of K words have a good chance of appearing.  If the lexicon has1000 equally likely words, the corpus needed for that training must have on the order of 1000 to the power K words. 

You may get more interesting results if you first try to partition the lexicon into a small number -- say 20 -- sets of words that are similar in their immediate context (1-2 words before and/or a after at most). Then you map each word in the corpus to the index of its subset, so the corpus becomes a "text" with the same length but whose vocabulary is the numbers 1 to 20.   Then you train a predictor to use the last 4 words of that text. For that, a corpus with on the order of 20^4 = 160000 words should be adequate. Then you take the output of that predictor and replace each index by a random word in the corresponding set, with the appropriate frequencies.  

You could  complicate this last step by making the choice of a word from each set depend on the previous 2-3 generated indices, or the previous generated word.  (But not on the last 4 indices, because then you would run into the same probele of insufficient examples to build the necesary probability tables.)

And anyway you should definitely map similar glyphs to the same glyph: a->o, r->s , ch->ee, etc. Anyone who tried to transcribe a significant amount of Voynichese surely must have concluded that 10% or more of those letters are just wrong -- the transcription says "o" when the Author meant "a", etc.

All the best, --jorge