06-04-2026, 10:12 AM
From my point of view, these experiments would suggest two different things, and it is important to keep them separate.
On the one hand, some patterns seem relatively robust. The Voynich text would appear to combine very low exact repetition with clear local similarity. Words would tend to resemble nearby words, but not in a strictly sequential or chain-like way. In that sense, models based on global sampling or simple copying from the previous token would not be sufficient, while a model based on selecting from a small local set of compatible forms would seem to reproduce these properties more naturally.
On the other hand, I am aware that part of the structure we observe may come from the assumptions we impose. What counts as a “word” depends on the transliteration and the use of spaces. Similarity is measured here using edit distance, which is only one possible definition. The notion of “families” is also introduced by the model and may not correspond to real discrete units in the manuscript. Different choices in these aspects could lead to different results.
So the model would mainly be useful in a limited sense. It would help rule out simpler mechanisms and suggest that local constraints are important. But it would not justify stronger claims about the exact generative process or the existence of well-defined families. Some of that apparent structure could still be an artifact of how the text is represented and analyzed
On the one hand, some patterns seem relatively robust. The Voynich text would appear to combine very low exact repetition with clear local similarity. Words would tend to resemble nearby words, but not in a strictly sequential or chain-like way. In that sense, models based on global sampling or simple copying from the previous token would not be sufficient, while a model based on selecting from a small local set of compatible forms would seem to reproduce these properties more naturally.
On the other hand, I am aware that part of the structure we observe may come from the assumptions we impose. What counts as a “word” depends on the transliteration and the use of spaces. Similarity is measured here using edit distance, which is only one possible definition. The notion of “families” is also introduced by the model and may not correspond to real discrete units in the manuscript. Different choices in these aspects could lead to different results.
So the model would mainly be useful in a limited sense. It would help rule out simpler mechanisms and suggest that local constraints are important. But it would not justify stronger claims about the exact generative process or the existence of well-defined families. Some of that apparent structure could still be an artifact of how the text is represented and analyzed