The structure of the Voynich text and how it may be generated

[quimqu]

I run some more tests and models today:

Category	Finding	What it suggests
Robust	Local similarity is higher than in global and positional shuffles (across multiple similarity measures)	There is a real local constraint structure, not an artefact of one metric
Robust	Best match comes from the local window, not from the previous token (2.19 vs 3.89)	The system behaves like a local pool, not a Markov chain
Robust	Past vs future is symmetric (difference ≈ 0)	No strong directional generation signal
Robust	Within-line shuffle preserves most of the signal	Order inside the line is less important than the set of nearby tokens
Representation-dependent	Character windows (char4, char5) still show structure	The effect is not purely due to spaces or tokenization
Representation-dependent	Token bigrams destroy the signal	Not every segmentation preserves the structure
Weak effects	Middle of line slightly more similar and dense than start/end	There is positional influence, but it is limited
Weak effects	Candidate pool size is stable (~19) across positions	No strong evidence that position changes the size of the allowed set
Model limitation	Local model overshoots similarity (too low Lev, too high density)	Simple local reuse is too strong and too compact

The text behaves like it is generated from a local set of compatible forms. What matters is the neighbourhood, not the exact order and not just the previous word. This is why bag-of-context works much better than sequential dependence.

At the same time, the system is not trivial. The signal survives different similarity definitions and even different segmentations, but not all of them. So it is not just noise or a simple reshuffling effect.

The main limitation of the current models is that they are too “tight”. They create clusters that are more coherent than in the real text. The Voynich seems to operate in a looser space, where many forms are allowed but still constrained.

So the useful takeaway is not that I have a good generator yet (?), but that the tests are narrowing the space of plausible mechanisms.