The structure of the Voynich text and how it may be generated

[quimqu]

Since the last experiment, I moved away from purely sequential models and tried a different approach based on local compatibility rather than direct copying. Instead of generating each token from the previous one, the model builds a local pool of compatible forms and selects from it, with a weak bias from recent context.

Aspect	Earlier models	New local compatibility model	Remaining gap
Generation mechanism	Sequential or global sampling	Selection from local compatible pool	Internal structure of the pool still unclear
Exact repetition	Too high or unstable	Very close to real text	Largely solved
Local similarity	Either too weak or too forced	Close to real behavior	Slightly too diffuse
Vocabulary control	Collapse or explosion	Balanced regime	Still somewhat smaller than real
Family persistence	Too strong (chaining)	Close to real low persistence	Mostly solved
Number of active families	Very low	Moderate increase	Still far below real text
Family distribution	Highly concentrated	Less dominated by a single family	Entropy still too low
Within-family coherence	Not well modeled	Partially captured	Families too loose internally
Overall conclusion	Sequential rules insufficient	Local selection works better	Need better structure of overlapping families

This changes the behavior quite a lot. The model no longer gets stuck repeating the same forms, and exact repetition is now very close to the real text. It also avoids the opposite problem of exploding vocabulary. In that sense, the global balance is much better than in the earlier models.

More importantly, it reproduces the idea that tokens are chosen from a local set of similar options rather than derived step by step from a single source. This matches earlier observations that unordered local context works as well as, or better than, strictly sequential context.
However, new limitations appear. The model still uses fewer active families than the real text and distributes probability too unevenly across them. At the same time, tokens within a family are not as tightly related as in the manuscript, and local similarity is slightly too diffuse.

So, it is possible to reproduce repetition, local similarity and entropy at the same time, but only if generation is treated as selection from a constrained local space. What is still missing is how that space is structured internally, in particular how many overlapping groups remain active and how tightly they are defined.