Automated Topic Analysis of the Voynich Manuscript

[quimqu]

The UMAP projection below

Could you please explain what are the two coordinates?

All the best, --stolfi

(Since now there is another "jorge" posting to this forum, I should sign "stolfi" from now on...)

Hi (Jorge) Stolfi, I'll try to explain UMAP coordinates.

UMAP is a way to turn complex data into a 2D picture so we can see patterns. In this case, each paragraph is first turned into a set of numbers (a vector) that describe how often different words appear together, sort of its fingerprint. Those fingerprints live in a huge mathematical space (maybe hundreds of dimensions).

UMAP takes all that information and squeezes it down to two coordinates, X and Y, in a way that tries to keep similar things close together and different things far apart. So, points that are close on the map have similar word patterns and points that are far apart use different kinds of words or structures.

The X and Y values themselves don’t have meaning like time or frequency. They’re just the result of UMAP arranging all the texts so that their distances reflect how similar their word patterns are.

And now, the strange thing about this UMAP. I have experimented the same models with headlines about crimes and sports. I got an accuracy of about 85%, with 300 headlines of each topic, which is quite good. But when I plotted its UMAP...

   

Well this is a typical UMAP, the one the the data scientists use to find when working with clusters and so on. Look how different the UMAP of the Voynich (at paragraph level) is!!!

   

In the Voynich UMAP plot, the paragraphs don’t form clouds or clusters as in normal texts, they align along narrow, continuous lines. This means that each paragraph is very similar to its neighbors (the previous and the next ones in UMAP space "line"), but much less similar to the rest. In other words, the text seems to follow a very regular pattern, where nearby paragraphs (in terms of UMAP) share nearly the same word combinations. This is amazing.

The logic tells me that if I take a paragraph, there must be two very similar paragraphs, but no more! I must admit I had to search and use some GPT help to undestand what is happening here, because I have never seen this. After explainin the plots and showing the plots, the output was:

That linear structure can have several logical explanations:

1. Strong local similarity – each paragraph is extremely similar to the next one, as if the text evolved by small gradual changes rather than introducing new combinations of words.
   → This produces a “chain” effect in UMAP: each paragraph links to its immediate neighbors.
   
2. Limited combinatorial diversity – if the text is generated by a small set of rules or templates (for example, repeating prefixes, suffixes, or word frames), the possible variations are restricted.
   → The UMAP projection then collapses those patterns into narrow linear tracks instead of broad clusters.
   
3. Sequential dependency – the text might have been produced in sequence, reusing nearby words or structures (like a Markov process), so neighboring lines are statistically dependent.
   → This continuity is reflected as “lines” in topic space.
   
4. Lack of thematic jumps – natural language paragraphs shift topics often; the Voynich does not.
   → Without those thematic jumps, UMAP finds no reason to break the flow into separate groups.