Automated Topic Analysis of the Voynich Manuscript

[quimqu]

If you want to analyze a given Naibbe ciphertext as if it were a synthetic Voynich B, divide each ciphertext into four equal portions, aka each ~5000-5500 tokens long, and then subdivide from there. Each fourth will roughly correspond with one of the original plaintext sections. There are no exact equivalents to folios in these ciphertexts, but you could explore the statistical effect of smaller subdivisions by treating each ciphertext as if it were a corpus of N different documents each one roughly (total/N) tokens long, just as you have been doing with the various folios of the VMS.

Hello Magnesium,

I ran a topic detection experiment using NMF on the file /kaggle/input/voynich/naibbe_Cleaned_52_01_10_word_lines.txt and wanted to share some observations.

Paragraph Structure: I grouped the text into paragraphs of 10 lines each. That is, every 10 lines were concatenated into a single paragraph for analysis.

Optimal Number of Topics: The optimal number of topics identified by the model was 6. Here's the topic distribution across paragraphs:




Since I’m not entirely familiar with the content structure of the Naibbe cipher, I can’t fully assess whether the topics align with actual semantic groupings. But here’s something I found interesting:

I generated topic–paragraph heatmaps to understand how distinctive each topic is. Here's how to read them:

• Y-axis: Paragraph index (each paragraph corresponds to 10 lines of the original text).
  
• X-axis: Topic number.
  
• Cell value: For each topic, I selected the high-weight words (those above the mean + 3×std). Then, for each paragraph, I counted how many of these words appear. This helps visualize whether a paragraph is dominated by its assigned topic or shares vocabulary with others.
  

That said, for the MS topics, the heatmaps look as follows:



As you can see, the topics are quite well-separated. Most paragraphs are dominated by the vocabulary of a single topic, with limited overlap. This suggests that the MS text, even if undeciphered, has internal structure that the model can detect.

But for the Naibbe cipher, the heatmaps look as follows:



In contrast, the Naibbe cipher shows significant vocabulary overlap between topics. Many paragraphs contain high-weight words from multiple topics, making them harder to distinguish. This suggests that the output from the Naibbe cipher is relatively homogeneous across the text and topics.

Interpretation

• The MS text appears to be more topically diverse and structured, which supports the idea that it contains meaningful internal variation (even if we don’t understand the language).
  
• The Naibbe cipher output seems too uniform across paragraphs, which might indicate it's closer to random output or lacks internal semantic variation — at least from the point of view of a topic model.
  

What do you think of these results?