Automated Topic Analysis of the Voynich Manuscript

[quimqu]

Hello, Obelus,

thank you for reading my posts and arise your comments.

a quantitative measure of effect size might help;  conventional for these data would be "Cramér's V."

I should say that one of my first metrics for calculating the optimal number of topics was Cramer's V. At that time, I didn't distinguish between language A's hands and language B's hands, i just focused in languages and hands. The results were here. You can see that for hands best topic number was 3, but I wanted more granularity, and 11 can also be OK (it is just after the bump), for languages, k=11 can also be OK (it is where the curve flattens):



Nevertheless, I calculated the Cramer's V of both contingency tables (Language A's hands vs topics and Language B's hands vs. topics). Results are surprisingly high for language A's hands vs. topics: 0,830 and not so good for language B's hands vs. topics: 0,264 (scale is from 0 to 1). Language a's hands vs. topics have an almost perfect association.

How was the text sample partitioned for classification?  The number of text blocks tagged as paragraphs in the RF transliteration is less than 300.

Here is a summary of the steps I followed to create the paragraphs:

• In the file ZL3b-n.txt (EVA transliteration), the start of a paragraph is marked with <%> and the end with <$>.
  
• I joined the marked paragraphs into single lines.
  
• The remaining lines (those outside of marked paragraphs) were left as they were. So in the end, I had a mix of paragraph lines and other types of content (e.g., labels or short sentences from astrological folios).
  
• I removed lines that contained only 1 or 2 "words", and repeated the process as many times as necessary to ensure that, for example, a line originally containing 3 words wouldn't leave behind a line with just 1 word after cleaning.
  
• At first, I considered removing the most common "words" for topic detection. However, I decided not to, since even if we suspect that words like daiin are not meaningful, we can't be completely sure—so I didn't remove any words at all.
  
• This resulted in a total of 891 "paragraphs".
  

If you have any further questions, please feel free to ask!