Why is there even a Voynich B?

[quimqu]

You are not allowed to view links. Register or Login to view. are four texts that could be used for that purpose.  The files are in UTF-8 encoding. They are non-overlapping extracts from the same novel, with three different spelling systems and translated into a different but closely related language.  Parags are separated by blank lines.  More details are in You are not allowed to view links. Register or Login to view.. For better comparison with Voynichese, you may want to ignore all lines starting with "#", delete the string "\emph", map everything to lower case, map all letters with diacritics to ASCII letters without them, and replace all punctuation by spaces.  You may also want to delete all parags that are too short (mostly dialog lines).

I will give it a try and let you know.

Hi Jorge,
I ran NMF topic modeling on the four Dom Casmurro files (DC1–DC4). Each file corresponds to a different version of the same novel:

• DC1 is the original 1899 Portuguese spelling,
  
• DC2 is the modern 1999 spelling,
  
• DC3 is a phonetic respelling of Portuguese,
  
• DC4 is the modern Spanish translation.
  

The NMF results show a very strong match between the learned topics and the file sections, with the best performance at K = 3 (AMI = 0.81, p ≈ 0). This means that the model clearly identifies three main linguistic clusters across the corpus. Given the setup, that outcome makes perfect sense: the two Portuguese spellings are grouped togethr, the phonetic respelling is separated apart, and the Spanish text stands apart also as its own distinct group.

As the number of topics increases, the AMI drops steadily, showing that three components are enough to capture the main differences between these texts. In other words, NMF is not detecting narrative themes but rather the orthographic and linguistic variation among the Portuguese and Spanish versions of Dom Casmurro. You can see here a comparison for two modles, NMF and LDA. NMF detects clearly the three different languages:

   

If we color each found topic and each paragraph to its asigned topic color, we see following (paragraphs ordered DC1, DC2, DC3 and DC4 from top to bottom):

   

You can see that a small part pof the paragraphs is given a wrong topic, but I asume this can happen due to the proximity of the three languages.

Finally, here you can see the top weighted words per topic that the model has found:

=== Main topic 0 ===
? Dialect (main topic words):
nao, ci, os, la, um, di, mi, nau, con, un, el, do, com, si, en, em, ao, que, las, lhe, los, us, du, para, lhi, pra, olhos, in, as, pero, me, mas, mae, su, au, disse, os olhos, dos, menti, voce

=== Main topic 1 ===
? Dialect (main topic words):
ci, di, nau, de, que, me, us, du, pra, lhi, la, in, nao, si, mais, se, no, au, menti, comu, os, um, eli, nu, el, por, para, min, mai, mas, pur, en, como, joze, padri, do, com, veis, ci nau, tamben

=== Main topic 2 ===
? Dialect (main topic words):
la, el, en, los, las, no, de, lo, pero, su, una, del, de la, le, al, es, sus, en la, con, por, madre, fue, en el, de mi, yo, mi madre, que, ojos, de los, ci, solo, habia, ni, tenia, nao, un, mi, despues, sin, lo que

Note that I deleted the paragraphs with less than 5 words.