Hi Darrin,
Your pre-processing drops some important accentuated characters (like "é") in the French file. Also it would be better to use a space to replace "'" and "-" than remove them, because they are separating words. Recent French news are much less interesting than Middle French texts for a comparison to a transliteration of Voynichese or a transformation thereof (which transformation works best is the question that matters). There is the same anachronism problem for other languages as well. Julius Caesar's Latin is not late medieval Latin, not only because the vocabulary is different, but because there are some systematic uses that impact letter n-gram statistics ("ae" and "oe" were simplified to "e", "ti" was often indistinguishable from "ci", "ihi" was often writen "ichi", etc.). Also there are huge differences in statistics between different texts/authors even when they use the same language, so a cloud of points (one point per sample text) would be far better than a single point on the graph.
(12-08-2020, 01:46 PM)dvallis wrote: You are not allowed to view links. Register or Login to view.All the source data files are at:
You are not allowed to view links. Register or Login to view.
Your pre-processing drops some important accentuated characters (like "é") in the French file. Also it would be better to use a space to replace "'" and "-" than remove them, because they are separating words. Recent French news are much less interesting than Middle French texts for a comparison to a transliteration of Voynichese or a transformation thereof (which transformation works best is the question that matters). There is the same anachronism problem for other languages as well. Julius Caesar's Latin is not late medieval Latin, not only because the vocabulary is different, but because there are some systematic uses that impact letter n-gram statistics ("ae" and "oe" were simplified to "e", "ti" was often indistinguishable from "ci", "ihi" was often writen "ichi", etc.). Also there are huge differences in statistics between different texts/authors even when they use the same language, so a cloud of points (one point per sample text) would be far better than a single point on the graph.


![[+]](https://www.voynich.ninja/images/collapse_collapsed.png)