"The Currier languages revisited" revisited

[kckluge]

then I see no way to meaningfully interpret the results of PCA at all.

If we knew nothing else about the pages in question -- if they were just text -- I'd agree with you (oshfdk). We do know more, however, because of the illustrations. The relative bigram frequency statistics (which PCA is just a method for visualizing) tell us a couple things:

* the qualitative differences in illustration type correspond in most cases to quantitatively distinct clusters in that space (the cosmological and zodiac page non-label text clusters together, the "Rose" sheet pages cluster in with Herbal B, and the Pharma pages cluster together in an interesting way with a subset of the Herbal A pages), and 

* within the set of pages with a single large plant drawing on them, the text falls into three quantitatively distinct clusters, (Herbal A, low folio numbers), (Herbal B), and (Herbal A, high folio numbers), and

* the cluster containing the high folio number Herbal A pages also contains the Pharma pages, which are also physically close to them in the binding.

Those aren't minor things to know, they constrain possible explanations of the text in substantial non-trivial ways.

* Chances of getting a useful insight from these analyses will improve if one uses fewer data so as to reduce the number of factors that affect them. Like comparing Herbal-A and Herbal-B only, thus hopefully eliminating the "topic" factor.  Then maybe one can figure out whether the difference is a change of spelling, or something else.  Once one gets some insight on Herbal-A vs. Herbal-B, one can then consider what is happening in other sections.

When using PCA to visualize things, absolutely. Post #5 shows this with plotting just the A language sections and just the B language sections.

* It is not surprising that word frequencies are different in each section.  Even for the most common words, which may or may not be "function" words like "much", "is", "find", "good"; not to mention "content" words like "herb", "star", "blood", etc.  

* If word frequencies change, digraph frequencies will change too, since they are determined by the digraphs that appear in the most common words.  As I mentioned before, "rb" is probably much more frequent in an herbal text, (Latin or English) than in a text about astrology.  (Unless the latter it talks a lot about "orbits"...)

* What is surprising is that (IIUC) Herbal-A differs from Herbal-B noticeably more than either differs from Bio or Stars. Thus, besides the difference of topic, we indeed have a difference of language or spelling (or encryption).  Maybe the texts of Herbal-A and Herbal-B were taken from sources in two different dialects, or two very similar languages.  Like Northwest Lower West Bavarian and Southwest Lower West Bavarian...

[...]

* And the chances of obtaining useful insights will improve a lot if one uses the good old "scientific method": make an hypothesis, then devise the simplest and most effective way to test for it, and do that.

Al the best, --stolfi

...and the way to do that is to take (say) a half dozen languages from the early 15th century, and within each language take a set of samples with different authors and subjects (ideally the cross product of both of those), and then look at how the quantitative differences in relative bigram frequency space between those factors compare to the quantitative differences between sections in the Voynich Mss. That can be done by

* looking at the frequency data as points in an N-dimensional Euclidean space and using PCA and/or cluster analysis to visualize their distribution

* looking at them as vectors in an N-dimenstional space and using cosine similarity, using heat maps or single-linkage hierarchical clustering to visualize their distribution

* Looking at them as discrete probability distributions and using a Chi^2 value to measure similarity, visualizing as with cosine similarity

I haven't done that; I don't think anyone else has done that. If someone does, I'm very confident where I'm betting my chips (and I don't think it's the same place you'd bet yours).