10-10-2025, 09:26 PM
The thing that makes PCA based cluster analysis (in particular) of sub-dialects tricky is that
* on the one hand we don't want to beg the question by bringing too much prior knowledge to bear in doing the analysis, while
* on the other hand, we *know* that the A-B split is the dominant souce of overall variance, meaning that if you include Q13 (in particular) and the Herbal B folios (to a lesser extent) then PCA will be forced to pick a dominant eigenvector that captures as much of that variance as possible, forcing all subsequent eigenfeatures to be orthogonal to that axis -- which is not necessarily going to produce good axes to tease apart other substructure.
While I've gotten around to running PCA-based k-means clustering on the running text of all the pages in the Mss. to compare with Rene's clustering results, I haven't had a chance to run it excluding Q13 or Q13 + Herbal B pages. That's definitely on my queue to see if that increases the separation between the A language "dialects".
* on the one hand we don't want to beg the question by bringing too much prior knowledge to bear in doing the analysis, while
* on the other hand, we *know* that the A-B split is the dominant souce of overall variance, meaning that if you include Q13 (in particular) and the Herbal B folios (to a lesser extent) then PCA will be forced to pick a dominant eigenvector that captures as much of that variance as possible, forcing all subsequent eigenfeatures to be orthogonal to that axis -- which is not necessarily going to produce good axes to tease apart other substructure.
While I've gotten around to running PCA-based k-means clustering on the running text of all the pages in the Mss. to compare with Rene's clustering results, I haven't had a chance to run it excluding Q13 or Q13 + Herbal B pages. That's definitely on my queue to see if that increases the separation between the A language "dialects".