sh_ and ch_ compose the same words

[farmerjohn]

Additionally I have calculated the token frequencies for all 612 ch/sh word pairs for all folios. The resulting correlation coefficient is +0.55. To make comparison easier I have also calculated the correlation coefficient for the word frequencies for all 612 ch/sh word pairs for the whole MS. This correlation coefficient is +0.93.

Pearson's Correlation(count(chWords@folios),count(shWords@folios)): +0.55 (n=225 folios*612 word pairs=137700,p=too close to zero to calculate)


Pearson's Correlation(count(chWords@VMS), count(shWords@VMS)): +0.93 (n=612 word pairs,p=4.54E-268)

Hi, Torsten.

I have some doubts about using correlation in these two cases.
To my mind correlation is applicable when we have some repeatable process (with two outputs) and each new iteration is independent of the previous one. These properties must hold: we can freely add measurements; or can take a subset of existing measurements - correlation coefficient in all cases will be the approximately same, just more or less precise. In simple words, correlation value can be calculated at any time from any place in any direction. So calculating "per page" is acceptable here, and new pages would be only to the good here.

But when we calculate "per words" we not only manually have chosen words, but also the only new measurements to add are {0, 0} (and all of them are absolutely legible: EVA-sheaktoiiin/cheaktoiiin are also words, why not to include them?). So there the resulting value does depend heavily on the selected data.