While I understand the usefulness of percentages in removing the spurious correlation induced by varying page-lengths, I second Torsten's question. I am also interested in understanding what can be the meaning of overall correlation being greater than that of the individual sections.
I have tried computing page-percentage plots and correlation measures for a few more couples of ch- / sh- words. The plot I get for chedy / shedy seems to me reasonably close (though not identical) to the one computed by Rene on the basis of Nablator's spreadsheet. I have included {0,0} pages. The red line is y=x.
As always, it is possible I have made errors somewhere.
These are the plots for the next four more frequent ch- words vs their sh- counterparts.
These are correlation values on the whole manuscript and on various subsections. I extracted subsections using the ivtff +I parameter. It seems that what I get is not always identical to the classification used by Rene. For instance, for Cosmo chedy/shedy, I also find matches in 68v3 and 70r2, while 85r2 (having no illustrations) is classified as "Text".
In general, it seems that the overall correlation is rather low. Per-section correlation also tends to be low, but there are a couple of possibly interesting exceptions, e.g. chey/shey in Stars. It seems clear that, with increasingly rarer words, data points tend to cluster on the two axis, with one of the two counts=0. Could it be that, for words which only have 1 or 2 average occurrences per page, it woud be more informative to measure folio or bi-folio percentages?
I have tried computing page-percentage plots and correlation measures for a few more couples of ch- / sh- words. The plot I get for chedy / shedy seems to me reasonably close (though not identical) to the one computed by Rene on the basis of Nablator's spreadsheet. I have included {0,0} pages. The red line is y=x.
As always, it is possible I have made errors somewhere.
These are the plots for the next four more frequent ch- words vs their sh- counterparts.
These are correlation values on the whole manuscript and on various subsections. I extracted subsections using the ivtff +I parameter. It seems that what I get is not always identical to the classification used by Rene. For instance, for Cosmo chedy/shedy, I also find matches in 68v3 and 70r2, while 85r2 (having no illustrations) is classified as "Text".
In general, it seems that the overall correlation is rather low. Per-section correlation also tends to be low, but there are a couple of possibly interesting exceptions, e.g. chey/shey in Stars. It seems clear that, with increasingly rarer words, data points tend to cluster on the two axis, with one of the two counts=0. Could it be that, for words which only have 1 or 2 average occurrences per page, it woud be more informative to measure folio or bi-folio percentages?
| N.Pages | Xhedy | Xhol | Xhey | Xhor | Xheey | |
| _ALL | 225 | 0.771 | 0.497 | 0.220 | 0.229 | 0.011 |
| HerbalA | 95 | - | 0.388 | 0.148 | -0.007 | -0.045 |
| HerbalB | 32 | 0.414 | 0.220 | -0.102 | -0.183 | 0.010 |
| Stars | 23 | 0.120 | -0.051 | 0.522 | 0.214 | -0.124 |
| Bio | 20 | 0.687 | 0.334 | 0.253 | -0.093 | 0.346 |
| Pharma | 16 | -0.066 | 0.513 | -0.107 | 0.209 | -0.446 |
| Cosmo | 12 | 0.647 | -0.124 | 0.349 | -0.079 | -0.217 |