These are the results of a page-by-page analysis. As always, I might have made errors in the process.
I normalized page measures according to the process You are not allowed to view links. Register or Login to view.: the measure is the difference with the expected number of occurrences based on the overall average measures and the number of word-couples in each page. A page that has exactly the expected number of Perfect-Reduplication and Quasi-Reduplication will be plotted at 0,0.
I have also processed individual sections. I used the Image classification in the ZL transliteration file: a few text-only pages are not classified and are not coloured in the plot (there only are a handful of these, so they do not significantly affect the results).
Theis table summarizes the results and lists the colours used in the attached plot:
__ALL__ pgs:207 corr: 0.338 p-val:0.0000006
------------------
HerbalA pgs:95 corr:-0.039 p-val:0.7046817 Blue
Pharma_ pgs:16 corr: 0.252 p-val:0.3464567 Orange
AstroCZ pgs:15 corr:-0.225 p-val:0.4199572 Purple
HerbalB pgs:32 corr: 0.311 p-val:0.0833608 Red
StarQ20 pgs:23 corr: 0.673 p-val:0.0004313 Yellow
Bio_Q13 pgs:20 corr:-0.137 p-val:0.5657872 Green
As can be seen, the overall correlation is not very high (0.34), but the number of samples is large enough to cause a quite low p-value. Here I have included the 19 pages that have a 0 count for both measures. If I remove them, I get correlation: 0.321, p-value:0.000007.
Since correlation is regarded as significant when p-value<0.05, these results confirm that the two phenomena are linked in some unknown way.
Some sections have correlations close to zero (HerbalA,AstroCosmo,Q13). All have high p-values, with the important exception of the Stars-Q20 section, with a 0.0004 p-value. While (Q20 aside) pages inside a section do not show much correlation between Perfect ant Quasi reduplication, the average values for the individual sections are different and result in a significant correlation (see You are not allowed to view links. Register or Login to view.). The overall result then appears to be due to two factors:
I normalized page measures according to the process You are not allowed to view links. Register or Login to view.: the measure is the difference with the expected number of occurrences based on the overall average measures and the number of word-couples in each page. A page that has exactly the expected number of Perfect-Reduplication and Quasi-Reduplication will be plotted at 0,0.
I have also processed individual sections. I used the Image classification in the ZL transliteration file: a few text-only pages are not classified and are not coloured in the plot (there only are a handful of these, so they do not significantly affect the results).
Theis table summarizes the results and lists the colours used in the attached plot:
__ALL__ pgs:207 corr: 0.338 p-val:0.0000006
------------------
HerbalA pgs:95 corr:-0.039 p-val:0.7046817 Blue
Pharma_ pgs:16 corr: 0.252 p-val:0.3464567 Orange
AstroCZ pgs:15 corr:-0.225 p-val:0.4199572 Purple
HerbalB pgs:32 corr: 0.311 p-val:0.0833608 Red
StarQ20 pgs:23 corr: 0.673 p-val:0.0004313 Yellow
Bio_Q13 pgs:20 corr:-0.137 p-val:0.5657872 Green
As can be seen, the overall correlation is not very high (0.34), but the number of samples is large enough to cause a quite low p-value. Here I have included the 19 pages that have a 0 count for both measures. If I remove them, I get correlation: 0.321, p-value:0.000007.
Since correlation is regarded as significant when p-value<0.05, these results confirm that the two phenomena are linked in some unknown way.
Some sections have correlations close to zero (HerbalA,AstroCosmo,Q13). All have high p-values, with the important exception of the Stars-Q20 section, with a 0.0004 p-value. While (Q20 aside) pages inside a section do not show much correlation between Perfect ant Quasi reduplication, the average values for the individual sections are different and result in a significant correlation (see You are not allowed to view links. Register or Login to view.). The overall result then appears to be due to two factors:
- different behaviour of the individual sections; this is similar to what Rene observed when discussing the You are not allowed to view links. Register or Login to view.;
- page-by-page correlation in Q20. This also shows in the attached plot, where the yellow squares spread diagonally from bottom-left to top-right.