01-09-2021, 12:48 PM
Suggestion accepted!
The "rightwardness" and "downwardness" measures are dimensionless coordinates that allow us to collate position information from multiple paragraphs... in search of layout-level patterns, I suppose. Perhaps such data could be represented directly as two-dimensional heat maps or histograms. In computing the example graphics below, I have ignored interesting tweaks and ramifications discussed in pfeaster's posts. They can easily be incorporated if it seems worthwhile.
Given some string-pattern of interest, let us compute the rightwardness and downwardness values for every match found in a text sample. The result is a list of {rightwardness, downwardness} ordered pairs that are coordinates in the dimensionless paragraph-space. We can immediately bin these string-pattern matches and plot them as a two-dimensional histogram. For example, here are counts of words beginning with "qo" in all of the 'paragraph'-style text, represented as grey level. Sisu the cat chose 10 word bins and 5 line bins by walking across my keyboard:
Following practice in this thread, the counts can be scaled by the total number of words in each bin to yield fractional populations:
Tabulating only rightwardness amounts to using only one downwardness bin:
It is characteristic of such data displays that patterns are visible, but quantitative values are difficult to extract. Here the fractional populations from the previous one-dimensional histogram are shown on Cartesian axes (orange), together with analogous results for words beginning with "o" (blue):
Thus the plots posted by pfeaster and MarcoP are satisfactorily reproduced, so there is reason to believe that we are on the same page computationally; remaining differences may be due to the ivtt extract used.
Finally, we can plot a two-dimensional histogram of the ("o"-initial)÷("qo"-initial) ratio:
The known rightward and downward trend is visible.
No specific question is addressed by these particular graphics, but the general form may be a natural way of representing paragraph-level structure. Binning artifacts discussed earlier in the thread are naturally still present.
The "rightwardness" and "downwardness" measures are dimensionless coordinates that allow us to collate position information from multiple paragraphs... in search of layout-level patterns, I suppose. Perhaps such data could be represented directly as two-dimensional heat maps or histograms. In computing the example graphics below, I have ignored interesting tweaks and ramifications discussed in pfeaster's posts. They can easily be incorporated if it seems worthwhile.
Given some string-pattern of interest, let us compute the rightwardness and downwardness values for every match found in a text sample. The result is a list of {rightwardness, downwardness} ordered pairs that are coordinates in the dimensionless paragraph-space. We can immediately bin these string-pattern matches and plot them as a two-dimensional histogram. For example, here are counts of words beginning with "qo" in all of the 'paragraph'-style text, represented as grey level. Sisu the cat chose 10 word bins and 5 line bins by walking across my keyboard:
Following practice in this thread, the counts can be scaled by the total number of words in each bin to yield fractional populations:
Tabulating only rightwardness amounts to using only one downwardness bin:
It is characteristic of such data displays that patterns are visible, but quantitative values are difficult to extract. Here the fractional populations from the previous one-dimensional histogram are shown on Cartesian axes (orange), together with analogous results for words beginning with "o" (blue):
Thus the plots posted by pfeaster and MarcoP are satisfactorily reproduced, so there is reason to believe that we are on the same page computationally; remaining differences may be due to the ivtt extract used.
Finally, we can plot a two-dimensional histogram of the ("o"-initial)÷("qo"-initial) ratio:
The known rightward and downward trend is visible.
No specific question is addressed by these particular graphics, but the general form may be a natural way of representing paragraph-level structure. Binning artifacts discussed earlier in the thread are naturally still present.
![[+]](https://www.voynich.ninja/images/collapse_collapsed.png)