[split] What can the structural peculiarities of the VMS tell us about the nature of the underlying text

[Jorge_Stolfi]

words that begin with qo are more common at line-start than elsewhere.  Now we know that one cause of that anomaly has nothing to do with those characters: it is just the bias towards longer words in that position that is created by the process of breaking text into right-justified lines -- and it so happens that qo-words are longer than average.  (This side effect of line breaking may not be enough to explain all the anomalies of qo-words; but it is still not known where there is anything beyond it.)

We do not know this.

We do know that the mere act of formatting a text into paragraphs enhances the frequency at line-start of words that are longer than average; and, therefore, depresses the frequency of words that are shorter than average.  

This line-breaking length bias (LBB) is not a theory; it is predicted by math and has been verified experimentally.  It is also independent of the language, topic, spelling system, encoding, etc.  And of the characters in those words.  (Provided that the decision to break is based on the physical distance between the text rails and the physical length of the next word; or at least on their character counts -- not on some fixed max count of words per line.)

Therefore, if qo-words are indeed longer than average and increasingly common at line-start, the LBB is (as I wrote ) one of the causes of that increase.  In sections where qo-words are less common at line-start, perhaps they are shorter than average?

Is the LBB the only cause of line-start qo anomalies?  Of all line-start anomalies? I don't know; but any investigation of those anomalies should estimate the LBB, and subtract it, before claiming that "something else" is going on.

The LBB can also cause anomalies beyond the first word of the line.  Suppose that, in some text, words that are longer than average tend to be followed by qo-words more than expected by chance.  Then the LBB will cause qo-words to be more common than expected as the second word of each line.

All the best, --stolfi