Opinions on: line as a functional unit

[dashstofsk]

Another point I'd make is that while the word wrap concept of longer words not fitting at line end is a potential explanation for the word length discrepancies, I don't see it as being a comprehensive potential explanation for the glyph discrepancies.  It might have an impact, but I don't see the evidence for it being the primary explanatory factor for why word types at line start are different.

If the tokens at line start are longer than average, longer word types must have higher frequencies in that position than elsewhere, and the opposite must be true for shorter word types.  For instance, the frequency of qokeedy should be higher at line start than at mid-line, while the opposite should be true for ar.  But indeed it would be important to verify and quantify these differences for the VMS.  

We would expect the word types or glyph clusters "missing" at line end to be excessively popular at line start.  In many cases, we do not see this.  Something else is going on.

Indeed, it is not certain that these differences in word type frequencies will cause differences in character frequencies, but is certainly possible.   And even if the line breaking length bias turns out to be insufficient to explain the line-start anomalies, we would have to subtract its effects in order to understand the real anomalies and infer their causes.

Tavie is tright. Something else is going on. And I have quantified. In case you might have skipped it I did present evidence to show that vertical pair repeats ( consecutive repeats in first-word first-characters ) occur far less that would be expected if the same first-word first-characters were placed randomly. I did compare the actual numbers against numbers obtained from simulations of such random placements to obtain a measure of the confidence in the conjecture that something significant is going on.

You are not allowed to view links. Register or Login to view.
You are not allowed to view links. Register or Login to view.