Opinions on: line as a functional unit

[Jorge_Stolfi]

 The question that remains is how much of the "LAAFU" phenomenon on the VMS can be explained by this effect.

If indeed any (for the reasons I've already set out earlier in this post or another thread)

The mere line breaking process definitely created differences in the line-initial, line-medial, and line-final word distributions.   It does that with any language, as long as the words have different lengths.   

If the three word distributions are different, the statistics of glyphs and digraphs at the three locations are almost certainly different too.  

I can believe that this Line Breaking Bias (LBB) was not the only cause of the differences we see in the VMS.  I expect that there will be other causes too; like the Scribe using am as an abbreviation of aiin, or y as a calligraphic variant of o, or squeezing words spaces and stretching glyph spaces so that transcribers incorrectly join or split words.  And maybe the line is indeed a functional unit.

One could test the effect of each of these possible causes, alone and then in combination.  Then, if the LAAFU theory is true, one would hopefully get a better understanding of it after subtracting all those other factors.

To test and measure the LBB effect, for instance, one could

• Take all parags in some hopefully homogeneous section, such as Herbal-A or Bio;
  
• Remove the parag head lines, and, for good measure, maybe also the second lines;
  
• Join all remainig lines of all parags into a single string of words;
  
• Insert line breaks with the trivial line breaking algorithm, for an arbitrary line width W;
  
• Compute the "LAAFU statistics" on the resulting text;
  
• Compare that with the "LAAFU statistics" of the same lines with original breaks.
  

The LBB effect should not depend much on the width parameter W, as long as there are half a dozen words per line.  It should also be somewhat independent of whether one counts the width of a line in EVA characters or any other "alphabet", e. g. counting combos like Ch or iiin as single "letters".

All the best, --stolfi