About the construction of lines in the MS

[Jorge_Stolfi]

So it seems that two things are happening at the same time. Lines fill the available space, but they also tend to end and start with specific families of patterns.

Thanks for doing these tests!  But indeed we should expect the VMS to have stronger line-end anomalies than those that are created by the simple line-breaking algorithm.  Because the real algorithm has two more complications.

First, the m character is probably an abbreviation for some other ending, that the Scribe could use when needed but also just when he felt like it.   I guess that the "some other ending" is iin.  That is, where the trivial algorithm said

  1. If the next word W fits in the remaining space, write W,
  Else break the line and write W.

his algorithm said

  1. If the next word W ends in iin, with X% probability do W = W/iin/m.

  2. If the next word W fits in the remaining space, write W.
  Else if W ends in 'iin', and Z = W/iin/m fits in that space, write Z. 
  Else break line and write W

where W/iin/m means W with iin replaced by m. and X% is the observed ratio of (freq of m-words) to (freq of m- plus iin-words) in the middle of the line. 

This extra step creates an excess of ms as the last word of the line.  I am guessing that this excess of m must be an important component of the line-end anomaly.

The second complication is best left for another post.  

For now, how can we account for the effect of those abbreviations in the control text?  After joining the body lines into a single token stream, I would try replacing all occurrences of m by iin, and then running the token stream through the line-breaking algorithm modified as above.

By the way: there are many pages where the lines are interrupted by plants.  An intrusion would trigger the line breaking algorithm just as if it was the right rail.  That is, the line-end anomalies should be observed on the last few words before the intrusion, and the line-start anomalies should be observed on the first word after it.  I vaguely recall this having been tested and found to be true.  Is t?  

All the best, --stolfi