03-09-2025, 06:37 PM
(03-09-2025, 05:24 PM)Jorge_Stolfi wrote: You are not allowed to view links. Register or Login to view.I can't reply to specific points now. But here are my general feelings:
I assume that the Scribe was copying from a draft on paper, provided by the real Author, and treated running text as he was used to do in Latin or whatever other language he normally used. Namely, he generally disregarded line breaks in the draft, and inserted his own breaks where needed to produce a block of left and right justified text, except for the last line.
His "algorithm" presumably was like this: when he got to within 3-4 cm of the right text rail, he looked ahead in the text and decided how many words would still fit. Then he stretched or squeezed those words so that they would end at the right rail.
This algorithm alone would produce anomalous statistics at the start and end of lines. Namely, the last word of a line would tend to be shorter than average, while the first word would tend to be longer than average. We could test it easily by applying it to any running text file.
This algorithm also can explain differences in glyph and glyph pair statistics, since these are dominated by the few most common words -- and the most common short words will have different char stats than longer ones. (If this algorithm is applied to English text, the most common short words probably have more "i" letters and "th" digraphs than long words, or words in general.
Other possible factors that could explain the anomalous statistics at start and end of lines are:
All the best, --jorge
- The Author may have told the Scribe that he could abbreviate "aiiin" as "am" if he really needs to.
- The Author may have told the Scribe to avoid breaks after or before certain words.
- The Author may have told the Scribe that he could break certain long words in certain places, say between gallows if the word has two gallows; but he should add an y or m or whatever, before or after the break, to indicate a split word.
- While the Scribe was supposed to ignore the line breaks in the draft, he may have (consciously or unconsciously) favored breaks where the draft had breaks. E.g., if, in the same situation, when the draft had "Chody ol Shedy" on the same line he would break like "Chody ol | Shedy", but if the draft happened to have "Chedy | ol Shedy" he would tend to chose the latter.
I agree that breaking up the text to yield a justified VMS text could skew line-by-line statistics. This idea is also consistent with Claire Bowern's recent results, as reported on Bluesky, that spaces between vords tend to shrink toward the right margin, as if the writer of a given bifolio was trying to cram as many tokens as possible onto a given line: You are not allowed to view links. Register or Login to view.

