About the construction of lines in the MS

[Jorge_Stolfi]

That's an interesting conjecture, but I believe we could test it more directly than via simulation.

If it's true that words starting with [Sh] are more likely to occur after longer words than shorter ones, then -- in the absence of meaningful line-based patterning -- we should see this effect elsewhere too.  If we were to exclude the beginnings and ends of lines from consideration and calculate the average length of mid-line words that precede words starting with [Sh], it should differ significantly from the norm.

I ran this test on the Starred Parags section (SPS).

First bad news: the results depend totally on whether I ignore commas or consider commas to be word spaces.  

First, I discarded parag heads, parag tails, and lines with fewer than 10 words.  Among the remainder, I looked at word pairs (w1,w2) where w1 is in position 4 to N-5 (counting from 1) where N is the number of words on the line.

In the tables below, with bar graphs, the first column is the length L of word w1.  The second column is the count of pairs where w1 has length L. The third column is the count of pairs where w1 has length L and w2 begins with "sh". The fourth column  is the ratio F of those two counts, that is, the frequency of a "sh" word after a word with length L.  The bar graph shows F, where each "X" is worth 0.005 (0.5%)

For this test I used my own transcription of the SPS, that differs slightly from Rene's.  Mostly, I was a bit more liberal with the use of commas, and I disagreed with some of his parag breaks.

In both cases there does seem to be a positive correlation between the length of w1 and the probability of w2 starting with "X".   Thus TLA or SLA should indeed enhance the frequency of "sh" words in position 2 of the line.

However, the counts of "sh" words are rather small, therefore affected by sampling noise; and they depend on the handling of commas in some complicated ways.  For instance,  if I ignore commas, there 12 words of length 1, but none is followed by an "sh" words. If I treat commas as spaces, there are 61 words of length 1, and 5 of them are followed by an "sh" word:

  l sheedain
  l shedy
  l sheo
  y sheo
  l shalshy

Also, if I ignore commas there are 16 words w1 of length 9, and 3 of them are followed by a "sh" word w2, creating a tall spike on the bar graph:

  oteedaiin sheedy
  cheolchal shchy

But if I treat commas as word spaces, there are 22 words of length 9, and only 2 are followed by an "sh" word:

  lsheedain shear
  oteedaiin sheedy
  cheolchal shchy

And these three w1 words do look like they are cases of missing word spaces.  That is, they should have had commas in them...

And we don't know which spaces the Scribe considered to be word breaks when applied his line-breaking algorithm...

All the best, --stolfi

statistics for words 4..-5
considering only non-head, non-tail lines with at least 10 words
ignoring commas
with at least 10 words
!! <f103r.2;U> 15 words, taking 4..10
!! <f103r.3;U> 14 words, taking 4..9
!! <f103r.6;U> 12 words, taking 4..7
330 head lines seen
330 tail lines seen
406 body lines read
314 body lines with at least 10 words
981 word pairs collected

001    12    0  0.000
002    55    1  0.018 XXXX
003    43    2  0.047 XXXXXXXXX
004   146   10  0.068 XXXXXXXXXXXXXX
005   256   15  0.059 XXXXXXXXXXXX
006   240   19  0.079 XXXXXXXXXXXXXXXX
007   142    9  0.063 XXXXXXXXXXXXX
008    58    3  0.052 XXXXXXXXXX
009    16    3  0.188 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
010     8    0  0.000
011     4    0  0.000
014     1    0  0.000
TOT   981   62  0.063 XXXXXXXXXXXXX

statistics for words 4..-5
considering only non-head, non-tail lines with at least 10 words
treating commas as word spaces
with at least 10 words
!! <f103r.2;U> 15 words, taking 4..10
!! <f103r.3;U> 14 words, taking 4..9
!! <f103r.6;U> 13 words, taking 4..8
330 head lines seen
330 tail lines seen
406 body lines read
380 body lines with at least 10 words
1568 word pairs collected

001   61    5  0.082 XXXXXXXXXXXXXXXX
002  121    4  0.033 XXXXXXX
003   94    5  0.053 XXXXXXXXXXX
004  261   19  0.073 XXXXXXXXXXXXXXX
005  404   22  0.054 XXXXXXXXXXX
006  352   27  0.077 XXXXXXXXXXXXXXX
007  186   14  0.075 XXXXXXXXXXXXXXX
008   62    4  0.065 XXXXXXXXXXXXX
009   22    2  0.091 XXXXXXXXXXXXXXXXXX
010    3    0  0.000
011    2    0  0.000
TOT 1568  102  0.065 XXXXXXXXXXXXX