30-01-2026, 11:14 AM
(30-01-2026, 10:32 AM)dashstofsk wrote: You are not allowed to view links. Register or Login to view.Also, look at the spline transfer plots of the line positions of words starting s. In both A and B words peak at the start of lines, but in A they rise in frequency towards the end.
Beware that the Scribe tend to leave a wider space after an s or r, which often gets transcribed as a word space. Also the shapes of r and s are often ambiguous so one is often transcribed as the other.
The prevalence of these problems will be different for different scribes (or different lengths of practice). And the width of the bogus spaces after r and s, like the widths of characters in general, may vary along the line as the Scribe becomes more conscious of the approaching right rail an starts to squeeze things closer together.
One way to mitigate these problems would be to delete all spaces, map all s to r, map all a and y to o, and then look for occurrences of a given substring like raiiin or or in the compressed lines. This trick would probably conflate unrelated words, but would mostly eliminate the noise due to the most common transcription errors.
By the way, smoothing the distributions with splines may obfuscate important details. The bias towards long tokens at line start is in principle limited to the first token only; the splines make it seem that the effect extends for several tokens after the first.
On the other hand, the word frequencies in the second position will be affected by the line-breaking algorithm, because there is correlation between nearby words. E.g. "ache" may be extra common in second position if "stomach ache" is common, because "stomach" is a long word. Consider also "take XXXX daily" where the XXXX can be one or more words.
All the best, --stolfi
