11-12-2025, 06:01 PM
(11-12-2025, 05:05 AM)Jorge_Stolfi wrote: You are not allowed to view links. Register or Login to view.What is the result of the same analysis applied to English or Latin text (formatted as filled parags)?
All the best, --stolfi
It would certainly be interesting to see what kind of purely statistically generated chunks would result. I'll see where I can get a sufficiently large corpus.
(11-12-2025, 11:33 AM)nablator wrote: You are not allowed to view links. Register or Login to view.If you want to do statistics on lines you should have only lines in your data, not labels, circular texts, etc.: the "P" lines of the IVTFF transliterations.
Actually "Pt" lines are on the same physical line as the previous paragraph line.
In some cases free-floating words look more like labels than lines to me:
<f68r1.5,@Pb> yky
<f68r1.6,+Pb> dary
<f68r1.7,+Pb> chkchykoly
All single free-floating words should probably be removed:
<f84v.42,@Pb> okar
<f84v.43,+Pb> ydairol
<f84v.44,+Pb> ychckhy
<f84v.45,+Pb> dshedy
<f84v.46,@Pb> okchdy
<f84v.47,+Pb> solchey
<f84v.48,+Pb> dairoldy
<f84v.49,+Pb> darchy
<f84v.50,+Pb> yskhy
<f84v.51,+Pb> ochedy
You're right, the result would probably be more accurate then. Maybe it would be best to only process +P0> lines?
(11-12-2025, 02:52 PM)dashstofsk wrote: You are not allowed to view links. Register or Login to view.This isn't the way to do it. You need to display the ratio of observed ocurrences to the number that would be expected if the start and end 'chunks' were distributed randomly. If then it appears that there is general parity then it would indicate that there is nothing significant.
Sorry, I don't quite understand what you mean.
