06-05-2025, 07:19 AM
(06-05-2025, 06:07 AM)tikonen wrote: You are not allowed to view links. Register or Login to view.(05-05-2025, 11:44 AM)ReneZ wrote: You are not allowed to view links. Register or Login to view.More specifically on the topic of word length distribution, when speaking of this, it is important to distinguish between the word type distribution and the word token distribution.
The binomial case is for word type lengths, meaning that the approach of throwing dice when writing out the text does not work.
For more information see here: You are not allowed to view links. Register or Login to view.
Correct, the distinct words of a length follow binomial distribution exactly, the token length distribution follows it approximately (less for labels).
The distinct words follow a binomial distribution exactly if you make the specific choices about what constitutes a single glyph that Stolfi did, i.e. ligatures and ligatured gallows are single characters but the word-final ai*[x] combinations are multiple characters -- for instance, from his data file:
437 shedy {Sh}{e}{d}{y} 4
403 chol {Ch}{o}{l} 3
[...]
219 qokain {q}{o}{k}{a}{i}{n} 6
[...]
115 cthy {CTh}{y} 2
Whether that is saying something about the correct decomposition of the glyphs is an interesting question...