12-11-2025, 11:18 PM
Here are some more data about prefix, sufix, stem and other position within the text and line types of the MS.
The tables below summarize how the word segments behave in different positions and line types. The goal with these tables is to see if prefixes and suffixes appear in consistent places, which would indicate a sort of structure (so no random segmentation).
Prefix frequency by line position (fraction of words whose first segment is a prefix):
It seems that prefixes are most frequent at the start of lines (around 0.75–0.80) and that they decrease gradually towards the end.
Suffix frequency by line position (fraction of words whose last segment is a suffix):
It seems that suffixes grow stronger toward the end of the line. This pattern is stable across groups.
Global balance of segment types:
About half the detected segments behave as suffixes, one third as prefixes, and very few as neutral stems. This seems to confirm that the segmentation algorithm identifies quite strong word boundaries.
Stability of the model (parameter sweep):
this shows how many segments are labeled as prefix/suffix/stem/other under different thresholds. Changing thresholds (r_prefix and r_suffix) barely changes the number of suffixes, which means suffix identification is stable; prefix and “other” counts shift slightly. It’s a basic sensitivity test(good models should be stable under moderate threshold variation).
Prefix-sufix co-occurrence:
Most words follow a prefix–prefix or prefix–suffix pattern. This might mean that the structure is repetitive and rule-like: words often start with a fixed opening element and end with a stable closing element.
Suffix coverage: 0.597% of all words end with one of the learned suffix patterns, showing that the suffix model generalizes well across the text.
The tables below summarize how the word segments behave in different positions and line types. The goal with these tables is to see if prefixes and suffixes appear in consistent places, which would indicate a sort of structure (so no random segmentation).
Prefix frequency by line position (fraction of words whose first segment is a prefix):
It seems that prefixes are most frequent at the start of lines (around 0.75–0.80) and that they decrease gradually towards the end.
Suffix frequency by line position (fraction of words whose last segment is a suffix):
It seems that suffixes grow stronger toward the end of the line. This pattern is stable across groups.
Global balance of segment types:
About half the detected segments behave as suffixes, one third as prefixes, and very few as neutral stems. This seems to confirm that the segmentation algorithm identifies quite strong word boundaries.
Stability of the model (parameter sweep):
this shows how many segments are labeled as prefix/suffix/stem/other under different thresholds. Changing thresholds (r_prefix and r_suffix) barely changes the number of suffixes, which means suffix identification is stable; prefix and “other” counts shift slightly. It’s a basic sensitivity test(good models should be stable under moderate threshold variation).
Prefix-sufix co-occurrence:
Most words follow a prefix–prefix or prefix–suffix pattern. This might mean that the structure is repetitive and rule-like: words often start with a fixed opening element and end with a stable closing element.
Suffix coverage: 0.597% of all words end with one of the learned suffix patterns, showing that the suffix model generalizes well across the text.
