02-06-2026, 03:32 PM
Interesting work — I've been looking at some of the same boundary-structure questions.
I found your work interesting, as I have also been testing some of these same issues related to boundary structure.
One thing I noticed separately is how boundary distributions change between Currier A and B when looking at the internal structure of the tokens, rather than just the n-gram co-occurrences. Why does this A/B split seem to produce concentration profiles with different boundaries? A shows that morpheme boundary positions are slightly more distributed, while B shows a higher concentration at the edges. Of course, it's not a drastic change, but it's very consistent even across different segmentation strategies.
Does your classifier capture this as part of the 89.2% signal, or is it primarily based on lexical distribution?
Does your classifier pick this up as part of its 89.2% signal, or is it mostly riding the lexical distribution (different vocabulary distributions A vs. B)?
I found your work interesting, as I have also been testing some of these same issues related to boundary structure.
One thing I noticed separately is how boundary distributions change between Currier A and B when looking at the internal structure of the tokens, rather than just the n-gram co-occurrences. Why does this A/B split seem to produce concentration profiles with different boundaries? A shows that morpheme boundary positions are slightly more distributed, while B shows a higher concentration at the edges. Of course, it's not a drastic change, but it's very consistent even across different segmentation strategies.
Does your classifier capture this as part of the 89.2% signal, or is it primarily based on lexical distribution?
Does your classifier pick this up as part of its 89.2% signal, or is it mostly riding the lexical distribution (different vocabulary distributions A vs. B)?