21-11-2025, 08:44 AM
That is quite a long sample indeed, so I would need to conclude empirically and objectively that this algorithm fails to detect word boundaries in a Thai text.
No, Thai words always have at least one consonant and one vowel. In fact, not all vowels are written, in particular the default short 'a' at the end of a syllable, or the default short 'o' when the syllable has a closing consonant.
However, I only know one word that consists of a single character: ณ , pronounced 'na', which is quite infrequent.
There are several possible ways in which the algorithm should have worked, but fails to. One is that a longish text will have 60-70 different symbols. If this exceeds a limit in the code, I suppose that the output would indicate this....
Also, not all symbols are vowels or consonants. There are four tone marks, of which two appear frequently.
That shouldn't really matter though...
(19-11-2025, 09:33 AM)quimqu wrote: You are not allowed to view links. Register or Login to view.My question: does Thai have single char words? If not, I could set the model not to take into consideration single chars and maybe have a better result.
No, Thai words always have at least one consonant and one vowel. In fact, not all vowels are written, in particular the default short 'a' at the end of a syllable, or the default short 'o' when the syllable has a closing consonant.
However, I only know one word that consists of a single character: ณ , pronounced 'na', which is quite infrequent.
There are several possible ways in which the algorithm should have worked, but fails to. One is that a longish text will have 60-70 different symbols. If this exceeds a limit in the code, I suppose that the output would indicate this....
Also, not all symbols are vowels or consonants. There are four tone marks, of which two appear frequently.
That shouldn't really matter though...