(09-08-2020, 11:52 AM)ReneZ wrote: You are not allowed to view links. Register or Login to view.Thanks Darrin for this interesting analysis.
I read the linked page, and it leaves me with a number of questions.
What exactly do you mean with:
Quote:Languages in non-Latin characters were machine transliterated before normalization.
If one uses Unicode, there is no longer any issue with non-Latin characters. One can still apply the language rules to translate upper case to lower case (for languages that use two cases).
Hi Rene
I used a Linux package call unidecode from github. See the description below. I definitely need to write the "one click down" article as you guys have a lot of excellent questions.
[b]Description:[/b]
Text::Unidecode provides a function, `unidecode(...)' that takes Unicode data and tries to represent it in US-ASCII characters (i.e., the universally displayable characters between 0x00 and 0x7F). The representation is almost always an attempt at *transliteration* -- i.e., conveying, in Roman letters, the pronunciation expressed by the text in some other writing system

