The 'Chinese' Theory: For and Against

[rikforto]

@JoeyB, the problem is not picking the highest quality LLM for Standard Chinese, the main written language of China, but tools for addressing the spoken languages. Because Cantonese is not typically written---most Cantonese speakers never read or write in it---an LLM that generates Cantonese is not a much sought after product, and the corpuses forming the training data are not large at all. Likewise, because writing in Classical Chinese is still in living memory (if barely), it casts a huge shadow over written Standard Chinese, and there is a parallel problem with Cantonese texts, that mean that a good deal of vocabulary that you would never learn through the oral/aural route is in the training data. We also need a human being who can judge the output credibly, and while I can spot some obvious problems with what has already been offered, I am unable to make close calls.