01-11-2025, 01:58 AM
In case you are interested, here is also a text in Mandarin Chinese, transcribed in pinyin with numeric tones. Sample:
ge4 wei4 ting1 zhong4 mei3 guo2 zheng4 fu3 jue2 ding4 jin4 yi1 bu4 dong4
jie2 mei3 guo2 jin4 chu1 kou3 yin2 hang2 xiang4 can1 yu4 zhong1 guo2
xiang4 mu4 de5 mei3 guo2 gong1 si1 ti2 gong1 de5 dai4 kuan3 zhong1 guo2
biao3 shi4 zhei4 xiang4 jue2 ding4 jiang1 you3 sun3 liang3 guo2 de5 mao4
It is a set of transcripts of the Voice of America broadcasts from the late 1990s. Word characters are lowercase Latin letters plus "ü" and the digits 1-5, in the iso-latin-1 encoding. Again all punctuation has been removed. Sentence and parag boundaries are not marked (that holds also for the previous Vietnamese samples). The total file is ~300'000 bytes and 59476 words, split in two because of forum limitations.
voa1.txt (Size: 145.36 KB / Downloads: 5)
voa2.txt (Size: 147.63 KB / Downloads: 5)
The language here is probably closer to current Mandarin Chinese, written by fluent reporters (rather than "Biblical" and translated from English, like the Vietnamese sample). On the other hand it may be rather formulaic. For example, the names "China" (zhong1 guo2) and "United States" (mei3 guo2) may be way too common.
All the best, --stolfi
ge4 wei4 ting1 zhong4 mei3 guo2 zheng4 fu3 jue2 ding4 jin4 yi1 bu4 dong4
jie2 mei3 guo2 jin4 chu1 kou3 yin2 hang2 xiang4 can1 yu4 zhong1 guo2
xiang4 mu4 de5 mei3 guo2 gong1 si1 ti2 gong1 de5 dai4 kuan3 zhong1 guo2
biao3 shi4 zhei4 xiang4 jue2 ding4 jiang1 you3 sun3 liang3 guo2 de5 mao4
It is a set of transcripts of the Voice of America broadcasts from the late 1990s. Word characters are lowercase Latin letters plus "ü" and the digits 1-5, in the iso-latin-1 encoding. Again all punctuation has been removed. Sentence and parag boundaries are not marked (that holds also for the previous Vietnamese samples). The total file is ~300'000 bytes and 59476 words, split in two because of forum limitations.
voa1.txt (Size: 145.36 KB / Downloads: 5)
voa2.txt (Size: 147.63 KB / Downloads: 5)
The language here is probably closer to current Mandarin Chinese, written by fluent reporters (rather than "Biblical" and translated from English, like the Vietnamese sample). On the other hand it may be rather formulaic. For example, the names "China" (zhong1 guo2) and "United States" (mei3 guo2) may be way too common.
All the best, --stolfi