The 'Chinese' Theory: For and Against

[Jorge_Stolfi]

Chinese absolutely does not have this kind of long-distance dependency, neither in syntax nor in phonetics. Chinese is an isolating language. Each character stands alone. The strokes, structure, or pronunciation of one character have zero mechanical influence on the next character.

That is quite false even if one takes "character" to mean a Chinese hanzi character -- that is, a syllable. 

For example, in the Shennong Bencao the character 益 is often followed by 气, the character 久 is often followed by 服 or 食, the character 一 is almost always followed by 名, etc.  The second character after a 生 is often 谷, and the second character after 味 is usuallly one of five characters. 

If the Voynich manuscript were a simple phonetic transcription of Chinese (like Pinyin), we still would not see this 51% vs 28% dependency between the end of one word and the start of the next.

Those dependencies do exist in the SBJ, because of the dependencies between words above.

The following file shows the counts of triples [letter][space][letter] in a Mandarin pinyin version of the Shennong Bencao.  


  last-to-first.txt (Size: 6.43 KB / Downloads: 6)

You can see, for example, that a syllable that ends in "g" (from "ng") is most often followed by a syllable that starts with "s", about five times more often than one that starts with "x" (511 times against 130).  

Whereas a syllable that ends in "i" is most often followed by a syllable that starts with "x", about twice as often as one that starts with "s" (151 times against 67).

That brings up two important points: 

1. Never make any claim about statistics of a language of text without checking it first.  You will almost always be surprised.
   
2. Don't waste your time with character-based statistics in alphabetic languages like Latin, pinyin, or Voynichese.  They are usually more confusing that illuminating.  Look at word-based statistics instead.
   

The reason for point 2 is that the frequency of a character (or character pair etc) in a text is determined by its occurrence in the most frequently used words and phrases in that text, and these word statistics are are all mashed together.  

For example, in English the combination "th" is very common at the start of a word.  But that is because there are a handful of very common words like "the", "that", "this", and "them" that happen to start with "th".   So the claims that, in English,

• Words often start with th.
  
• A word that starts with th is often followed by a word that starts with i.
  

while they may be true (I should have checked!), they do not really say anything useful about the English language, and are in fact misleading.  Much more useful would be to say

• The most common word is the.
  
• The word this is often followed by the word is, while they is often followed by are.
  

Character and digraph frequencies (or entropies, correlations, etc) may be useful when you start studying a text in an unknown script; but after that one should look instead at words and word patterns.  For the VMS, the window when character-based statistics were appropriate ended some 80 years ago.

All the best, --stolfi