The 'Chinese' Theory: For and Against

[Jorge_Stolfi]

What justifies dismissing this information as unproductive before you can explicitly explain it?

First of all let me say that I will use the word "error" as a shorthand for any variant spelling of the same word -- whether it results from actual dictation, copying, and digital transcription errors, or from sandhi (like English "a" changing to "an", or Japanese "hiaku" changing to "biaku" or "piaku"), pronunciation variations, grammatical inflection, non-phonetic markup (like like our upper and lower case, or the English apostrophe in "the monks prayed" vs. "the monks' lodging" vs. "the monk's robe"), an alternative to separate punctuation, etc.  

And, for brevity, I will say "character" to mean either a single character or a digraph, trigraph, etc.

Character-based statistics could lead to understanding of some low-level features of the VMS creation process. Like, they could reveal that there is some percentage of errors affecting certain characters.  

But, by themselves, no amount of character statistics can lead to an explanation of more complex processes.  They will not even distinguish between the above causes of "errors".  They will not even tell whether any detected anomalies are properties of the characters, or rather of some common words that happen to use those characters.  See again the hypothetical example of "th"-words and "i"-words in English.

To have any chance of understanding what the anomalies actually are, and infer their probable causes, you must look at word-based statistics.  As you did -- thanks! 

The "attraction" that you see is the sum of individual word attractions, so it is dominated by a few high-frequency words.

That is precisely the explanation I have already tested. And it is not sufficient [because the attraction shows also among hapaxes]

IF most hapaxes are indeed "errors" (as I suspect), then most Sh-hapaxes will tend to be variants of the most common Sh-words.  Then the next-word statistics for the Sh-hapaxes as a whole are expected to be similar to those of the most common Sh-words. 

You are introducing an additional assumption here: that the rare forms are actually erroneous variants of frequent words. That may or may not be the case. It is another unsubstantiated assumption on your part.

Indeed I do not have strong evidence to support this hunch of mine, and I don't recall why exactly I came up with it.  But I seem to recall that, when I worked out my word structure model, I found that most of the tokens that violated it (some 5-6% of all tokens) were hapaxes.  

I don't know how we could get a conclusive proof or disproof of this conjecture, without deciphering the meanings of the words.  Until then, I will consider this conjecture at least possible, and thus mostly disregard any statistics computed over the hapaxes.

But now let's look at your table of frequencies and qo-attractions for the main Sh- and Ch-words.

In my VMS transcription file, ignoring commas, I count 32189 words that starts with at least two EVA letters [a-z][a-z].  Of these, 2972 (~9%) are Sh-words, 5659 (~18%) are Ch-words, and 4903 (~15%) are qo-words.  So, if successive words were chosen independently at random, a Sh-word or a Ch-word would followed by a qo-word ~15% of the time.

Looking at your table, I see that the frequency of qo-words after a Sh-word varies a lot, from 9.6% for Sho to 53% for Sheedy.  

Thus the hypothesis that "qo-attraction" is a property of word-initial Sh- is not tenable.  Indeed, the words Shar, Sho, Shol, Shor and Shy seem to exhibit "qo-repulsion", while Shear is practically indifferent to qo's charms.   If anything, it seems that, among the words you tabulated, "qo-attraction" is a feature of the endings -iin, -y (except for Shy!), -eol, and -eor, rather than of the initial Sh-.  (But I doubt that this alternative "qo-loving suffixes" hypothesis will hold for all the other words that are not listed in your table.)

Looking now at the Ch- words, it seems that only ChCKhy, CheCKhy, Chedy, Cheedy, Cheey, Cheody, Chey, and Chody show "qo-attraction", while Chaiin, ChCThy, and Chdy are practically indifferent, and the other Ch-words generally show "qo-repulsion" -- sometimes dramatic, like Chor (only 6 in 205, or 2.9% are followed by a qo-word) and Cheo (only 1 in 76, or 1.3%).  Again, these numbers contradict the thesis that qo-attraction (or repulsion) is a property of word-initial Ch-.

However, qo-attraction seems to be roughly similar for similar words, like Shedy, Sheedy, and Sheey.   These similarities could be explained by "errors" -- namely, if most of the Sheey (144 occurrences in your table) and Sheedy (83) are variant spellings of Shedy (412).  

Same for the "qo-repulsion" of Sho (125 occurrences), Shol (182), Shor (94), and Shy (87). Indeed I have long suspected that the ending -o is an error from -y, or the result of a bogus space being inserted after the o in longer words that contain it (like Shol or Shor).

All the best, --stolfi