The 'Chinese' Theory: For and Against

[Jorge_Stolfi]

Any professional statistician is going to swat the robustness of the analysis in your report.

OK, so why don't you ask a statistician what he thinks of the match?

your hypothesis that there is a match between  aiin words and 'zhu'.

The hipothesis is not about "aiin words".  There is a close match between daiin specifically (alone or as a suffix) and 主.

The shape of the histograms of paragraph sizes are the shape of any random distribution.

OK, then I make the same challenge I made to @oshfdk and others: find me any old manuscript -- any language, any origin, any topic, any date before 1500 -- whose parag count and min, max, and average size are anywhere close to those of the SBJ as the SPS numbers are.

And if you find such a book, see if you can find one parag and one word whose places of occurrence in that parag match those of 主 in the Rooster recipe, even with 10% error.

I will be waiting. I am still waiting.

The long paragraph does not give enough data on which the methods of statistical hypothesis testing can be used.

Sigh.  Why do I have to do all the work myself? 

The relative positions of the seven occurrences of 主 in the Rooster recipe are 0.19, 0.87, 0.97, 1.11, 1.35, 1.51, 1.79.

The relative positions of the five occurrences of daiin in f105v.32 are 0.05, 0.70, 0.90, 1.20, 1.62.

So what is the probability that the second set of  five numbers "matches" five of the other seven numbers by chance, if we allow an arbitrary shift C (the same for all five numbers)?

With the arbitrary shift, the second set is 0.05+C, 0.70+C, 0.90+C, 1.20+C, 1.62+C  Obviously if we pick C = 0.14 the lowest numbers will match.  To simplify the problem, let's instead take the differences between the first number and the other numbers

For the Rooster recipe, the six differences are 0.68, 0.78, 0.92, 1.16, 1.32. 1.60.

For the f105v.32 parag, the four differences are 0.65, 0.85, 1.15, 1.57.  Note that the "C" cancelled out.

You should admit that there is a fairly good match between these last four numbers and four of the six numbers above.  The discrepancies are -0.03, -0.07, -0.01, and -0.01.

But computing the probability of these matches being due to chance is complicated because the two lists are automatically sorted.  So instead let's look at the differences between consecutive numbers.  That is, the number of characters between successive occurrences of 主 in the SBJ and the number of EVA characters between successive occurrences of daiin in the SPS.  Both divided by the respective average parag sizes.

For the SBJ, the six differences are 0.68, 0.10, 0.14, 0.24, 0.16, 0.28.

For the SPS, the four differences are 0.65, 0.20, 0.30, 0.42.

The alleged match above then becomes: the bottom four numbers match the top six, if we are allowed to add any two pairs of them, or any three.

In particular, we claim the match 065≈0.68, 0.20≈0.10+0.14, 0.30≈0.24, and 0.42≈0.16+0.28.  The discrepancies are -0.03, -0.04, +0.06, and -0.02.  

How good is this match?  The smallest relative gap in this case is 0.10 and the largest is 0.68, so let's assume that, if we just picked at random a parag and a word such that the word occurs five times in it, each if the four successive differences would be uniformly distributed in the range [ 0.10 _ 0.68 ].  the probability that each of those four numbers fell within ±0.06 of the corresponding SBJ number would be at most 0.12/0.58 ≈ 0.184.  The probability that all four fell within that range would be less than 0.184^4 ≈ .00183, or less than 2 in 1000.

But we must account for the pair sums.  There are 14 ways to reduce a list of six numbers to a list of four by adding consecutive numbers.  So the probability that a random (parag,word) with 5 occurrences would match the Rooster recipe with any allowed fudging (C shift and additions) and with discrepancy less than ±0.06 is 14 times .00183, that is, less than 2.8 percent.

Now you may say that there may be more than a hundred pairs (parag,word) in the SPS with at least 5 occurrences, so there is a substantial probability that some such pair would provide a match as good as the above.  But my (parag,word) pair was not picked that way.  Again, I picked daiin because it was the most common word in the SPS, and I picked f105v.32 because it was the longest parag (both in words and in EVA characters).  And the same for 主 and b1.4.096.  There was no cherry-picking there.  

Besides, to match the last 主 in Rooster, at relative position 1.79, the length of the candidate SPS parag would have to be almost twice the average length.  That condition drastically reduces the number of (parag,word) pairs available to cherry-pickers.

But you still need more evidence?  Sigh, okay, it is coming...

All the best, --stolfi