The 'Chinese' Theory: For and Against

[rikforto]

As for their positions, the counts of EVA letters between consecutive matched cribs are 8, 100, 15, 22, 34, 20, 43, 23, and 52; which differ from the expected counts by -6, -3, 0, +7, 0, 0, +3, -1, and +2 EVA letters.  Looking at the Chinese side, the gaps are 3, 21, 3, 3, 7, 4, 8, 5, and 10 hanzi, and EVA count errors above are equivalent to -1.2, -0.6, 0, +1.4, 0, 0, +0.6, -0.2, and +0.4 hanzi. So the "butchering freedom" here is a standard deviation of ~4 EVA letters (equivalent to 0.8 hanzi) from the expected positions of the 主 cribs.

Is that to much "butchering freedom"?  Sorry, but I don't think so...

This account leaves out that the "butchering freedom", as we're calling it, over-generates potential Voynichese cribs. The Rooster example has 17 cribs in the SPS and 10 in Shennong's Classic. Selecting 10 items from 17 where the order of the selection doesn't matter (because we are going to keep them in the order they appear in the SPS) is 17C10 = 19,448 potential matches. Because the output of this function is very sensitive to inputs, this must be tracked carefully. A smaller recipe that generates 10 SPS cribs for 7 hanzi has "only" 120 choices, but if there are two fewer hanzi cribs it exceeds double that. This smaller finding is offset somewhat by the fact that middling recipes have more potential matches; you give 21 for a particular recipe, and the sum of them may approach a large fraction of 19,448.

If the size is 2,520  (= 21*(10C7)) or 19,448 is going to have to be pinned down to make a precise evaluation of the odds, but the fact that the "butchering freedom" generates these large search spaces and then you search for one that minimizes the errors has huge consequences for claims about how likely it is that the errors here are small! I did take a look at the problem a few weeks ago, and there is a tendency for your Voynich cribs to significantly exceed the number of Chinese ones, both on average and between proportionally similar entries. I might do this in more detail when you publish a clearer picture of what texts you're using, how you got them, and the parameters of your algorithm. But I feel confident saying the algorithm generates a large match space and then selects a favorable result for your claims.

If I'm following your write-ups correctly, you are also running the matching algorithm repeatedly with different crib sets. Transparently, I am not completely clear on the how and why, so it is less clear to me how much this is skewing things. But I can say the decision to selectively exclude cribs is another analytical choice that makes the search space larger than you may realize. If you exclude some cribs and calculate the odds based on that exclusion, you are not calculating the odds for everything you searched. The fact that you have to make the exclusion is important information!

There are two points here. The first is that these are analytical choices. They are of a different quality from the last-mile arbitrary interpretations some solvers do, and I do find that sincerely laudable. They are, nonetheless, choices you as the analyst are making. They reflect a pair of texts that you have created based on judgements you have made. Second, the way those choices distort the probability of the match is absolutely insensitive to the strength of your arguments for making these analytical choices. (I am purposely not disputing those choices in this post, which is not to say I think they are all bulletproof.) The fact that you can discard thousands if not 10s of thousands of similarly defined matches must be accounted for in your argument of the odds even if---perhaps especially if!---those choices "make sense". You are correct that scientists can carefully process data to make a match, but that work requires extremely careful guardrails to make sure they are not effectively creating the data.

That a number of your choices are invisible to you when evaluating the strength of the match surely means you are underestimating the degree to which you are analyzing changes you have made to the two texts, not the texts themselves!