The Voynich Ninja

(28-06-2026, 04:39 AM)JoJo_Jost Wrote: You are not allowed to view links. Register or Login to view.If it were just a matter of y and qo, but of course it isn't...

Of course it is not!  It is expected that in ANY text in ANY language, the statistics of ANY property of a word will be affected by ANY property of the following word.  It would be noteworthy if that was not the case.

Here are statistics for the Starred Parags section of the VMS (my transcription, which is slightly different from Rene's current IVT).  The first property (P1) is "the last EVA letter of the word", and the second property (P2) is "the first EVA letter of the next word":

  4298 -y.  | 1313 -y.q- | 143 -l.k- |  100 -y.t-
  2095 -n.  |  172 -n.q- | 102 -y.k- |   40 -l.t-
  1499 -r.  |  136 -l.q- |  25 -r.k- |   12 -n.t-
  1416 -l.  |   92 -r.q- |  16 -o.k- |   10 -r.t-
   317 -o.  |   49 -d.q- |  10 -n.k- |    6 -m.t-

The first column is the counts of P1 in the whole section. It says that there are 4298 words that end with "-y", 2095 that end with "-n", etc..  

The second column is the counts of P1 before words that begin with "q-". It says that, before words that begin with "q-", there are 1313 words that end in "-y", 172 words that end with "-n", and so on.  As you noted before, the ratio of "-y" words to "-n" words is ~2:1 in general, but jumps to ~7:1 before "q-" words.

The third column shows counts for P1 when the next word begins with "k-".  In this context, words that end with "-y" are scarce, so much that the most common ending is "-l"; and the ratio of "-y" to "-n" is ~10:1.

The last column shows the situation before words that begin with "t-".  In this context, the most common ending is still "-y", but the second most common is "-l", and the ratio of "-y" to "-n" is ~8:1.

But it is more interesting to let P1 be the first EVA letter of the first word, and  P2 be the last EVA letter of the next word:
      2420 o-. | 350 o-.-r | 314 o-.-l | 945 o-.-y | 486 o-.-n
      1835 q-. | 246 q-.-r | 231 q-.-l | 829 q-.-y | 371 c-.-n
      1747 c-. | 243 c-.-r | 215 c-.-l | 643 c-.-y | 328 q-.-n
       961 s-. | 116 a-.-r | 129 a-.-l | 435 s-.-y | 213 s-.-n
       841 a-. | 110 l-.-r | 118 s-.-l | 300 a-.-y | 156 a-.-n
       767 l-. | 104 s-.-r | 102 l-.-l | 277 l-.-y | 155 l-.-n
       483 d-. |  56 d-.-r |  80 d-.-l | 200 d-.-y |  75 y-.-n
       348 y-. |  47 p-.-r |  42 k-.-l | 139 y-.-y |  67 d-.-n
                                               
  q:c=  1.05   |  1.01     |  0.98     |  1.29     |  0.88

The first column here says that in the SPS there are 2420 words that begin with "o-", 1835 words that begin with "q-", 1747 words that begin with "c-", etc.   

The second column says that, before words that end with "-r", there are 350 words that begin with "o-", 246 words that begin with "q-", and so on.  

The other columns give the analogous initial-letter statistics for words before words that end with "-l", "-y", and "-n".

Note that the ratio of "q-" words to "c-" words in general is 1.05, and it is more or less the same also before "-r" words and "-l" words.  

But before words that end with "-y", there is a significant excess of words that begin with "q-" rather than "c-" (ratio 1.29); whereas, before words that end with "-n", the words that begin with "c-" are more common than those that begin with "q-" (q:r ratio 0.88).  

So, from these statistics, the influence of the end of a word on the beginning of the previous word does not seem as dramatic as the "-y.qo-" effect.  However, given the size of the counts, it seems statistically significant.  (There are more dramatic effects before words that end with "-k" or "-s" or "-t", but these counts are small thus the effect may be just sampling noise.)

Are these influences due to some long-range phonological or grammatical property of the final letter that can attract or repel certain letters almost two tokens away?  Quite probably not.  The "q-.-y" anomaly is probably due to a few common word pairs that happen to be of the form "q***.***y", like "qokedy.okeey", rather than "c***.***y", like "chedy.lkeedy".

All the best, --stolfi

Stolfi,

I think we’re still talking about two different levels of structure here.

I’m well aware that in any natural text, the properties of adjacent words can depend on one another (though not always). Complete independence would actually be quite surprising.

But that’s not the specific pattern; it’s not the structure I’m pointing out. In the Voynich manuscript, there’s a deeper structure that can’t be explained by such simple relationships.

I’ll go a little deeper here, because this calls into question your line of reasoning so far.

This strange long-range effect isn’t limited to “edy,” but is found in several parts of the “e” family.

The interesting thing is: the same ranking emerges across multiple e-family contexts. The percentages are rounded:
                t                 k                 ch                sh
-edy      25% (351)    27% (564)    37% (1232)  49% (658)
-ey        15% (151)    17% (297)    24% (883)    37% (444)
-eey      20% (283)    22% (803)    22% (322)    37% (197)
-eody    13% (86)      22% (134)    28% (138)    34% (74)
-eedy    35% (245)    37% (650)    44% (105)    47% (115)

So the same pattern emerges time and again:

t < k < ch < sh

That’s not the same as saying, “-y frequently precedes q-.” Nor is it simply “dy before qo.” The mere “dy” occurs in about 18% of cases (n = 231), while “-shedy” occurs in about 49 % of cases.

The character preceding the e-nucleus thus behaves like a recurring structural factor.

Your examples of general P1/P2 correlations cannot explain this. This table shows that the same internal sequence repeats within the e-family.
And on top of that, there are the structural features of the other VMS families, which, for example, avoid “qo.” As I said, it’s not about individual parts but about this entire system that underlies the VMS.

The explanation based on word pairs doesn’t hold up here, especially since you can remove parts of the most common word pairs and the table’s structure remains largely intact. To verify this, however, we would need to agree on a test.

Taken on their own, however, they still do not explain why the same sequence t < k < ch < sh occurs in several contexts within the e-family, without depending on edy...

And if this would be part of the model of Chinese syllables, what exactly do t, k, ch, and sh do before the e-nucleus, and why do they produce the same order in terms of the probability of following qo?

As I said, this is going to be very difficult to explain.

(28-06-2026, 09:11 PM)JoJo_Jost Wrote: You are not allowed to view links. Register or Login to view.The percentages are rounded:

The percentages make more than 100% horizontally and vertically, how are they calculated?

(28-06-2026, 10:16 PM)nablator Wrote: You are not allowed to view links. Register or Login to view.The percentages make more than 100% horizontally and vertically, how are they calculated?

The numbers aren't related; they show how often “qo” follows in each field.

So t + edy = 25% (351), which means that in 75 percent of cases, it doesn't

sh + edy = 49% (658), so in about 51% of cases, it doesn't

(28-06-2026, 06:29 AM)JoJo_Jost Wrote: You are not allowed to view links. Register or Login to view.Sorry, there was a mistake in this post and I had downloaded the Bencao file and noticed a few errors—probably from Google Translate. I've now decided to wait until Stolfi finishes the new version.

That file you downloads (F1) does have errors of several kinds:

• It is not the text that the Author would have access to in 1400 CE. That file F1 derives from a modern academic attempt to reconstruct the SBJ as it may have existed in 200 CE or earlier.  The version that got transcribed into the VMS must have been one embedded in some (much larger) printed medical encyclopedia that were created in 1080 CE and remained in use until the 1800s or later.  I don't have a clean file of this "right" version of the SBJ. I have a file (F2) that is an OCR of a later printing of that presumed encyclopedia, but the SBJ text in it is not cleanly marked, and I depend on the lalamos to extract it, one recipe at a time.  The differences between F1 and the version extracted from F2 are not great but are critical to my matching efforts: the main one is that F1 often uses 主治 instead of 主, or sometimes omits the 主 -- and that is the main crib that I use in the matching.
  
• That file F1 apparently was created by volunteers working through a wiki tool.  Thus it has many localized errors, like a missing recipe here, two merged recipes there.  Actually I got two versions of F1 (F1a from the Chinese Texts Project and F1b from the Chinese Wikisource); both are buggy, but not in the same places, so I was able to fix many errors by comparing them.  Needless to say, I did not correct any errors that may have existed in both files.
  
• The files F1a and F1b were in Chinese characters (hanzi).  I used Google Translate to convert them to modern Mandarin in the pinyin encoding.  By I don't know what language the Dictator and the Author used.  If it was one of the 50+ Sinitic languages ("dialects"), the mapping should be mostly one-to-one from hanzi to syllables; but not quite, since a few different hanzi have the same pronunciation in Mandarin but different pronunciations in Cantonese, and vice-versa, or a hanzi may have two different readings in Madarin depending on context, but only one in Cantonese, or vice versa, etc.  And if it was some non-Sintic language, like Vietnamese, the mapping may have been quite a bit messier.
  
• The Author apparently omitted certain parts of the recipes that made no sense outside of China (or even inside it).  Most recipes have a "[Flavor]" field at the beginning, right after the remedy's name, with 2-3 hanzi that classify the remedy in the theoretical framework of Traditional Chinese Medicine.  Most recipes also have fields [Other name] at the end that give Chinese aliases for the name, and a field [Provenance] that specifies where the remedy grows or is collected -- usualy 2-3 hanzi, like "flat valleys" or "mountain streams".  As far I have checked those fields were always omitted by the Author.  It seems that the Authos also omitted some comments inside some recipes that also would make no sense in Europe, like "the heads of chickens that were hung over the East gate are particularly good at eliminating ghosts."   The file F1 still includes all those fields.
  

Still, that F1 file should be good enough for investigations like word structure, Levenstein neighbor graphs, letter correlations across word gaps, etc.  Only the last item above may have an impact on some of those studies -- because those omitted fields are highly repetitive, and they may be numerous enough to affects statistics.

I am preparing a new file F3 with some of those errors fixed, but it may still take a while.

All the best, --stolfi

On my bike ride today, it occurred to me that I could probably illustrate the difference between the Classical and vernacular language with a translation of Sun Tzu's Art of War. I cannot speak to the details, but in broad strokes You are not allowed to view links. Register or Login to view. is what I expected. Before even looking at it, the fact of the translation should be enough to cast doubt on the idea they are the same language, but the differences are striking enough that no matter how intimidating Chinese characters seem, it should be obvious at a glance this the Classical and modern languages are different. I have hacked it up so that each sentence in the Classical appears with its written Mandarin counterpart; the Classical is on the top and the modern Chinese is on the bottom.

Quote:孙子曰：兵者，国之大事，死生之地，存亡之道，不可不察也。
孙子说：战争是国家的大事，它关系着人民的生死和宗庙社稷的存亡，是不可不认真考察了解的。

Quote:故经之以五(事)，校之以计，而索其情：一曰道，二曰天，三曰地，四曰将，五曰法。
所以，要以如下五个根本方面的因素为基础，去对敌我双方的情况进行比较分析和评估，从而探索战争胜负的情势。这五个根本方面即：一是“道”，二是“天”，三是“地”,四是“将”，五是“法”。

Quote:道者，令民与上同意也，故可(以)与之死，可(以)与之生，而不(畏危)〔诡也〕。
所谓“道”，就是要使民众与君主同心同德，可与君主死生与共而无违疑之心。

Quote:天者，阴阳、寒暑、时制也。
所谓“天”，就是指昼夜、寒暑与四时节令的变化。

Quote:地者，远近、险易、广狭、死生也。
所谓“地”，就是指道路的远近、地势之险厄平易、开阔狭窄与高低向背等地理条件。

Quote:将者，智、信、仁、勇、严也。
所谓“将”，就是要求将帅要具备智谋、信实、仁爱、勇敢和严明等五种品格。

Quote:法者，曲制、官道、主用也。
所谓“法”，就是指军队的组织编制、将吏的职分管理与军需物资的掌管使用。

Quote:凡此五者，将莫不闻，知之者胜，不知者不胜。
凡属上述五个方面的事，身为将帅，都不能不过问。了解这些情况，就能打胜仗;不了解这些情况，就不能打胜仗。

They are remarkably different languages, so much so I've long wondered if comparison to the Romance languages might be underselling them. At any rate, the "Mandarin" You are not allowed to view links. Register or Login to view. should look comparably different from the Classical text, and certainly not word-for-word the same

(29-06-2026, 11:57 PM)rikforto Wrote: You are not allowed to view links. Register or Login to view.They are remarkably different languages, so much so I've long wondered if comparison to the Romance languages might be underselling them. At any rate, the "Mandarin" You are not allowed to view links. Register or Login to view. should look comparably different from the Classical text, and certainly not word-for-word the same

And yet there are common symbols shared by both versions that roughly maintain their relative positions within the sentences in both versions. 

I typed out both into paint and marked the symbols 不 (in red) and 天 (in green). It's not super scientific how i've done this, but I basically typed both out (all sections), and then squished the longer version to be the same width as the shorter version to get the relative positions of characters. It's obviously not perfect - it's like a Back-of-the-envelope version of this analysis - but you do get results reminiscent of the daiin results from Jorge's first SPS-SBJ post: 

[attachment=16238]

(29-06-2026, 11:57 PM)rikforto Wrote: You are not allowed to view links. Register or Login to view.On my bike ride today, it occurred to me that I could probably illustrate the difference between the Classical and vernacular language with a translation of Sun Tzu's Art of War. I cannot speak to the details, but in broad strokes You are not allowed to view links. Register or Login to view. is what I expected. Before even looking at it, the fact of the translation should be enough to cast doubt on the idea they are the same language

I don't get your point here.  

Sure, the language(s) spoken by the "Divine Farmer" and Sun Tzu in some royal court before 300 BCE were vastly different in vocabulary, phonetics, and syntax from any language spoken anywhere today, or in 1400 CE.  Those authors may also have opted for a rather succinct style in writing, did not use any punctuation, and may have omitted inflections and particles that may have existed then.  And many characters that they used changed their meaning to various degrees since then.  Thus, if one wants to convert texts "of any length" (= "long enough") from that period to natural fully grammatical running prose in any modern language, a one-hanzi-for-one-syllable will not work.  One must change the syntax, replace characters, insert missing connectives and punctuation, even replace whole idioms with completely different idioms.  

And I never claimed otherwise...

But that is totally irrelevant for the proposed dictation scenario and the claim "SPS ≈ SBJ".  The syntax of Shennong Bencao is extremely simple: mostly enumerations of 2-5 hanzi phrases that are single dictionary compounds, or noun-adjective pairs, or verb-object pairs.  As I explained before, if one does a one-hanzi-to-one-syllable reading in modern Mandarin or Cantonese, one get a spoken text that, while slightly ungrammatical by the rules of those languages, it still perfectly understandable by anyone who speaks only that modern language.  Except that this hypothetical listener may not know many of the terms, because they are technical names of exotic plants or specific medical conditions; but this problem would remain even with a free translation.

Just You are not allowed to view links. Register or Login to view. such literal one-to-one readings of the beginning of the "rooster" recipe into modern spoken Mandarin and Cantonese, and the minimal changes that (according to Google AI) would make those sentences grammatically correct in the respective languages.  The changes were small, and mostly confined to the keywords that introduce  the real contents.   Like replacing [zhǔ] "[main]" in the Mandarin reading by [zhǔ zhì] "[main uses]", or (nǚ rén) "(women)" by (nǚ rén de) "(for women)".  Obviously, these syntactic flaws would be the least of the obstacles to the Author's understanding of the text.

All the best, --stolfi

(30-06-2026, 03:05 PM)Jorge_Stolfi Wrote: You are not allowed to view links. Register or Login to view.The syntax of Shennong Bencao is extremely simple: mostly enumerations of 2-5 hanzi phrases that are single dictionary compounds, or noun-adjective pairs, or verb-object pairs.

The syntax of The Art of War is no more complex, making extensive use of the 4-word phrases and their variants that were extremely important in Classical Chinese and are only used sparingly and as a literary affect in Mandarin. These clauses are famously tricky for their austerity and very precise syntax, not in spite of it, so this remains a strange line to me. Either way, the example illustrates how common they were, and how translation does not respect them. This is because the syntax and vocabulary that made them possible have changed enormously. This is especially obvious in the first and last lines of my example, but continues throughout. Again this matters because the way for The Author to learn these forms is through participation in the literate culture because they do not make sense in the later languages, both as grammatical objects and also as because they cannot be usually be parsed aurally and must each be memorized.

Also, you cannot account for the output of an LLM, and you cannot expect me to do it for you. I thought I had answered this, but because the conversation has sprawled I apparently cut it. Nonetheless, You are not allowed to view links. Register or Login to view. several of the mistakes Google's LLM made and showed that if the neural net is not contaminated with the Classical original it arrives at a very different output and briefly explained why that is expected. I stand by those remarks. The bottom line is that if I wanted opaque output from Google's LLM, I would cut out the middle man and ask it myself. I have offered lexicographical accounts, the judgement of experts, and now translations by and for humans that back my position. I am not going to pretend "I asked an LLM" outweighs that.

Eggyk, if Jorge retreats to your position I will address it in more detail, but I think that graphic very much highlights the substantial challenges proving those texts match. There are also implications for the nature of Voynichese itself if the number of Voynich letters per Chinese character falls below the average number of letters per Voynichese word, and that would have to be addressed. Either way, I'm not misunderstanding Jorge here, he keeps returning to the idea that Chinese can be translated morpheme-by-morpheme, and that is just not true, and his argument must account for that.

(30-06-2026, 05:08 PM)rikforto Wrote: You are not allowed to view links. Register or Login to view.The syntax of The Art of War is no more complex [than that of the SBJ]

That is obviously false.  The Art of War is a discursive text. Your example sentences are long and complex with subordinate clauses, conditionals, logical ties, tenses, moods, etc.  The SBJ has practically none of that.  It is almost like comparing Hamlet to a train timetable.

Quote: this matters because the way for The Author to learn these forms

The dictation scenario does not require the Author to learn any Classical Chinese quirks.  It does not require that ability even from the Dictator.  The latter could have known only the meaning of the individual hanzi.  Not even of the compound terms.  Mastering the old Classical Chinese syntax would have been of little help, for both of them.

Quote:you cannot account for the output of an LLM, and you cannot expect me to do it for you.

Well, then why should I take your objections seriously?  

Quote: You are not allowed to view links. Register or Login to view. several of the mistakes Google's LLM made

Not really.  In that post you showed how speakers of various languages might write "rooster" in hanzi.  That is irrelevant (and is not what I asked GAI about). The Dictator had one written text with 雄鸡 (but in traditional characters) on it, and the Author knew one spoken language, say Cantonese.  They did not need to know that other languages existed and that there were other ways of writing "rooster"; it would make no difference at all.

Quote:I have offered lexicographical accounts, the judgement of experts, and now translations by and for humans that back my position.

And I have refuted all those arguments (or showed that your positions were irrelevant to the issue at hand).

But you have not answered two simple questions: if a Dictator today read aloud the Rooster recipe 
丹雄鸡味甘微温主女人崩中漏下赤白沃补虚温中～
to a Cantonese speaker who does not know Mandarin or Classical Chinese, by reading each hanzi as a Cantonese syllable,

daan1 hung4 gai1 mei6 gam1 mei4 wan1 zyu2 neoi5 jan4 bang1 zung1, lau6 haa6
cik1 baak6 juk1 bou2 heoi1 wan1 zung1 ...

what would the guy miss because of the wrong syntax?   And if the Dictator instead read 

daan1 hung4 gai1 mei6 gam1 mei4 wan1 zyu2 jiu3 hai6 ji1 neoi5 jan4 ge3 bang1 zung1 lau6 haa6
cik1 baak6 juk1 zung6 ho2 ji5 bou2 heoi1 wan1 zung1 ...

which is supposed to be valid Cantonese syntax -- would the guy understand it any better?

If a flight info site says

  ✈ AA1211 [Plane taking off Icon] 11:00 LAX [plane landing icon] 18:10 SFO

and a text-to-voice software reads it out as

  "flight A A one two one one departure one one zero zero L A X arrival one eight one zero S F O"

would a blind person get totally lost, because that is not grammatical English? Of course it would be easier for him if the software said instead

  "the flight with number A A one one two one will depart at eleven hours from the L A X airport
   and then arrive at eighteen hours and ten minutes at the S F O airport"

but would he really need this grammatical stuffing to understand the important info? 

Why do I get this feeling that you do not really care about the Chinese Origin theory, but are upset about some positions of mine about language politics in China that you imagine are implicit in the dictation scenario?  Let me assure you that I have no such intentions, and I don't really care about that topic...

Quote:There are also implications for the nature of Voynichese itself if the number of Voynich letters per Chinese character falls below the average number of letters per Voynichese word

So far the two still match as well as one could expect, considering all the uncertainties about word spaces.

All the best, --stolfi

PS. By the way, You are not allowed to view links. Register or Login to view. seems to be a good intro to the story of Mandarin in China.