The Voynich Ninja

So, I at least now understand where the misunderstanding is. To use the Suassarian formulation, Chinese characters are not the signified, they are the signifier. So in any given language it's not that a character represents the lexeme of "house, home, dwelling, abode" it's that it represents one morpheme. In a lot of informal situation, this is pedantry and people speak of a character as though it is the signified, but this is very important when thinking about language correspondence in principle.

For instance, all of "maison", "domus", and "casa" would have different characters, and all those characters would be in all three of French, Latin, and Spanish because all three of those morphemes exist in each language. Were that pictograph above "casa", the French would know it as "chez". (French speakers will have to tell me how great the connotative difference is, but I gather it is not trivial.) There is a complication, largely irrelevant in Sinitic languages but very active elsewhere, that borrowings in unrelated languages included adding native words to the reading set. History of English being what it is, an English speaker would have "house" available as a reading on some Romance word---probably the character for <domus> thinking about the history of Latin in England, but best not to get bogged down. Importantly, though, it is not typically available on any other character, and speakers in this kind of system are very good at picking the intended one, and very much view other choices as illiteracy if they do not feel playful or deliberately transgressive. And if the passage is Latin, not English, they will probably scoff at the idea that it says "house" even if the reading exists in an an English character dictionary and they are correct because only an idiot would think a Latin passage had the signifier "house".

If we stick to Romance readings---perhaps because context tells us it was a Roman settlement---an English speaker educated in Latin logographs for writing English but not Latin would have some clues. I am not far off this person, as I am literate and know my roots, but the closest I come to knowing Latin is I can get by in Spanish. 

Quote:  domus  militis  regis  fenestram rotundam habet.
domicile military regent fenestra  rotund   have

Ah, of course, the house of the military dictator standing in until the king comes of age has fat windows. I am being a little difficult here, but the thing I am being worst about (rotund as "fat") is offset by the great unlikelihood that illiterate English speakers would readily recognize "rotund", "fenestra", and perhaps even the "dom-" root, though I will not be so forceful about that. The roots only get you so far, and can actively mislead. Japanese is the analogy here if you imagine English doublets, and that's the Eastern language have the most formal study of, and on'yomi-kun'yomi correspondences are as perilous as they are helpful. Opening up my kanji old textbook, I see a correspondence between "polish" and "toilet"; "reply" and "turn"; "run" and "to write hastily". In each case, seeing how they got associated makes this seem inevitable, but at the same time, you illustrate that having "ball" and "round" share a word could well lead people astray.

When I look at the readings You are not allowed to view links. Register or Login to view., this is what I see. (I also see the verb has been deleted from a bunch of the sub-entries in the Classical Chinese and Mandarin readings since I last looked closely. Is that an error?) Cantonese-Mandarin correspondences can be quite obscure, but after spot-checking the thorniest examples, they all seem to share a "root". They are the Chinese equivalent of "Domicile military regent fenestra rotund have." I can tell from a glance the translation Gemini is giving you is not colloquial, spoken Chinese because I already documented the mistake it's making with You are not allowed to view links. Register or Login to view. in advance of it making it and several other words that got passed through untouched appear to be that sort of very ancient construction that would not be familiar to someone avoiding the characters. I am chuckling---Google has managed to make an informant in diglossia that can't distinguish the H and L language. What a time to be alive!

For translating Chinese, the best models are Qwen class models / Deepseek, as they are actually trained heavily on Chinese usage. ChatGPT 5.5 would be a close second, put it in Thinking mode. 

There is a paper comparing Deepseek to a prior ChatGPT model on usage etc, you may find it useful: You are not allowed to view links. Register or Login to view.

For translating Chinese, the best models are Qwen class models / Deepseek, as they are actually trained heavily on Chinese usage. ChatGPT 5.5 would be a close second, put it in Thinking mode. 

There is a paper comparing Deepseek to a prior ChatGPT model on usage etc, you may find it useful: You are not allowed to view links. Register or Login to view.

(Editing to add three testing links:

Free Qwen class model: You are not allowed to view links. Register or Login to view.

More Powerful Qwen model: You are not allowed to view links. Register or Login to view. 

Free Deepseek link: You are not allowed to view links. Register or Login to view. )

(25-06-2026, 03:18 PM)JoJo_Jost Wrote: You are not allowed to view links. Register or Login to view.My point was not about raw character frequencies! It was about a conditional dependency across a visible token boundary: the internal form before final y changes the probability that the next token begins with qo.

Yes! Because such correlations are not features of characters, but of words.  It is certain words that tend to be followed by certain other words; and the first happen to often end in y when the second happen to begin with qo.  

Quote:If qo is “and”, the distribution should mainly follow syntax or enumeration structure, not the internal shape of the previous token!!!

The following counts are from the herbal section proper of Culpeper's Herbal.  The first columns is the count of tokens that end with each letter.  The second column is the count of tokens that end with each letter and are followed by the word 'and'.

    22446 e   2030 s!and
    18097 s    736 e!and
    14166 d    633 d!and
    10477 t    574 r!and
     9366 n    501 t!and
     8464 r    436 y!and
     8128 h    389 n!and
     6821 y    378 h!and
     5406 f    303 g!and
     4493 o    200 l!and
     3807 g    180 k!and
     3411 l    136 m!and
     2127 a     54 b!and
     1770 m     49 o!and
     1448 k     48 p!and
      858 p     45 f!and
      773 w     38 w!and
      571 b     24 a!and
      284 i     23 x!and
      231 u      3 u!and
      223 x           
      110 °           
        9 v           
        3 z           
        2 '           
        2 c

As you can see, the word before an 'and' is almost three times more likely to end with "s" than with "e"; whereas, in general, a word is slightly more likely to end with "e" than with "s". 

Is that because of some mysterious rule that makes the word 'and' attract the letter 's'?  Not quite.  Here are the counts for the words that occur before the word 'and':

   262 vertues!and
    95 bruised!and
    87 wine!and
    83 liver!and
    79 them!and
    78 long!and
    62 july!and
    56 leavs!and
    52 stomach!and
    51 june!and
    50 courses!and
         ...
     1 yonger!and
     1 zedoary!and

(1645 different word types in all).  

While the count of 2030 for "-s" before the word "and" is due to 494 different word types, note that 262 of those occurrences (13%) are due to a single word type, "vertues", that is often followed by "and" -- because, in the entry of every plant, there is a sub-entry titled "Vertues and use" or "Vertues and uses".

On the other hand, the most common word types overall in that text are

 10158 the
  6780 and
  4362 of
  3045 in
  2274 or
  2242 to
  2213 it
  2009 is
  1924 a
  1267 with
   ...

Yet there is not even one occurrence of "the and" or "and and", and only a handful of "of and", "in and".

Thus it is not that the word "and" attracts the ending "-s"; rather, the word "and" attracts certain word types, like "vertues", and it just happens that many of these words end in "-s".

Quote:If y is a pitch marker, the model still has to explain why the material before y changes the following qo probability so strongly.  what is the concrete mechanism that connects the internal pre-y form with the next token’s qo?

Same answer.  

Saying "the internal pre-y form is connected with the next token’s qo" is an obfuscated way of saying "the frequency distribution of words before words that begin with qo is different from the frequency distribution of words in the text as a whole".  

In particular, the former distribution has more words that end in -y than the latter.  Why?  For the same reason that, in Culpeper's book, words that end in "-s" are more common before "and" than in the text as a whole.  Basically, "just because". 

All the best, --stolfi

@JoeyB, the problem is not picking the highest quality LLM for Standard Chinese, the main written language of China, but tools for addressing the spoken languages. Because Cantonese is not typically written---most Cantonese speakers never read or write in it---an LLM that generates Cantonese is not a much sought after product, and the corpuses forming the training data are not large at all. Likewise, because writing in Classical Chinese is still in living memory (if barely), it casts a huge shadow over written Standard Chinese, and there is a parallel problem with Cantonese texts, that mean that a good deal of vocabulary that you would never learn through the oral/aural route is in the training data. We also need a human being who can judge the output credibly, and while I can spot some obvious problems with what has already been offered, I am unable to make close calls.

(25-06-2026, 09:24 PM)rikforto Wrote: You are not allowed to view links. Register or Login to view.So, I at least now understand where the misunderstanding is. To use the Suassarian formulation, Chinese characters are not the signified, they are the signifier. So in any given language it's not that a character represents the lexeme of "house, home, dwelling, abode" it's that it represents one morpheme. ... And if the passage is Latin, not English, they will probably scoff at the idea that it says "house" even if the reading exists in an an English character dictionary and they are correct because only an idiot would think a Latin passage had the signifier "house". ...

Sorry, but you still are hopelessly confused about the situation.

In most scripts, indeed the glyphs (or digraphs, trigraphs, etc) represent sounds, and then combinations of sounds represent concepts.   That is because most scripts (even English!) are basically phonetic.  And both steps are different for each language.

But that is not true for the Chinese script -- because it is not phonetic!  In Chinese writing, each symbol (or pair of symbols) represents directly a concept.  Each symbol also represents a sound; but it represents a different sound for the speakers of different languages, while the concept is essentially the same for all Sinitic languages (and even, to some extent, for some non-Sinitic ones). 

[attachment=16201][attachment=16202]

In every language, the mapping between sounds and concepts is a table with 50'000 or more entries, which is mostly learned at home in infancy, before age 10 or so.  That table of course is different for each language.

In the case of phonetic scripts, the mapping from glyphs to sounds is relatively simple and is learned in the first year of grammar school. (Except perhaps for English, where one must memorize the pronunciation of every word, even though the script is still mostly phonetic.)

In the case of the Chinese script, in contrast, learning the mappings from glyphs to sounds and from glyphs to concepts requires memorizing another table with 5'000 entries or so. That learning normally takes on the order of 10 years at school.  (In theory one would have to learn only one of the two mappings, but in practice details like homophones and synonyms mean that the two tables must be learned together.)  Again, while the mapping from glyphs to sounds is different for each language, the mapping from glyphs to concepts is very much the same for all Sinitic languages.

This common writing system is a big advantage for communication.  Proper translation between spoken or written Turkish and spoken or written Lithuanian must go through the concepts; and since concepts live only in one's head, if the two persons who need to communicate know only their respective languages, the translation requires a third person who is fluent on both languages.

Whereas for translation between Mandarin and Cantonese there is the option of going through the glyphs rather than the concepts.   That is, the monolingual Mandarin speaker writes the message hanzi, and the monolingual Cantonese speaker reads that message -- without the need for an interpreter.

And this advantage exists also with other ideographic scripts used in the West, such as Arabic (or Roman) numerals, chemical formulas, and metric units.  The Finnish shopkeeper who must tell the price of a stuffed reindeer to a Maltese tourist can either use the services of a guide who knows both languages, or write the price on a slip of paper and show it to the tourist.  But only the first option is available when he must warn the tourist to not try to fly off a roof riding on that souvenir. 

Quote:For instance, all of "maison", "domus", and "casa" would have different characters, and all those characters would be in all three of French, Latin, and Spanish because all three of those morphemes exist in each language.

"Morpheme" here is the wrong term.  A morpheme is a part of a word that has an identifiable meaning, even if fuzzy and vague.  In "antifungals" the "anti-" is a morpheme with the general sense of "contrary opposite", "-fung-" is a morpheme (the "root" one) with the vague meaning "fungus", "-al-" is a morpheme with the vague meaning "related to", and "-s" is a morpheme that indicates plural nouns.  Whereas "antifungals", "an", "tifu", "fun", and "gals"  are not morphemes. (Even though "fun" elsewhere has a meaning, the "fun" in "antifungals" does not.)

Quote:an English speaker would have "house" available as a reading on some Romance word---probably the character for <domus> thinking about the history of Latin in England [...] And if the passage is Latin, not English, they will probably scoff at the idea that it says "house" even if the reading exists in an an English character dictionary and they are correct because only an idiot would think a Latin passage had the signifier "house".

No, no, no.  

Caesar may have though of the sentence in Latin, but what he wrote in ideograms was not Latin. That written sentence still had a bit of the Latin syntax (the specific word order),but no Latin words at all.  The drawing of a house did not represent the word "domus"; it represented the concept of a house.

Quote:an English speaker educated in Latin logographs for writing English but not Latin would have some clues....

Quote:domicile military regent fenestra  rotund   have

That reading would be totally silly for a number of reasons.  First, the English native would almost certainly not know Latin.  He would not even know that whoever wrote that sentence first thought of it as Latin. Even if he knew, he would not know which English words are derived from Latin words --- that "house" does not but "domicile" does, etc..

And the Martians did not know any of that either.  They knew English; so, for the exercise to make any sense, the native would have to read the symbols in English.

And, finally, that reading above would be wrong, because it would not mean what the sentence was supposed to mean.  The concept of "domicile" is not quite the same as house, a "regent" is definitely not a king, "fenestra" is not an English word, "rotund" means plump or fat rather than round...  Caesar did not mean the regent's soldier, or that the window should eat less and do some exercise...

And so I must insist: THE ETYMOLOGY OF A WORD IS ABSOLUTELY TOTALLY UTTERLY IRRELEVANT TO ANYONE IN ANY SITUATION except to linguists studying the history of languages.  Apologies for shouting...

All the best, --stolfi

(26-06-2026, 04:12 PM)Jorge_Stolfi Wrote: You are not allowed to view links. Register or Login to view.But that is not true for the Chinese script -- because it is not phonetic!  In Chinese writing, each symbol (or pair of symbols) represents directly a concept.  Each symbol also represents a sound; but it represents a different sound for the speakers of different languages, while the concept is essentially the same for all Sinitic languages (and even, to some extent, for some non-Sinitic ones). 

I'm not sure if it's relevant to your argument in the end, but this seems like a simplification. While I'm no expert in Chinese, I'm reminded of You are not allowed to view links. Register or Login to view. which I had a passing interest in once upon a time as a student of Japanese.

(26-06-2026, 05:44 PM)igajkgko Wrote: You are not allowed to view links. Register or Login to view.I'm not sure if it's relevant to your argument in the end, but this seems like a simplification. While I'm no expert in Chinese, I'm reminded of You are not allowed to view links. Register or Login to view. which I had a passing interest in once upon a time as a student of Japanese.

That article seems to be straining to make a distinction between "lexographic" (a rare term?) and "ideographic". 

Indeed Chinese characters were not created by a society of Silent Monks.  They were not invented to represent some pure ideas out of nothing, but rather the chunks of ideas that were associated to certain words of the language spoken by their creators.  That partition of the "idea space" and the writing evolved together for millennia, resulting in the current situation.   Namely, the hanzi 人 represents the fuzzy set of ideas that are evoked by the Mandarin word rén or the Cantonese word jan4.  Which is very roughly the same set of ideas that an English speaker thinks of on reading or hearings the word "person".

But it is quite misleading to say that the hanzi "represent words, not ideas".  The hanzi 人 does not represent any specific string of sounds.   Presumably it did when people started writing the symbol that evolved into this glyph; but that word and its language have died long ago and do not matter any more.  And probably that language had tens of thousands of words that never received a written symbol. 

Each Sinitic language has settled on a different mapping of hanzi to spoken words (syllables).  But the carving of the "idea space" into lexical chunks associated with the hanzi has become practically the same for all Sinitic languages.  That is what makes written Chinese an effective means of communication between speakers of different Sinitic languages; and this use is what has stabilized that "mental geography" against the natural tendency of languages to diverge over time.  

The description by Matteo Ricci (and MANY other who have had extensive experience in China) is still quite valid:

Quote:the Chinese have a system of writing similar to the hieroglyphic signs of the Egyptians ... they do not express their concepts by writing, like most of the world, with a few alphabetic signs, but they paint as many symbols as there are words. ... each word has its own hieroglyphic character, ... there are no fewer symbols than words.

Except of course that we now know that the Egyptian hierglyphs were mostly phonetic, not ideogaphic or "lexographic".

The "mental map" whereby ideas are divided into "lexical chunks" was clearly not stabilized  in the European languages. Even the Romance languages suffered substantial and independent "redistricting" of the lexical concepts since they split off Vulgar Latin. The meaning of single word of French often overlaps partly with the meanings of several non-equivalent words of Italian, and vice-versa: "bleu" can be "blu" or "azzurro", "nipote" can be "petit-fils", "petite-fille", "neveu", or "nièce" -- and there are many,many more examples.

All the best,--stolfi

(26-06-2026, 04:12 PM)Jorge_Stolfi Wrote: You are not allowed to view links. Register or Login to view.But that is not true for the Chinese script -- because it is not phonetic!  In Chinese writing, each symbol (or pair of symbols) represents directly a concept. 

It is not that simple. There is a large group of characters which consist of a radical plus a sound element.
At some point in time, for some version of the language, these sounds helped define the character set. Someone better versed in the history of the language may be able to say more about that.

(26-06-2026, 10:17 AM)Jorge_Stolfi Wrote: You are not allowed to view links. Register or Login to view.Is that because of some mysterious rule that makes the word 'and' attract the letter 's'?  Not quite.  Here are the counts for the words that occur before the word 'and':
   262 vertues!and
    95 bruised!and
    87 wine!and
    83 liver!and
    79 them!and [...]

This is an interesting case, but let's consider another one.

I just ran some quick statistics on Culpeper's Herbal myself that could surely be improved on with more care and effort but should be at least roughly correct.

The most common final characters of words overall:

[e] 18.6%
[s] 15.7%
[d] 10.5%
[t] 8.7%
[n] 7.3%

That matches the top five you cited.

Meanwhile, the most common final characters of words followed by the word "are":

[s] 36.1% -- over a third
[y] 21.5% -- the first two options cover over half of all cases
[t] 12.7%
[e] 7.8%
[d] 6.1%

In the latter set, the final character [s] is heavily over-represented because it's the plural marker.  By contrast, the final character [y] is due almost wholly to its occurrence in the pronoun "they."

For your "and", the percentage of [s] at end of preceding words is 26.1% -- still higher than average, though not by as much.  This appears to be due largely to several common grammatical structures containing "and": sequences of nouns including plurals ("ponds and ditches"), sequences of verbs in third person singular ("spreads and creeps"), etc. Not as good a story, but also not "just because."
 
Neither your example nor mine shows quite the kind of word profile we actually find around word-break combinations with skewed statistics in the Voynich Manuscript, in my experience. Importantly, I don't recall seeing anything like your "vertues" or my "they" that suggests whole words are driving the patterns as opposed to individual morphological elements -- glyphs, bigrams, and such.

But even if we look just at Culpeper, that 36.1% for [s] before [are] seems to demonstrate that the kind of statistic we're considering here can be meaningful and revealing.