- Options
- Login
- Desktop Style
Options
One funny word on f114r.30
- One funny word on f114r.30
-
RE: One funny word on f114r.30
Vuk88 > 08-07-2026, 05:09 AM
@You are not allowed to view links. Register or Login to view.
Since I’m currently working on some analyses that might prove interesting once all the control tests are done, I tested your Core Alphabet to clean up the IVTFF corpus.
The 1-to-1 compression turned out to be very effective in removing the background noise.
Out of curiosity, I ran a script to calculate the Greenberg Synthesis Index on the cleaned text (basically, the algorithm calculates how many "pieces" or morphemes make up an average word). I got a score of 2.86.
If my calculations are correct, that score means the text "could" be compared to a highly agglutinative or even polysynthetic language, where words are built by concatenating numerous small suffixes to one another.
This result made me reflect on your estimate of a 10% error rate. What if lkeede is not a spelling mistake for lkeedy at all? If the language "glues" concepts together, perhaps your core character e and your core character y aren't just simple orthographic variants, but actual grammatical suffixes? For instance, hypothetically speaking, one could indicate the plural, and the other a verb tense.
If we look at the final characters as specific morphological tags rather than transcription errors, maybe the scribe wasn't making mistakes at all.
Furthermore, the hypothesis of an "agglutinative language" would perfectly explain the well-known anomaly of "Word Doubling".
Forgive me if I may, but to me, this data looks very similar to the example you yourself brought up regarding ancient Chinese words, where the repetition "pain pain" means "much pain"… Now, even though I am not a linguist, through a simple web search I noticed that European inflectional languages tend to suppress the immediate repetition of words. But Asian or Austronesian languages (like Chinese or Indonesian) use it constantly to form plurals or to intensify meaning.
To have a counter-proof, I had over 30,000 words from the Indonesian Wikipedia analyzed (removing the hyphens to mimic the raw orthography of the Voynich). The real doubling rate turned out to be 2.06 times higher than chance. This is a value very close to the rate (about 2.2x) that I calculated by running my stochastic models on the IVTFF corpus. In practice, your intuition about Chinese seems to be mathematically supported by the data.
If this were really the case, is it possible that this isn't a text full of distracted scribal errors, but simply a non-European language that makes extensive use of suffixes and reduplication?
I would be very interested to know if you think your alphabet could work by interpreting the word endings as micro-morphemes.
Alfredo -
RE: One funny word on f114r.30
Jorge_Stolfi > 08-07-2026, 09:34 AM
(08-07-2026, 05:09 AM)Vuk88 Wrote: You are not allowed to view links. Register or Login to view.Since I’m currently working on some analyses that might prove interesting once all the control tests are done, I tested your Core Alphabet to clean up the IVTFF corpus.
The 1-to-1 compression
You mean what I originally called the "crust-mantle-core" model of word structure? (It has by now evolved into a "7-slot plus OKOKO model", Q^q D^d X^x G^g H^h X^y D^e N^n with q+d+e+n and x+h+y in 0..3, g+h in 0..1, with at most three O elements inserted before, between, and after the other elements). What do you mean by 1-to-1 compression? Treating each of my "elements" as a single character?
Quote:that score means the text "could" be compared to a highly agglutinative or even polysynthetic language, where words are built by concatenating numerous small suffixes to one another.
Perhaps, but in the agglutinative languages I know about the morphemes are often whole syllables, so the words end up being (1) rather long, and (2) with clear alternation between "consonants" and "vowels". Which are known non-features of the VMS. (Jacques Guy may have been the first to run Sukhotin's V-C identification algorithm on the VMS, back in the late 1990s, and it did not work.).
The symbol repertoire seems to be too small for them to stand for syllables, but just right for them to stand for individual phonemes or other simple phonetic features. All the word structure models, including mine and Zattera's, allow for a fixed number of slots, where each slot may be filled by a small number of alternative glyphs of glyph pairs, or left empty. This is very much like the structure of syllables in some languages like Cantonese or Vietnamese.
Quote:This result made me reflect on your estimate of a 10% error rate. What if lkeede is not a spelling mistake for lkeedy at all? If the language "glues" concepts together, perhaps your core character e and your core character y aren't just simple orthographic variants, but actual grammatical suffixes?
My argument for lkeede being an error is that the {de} combination is rare compared to {d} (except in tokens that can be parsed as ...{d}{ee}... or ...{d}{eee}...); whereas {ke}, {te}, {che}, {she}, {eee}, {cthe}, and {ckhe} seem to be common compared to the versions without the final {e} (with the same caveat). In my transcription {d} occurs 13194 times, while {de} only 49 times; whereas {k} occurs 10962 times, and {ke} 1787 times. And also {d}{y} is a common combination (6952 occurrences), and {e} can be an {y} that lost its tail...
Quote:Furthermore, the hypothesis of an "agglutinative language" would perfectly explain the well-known anomaly of "Word Doubling". ... If this were really the case, is it possible that this isn't a text full of distracted scribal errors, but simply a non-European language that makes extensive use of suffixes and reduplication?
I doubt it. Agglutinative languages are rather similar to European ones in their use of suffixes, just allow a lot more of them in the same word. (Italian can have "portandocelo" = "port-" + "-ando-" + "-ce-" + "-lo" or "compartimentazioni" = "com-" + "-part-" + "-imenta-" + "-zion-" + "-i" ...) The same mechanism that seems to suppress doubling in European languages presumably would apply to them too.
Quote:But Asian or Austronesian languages (like Chinese or Indonesian) use it constantly to form plurals or to intensify meaning.
Not just for intensification or plural. IIUC, in modern Mandarin 刚 alone generally means "hard", but 刚刚 means "exactly" or "just now".
And indeed, IIRC, Vietnamese uses repetition more often than Mandarin.
I suppose that this propensity to duplication follows from those languages being monosyllabic and non-inflecting. Thus their use of duplication may be more appropriately compared to the occurrence of duplicate syllables in European words ("abitata", "adoreremo", "allevava", "bibita", "candidi", "cocomero", "cuculo", "demagogo", "filologo", "nono", "panini", "papa", "pensoso", "pepe", "popolo", "rara", ...)
All the best, --stolfi -
RE: One funny word on f114r.30
Vuk88 > 08-07-2026, 11:17 AM
Jorge_Stolfi Wrote:What do you mean by 1-to-1 compression? Treating each of my "elements" as a single character?
@Jorge_Stolfi,
Regarding the 1-to-1 compression, yes, I meant exactly treating each of your "elements" as a single character. Practically speaking, before passing the text to any entropy scripts, I applied a Python mapping to assign a unique ASCII character to each complex element, so the algorithm wouldn't split up your "core elements". This is the exact snippet of the compression function I used:
Code:def compress_to_atomic(w):
w = w.replace('cth', 'T')
w = w.replace('cph', 'P')
w = w.replace('ckh', 'K')
w = w.replace('cfh', 'F')
w = w.replace('ch', 'C')
w = w.replace('sh', 'S')
w = w.replace('iiin', 'W')
w = w.replace('iin', 'M')
w = w.replace('ee', 'E')
return w
Jorge_Stolfi Wrote:Perhaps, but in the agglutinative languages I know about the morphemes are often whole syllables, so the words end up being (1) rather long... This is very much like the structure of syllables in some languages like Cantonese or Vietnamese.
Regarding your observation about agglutinative languages, I believe you are right about the analogy with Cantonese and Vietnamese. After reading your post, I went online to read up on the syllable structure of those languages and I realized my conceptual mistake. Polysynthetic languages have huge words, while VMS "words" are notoriously very short.
When my algorithm calculated an internal breakdown index of ~2.86 for each word in Voynichese, I mistakenly used the term "polysynthetic". Actually, in light of your point, the algorithm was just mathematically detecting the presence of roughly 3 rigid "slots" inside a short word. This fits perfectly with your slot model and with the structure of Isolating / Monosyllabic languages.
If I understood correctly... from what I read, a language like Vietnamese builds its short words using strict and unchangeable slots in this way: [Initial Consonant] + [Vowel Nucleus] + [Tone Tag]. If we apply this to the VMS, maybe your final characters like "e" or "y" are simply filling that specific final 'Tone Tag' or 'Grammatical Particle' slot of the syllable. Is this the kind of architecture you have in mind?
Jorge_Stolfi Wrote:I suppose that this propensity to duplication follows from those languages being monosyllabic and non-inflecting. Thus their use of duplication may be more appropriately compared to the occurrence of duplicate syllables in European words ("abitata", "papa", "cocomero"...)
If we view Voynich "words" as single syllables with fixed slots, then the math I pulled on word doubling makes a lot more sense. My script that measured the 2.2x doubling rate on the VMS doesn't calculate the repetition of internal syllables (like in the Italian word 'pa-pa' or 'cu-cu-lo'), but calculates the repetition of entire text tokens separated by spaces. If the VMS were a European inflectional language, the doubling of consecutive space-separated tokens would be drastically suppressed by statistical models. Instead, it behaves statistically almost identical to an Isolating/Austronesian language (like Indonesian) that uses full-word morphological reduplication to simulate grammar.
If we accept your intriguing hypothesis that the VMS has the slot structure of an Asian/Isolating language, do you think it's possible that the final combination "de" (even though it's statistically very rare, just like some specific Asian grammatical particles are) could be explained on a morphological level, without having to assume it's a distracted error?
Alfredo -
RE: One funny word on f114r.30
Mauro > 08-07-2026, 01:26 PM
(08-07-2026, 05:09 AM)Vuk88 Wrote: You are not allowed to view links. Register or Login to view.If my calculations are correct, that score means the text "could" be compared to a highly agglutinative or even polysynthetic language, where words are built by concatenating numerous small suffixes to one another.
Actually, prefixes, infixes and suffixes, everything is possible. But one also needs roots! and this is the problem with Voynichese: after removing prefixes and suffixes most of the times nothing remains.
(08-07-2026, 05:09 AM)Vuk88 Wrote: You are not allowed to view links. Register or Login to view.This result made me reflect on your estimate of a 10% error rate. What if lkeede is not a spelling mistake for lkeedy at all? If the language "glues" concepts together, perhaps your core character e and your core character y aren't just simple orthographic variants, but actual grammatical suffixes? For instance, hypothetically speaking, one could indicate the plural, and the other a verb tense.
If we look at the final characters as specific morphological tags rather than transcription errors, maybe the scribe wasn't making mistakes at all.
This has been proposed many times, up to Voynichese being a You are not allowed to view links. Register or Login to view.. The problem is fleshing this idea out.
(08-07-2026, 05:09 AM)Vuk88 Wrote: You are not allowed to view links. Register or Login to view.Furthermore, the hypothesis of an "agglutinative language" would perfectly explain the well-known anomaly of "Word Doubling".
Well, no. A language being agglutinative is one thing, while having productive word reduplication is a different thing. Neither implies the other one. -
RE: One funny word on f114r.30
Vuk88 > 08-07-2026, 02:46 PM
Mauro Wrote:Actually, prefixes, infixes and suffixes, everything is possible. But one also needs roots! and this is the problem with Voynichese: after removing prefixes and suffixes most of the times nothing remains.
...That's only true if we keep looking at it through the lens of someone desperately searching for a spoken European language. But what if we try to look at it as a compressed data structure or a cipher? In such a system, a "root" doesn't necessarily need to be 3 or 4 letters long to trigger a word with complete meaning. Sometimes a single character in the middle (like a gallows) is enough to act as a pointer or a category index. Why do we have to fixate solely on classical phonetic linguistics? I really don't get this "therapeutic stubbornness", as if someone were forcing us to.
Mauro Wrote:This has been proposed many times, up to Voynichese being a philosophical language. The problem is fleshing this idea out.
That's exactly the point "fleshing it out". And that's precisely what I'm working on. I'm not just spitballing hypotheses here; I've started measuring this mathematically. From my data tests, a crazy asymmetry emerges: if you change a word's prefix, the mathematical probability of a specific suffix attaching (instead of another one) completely flips. There is a rigid interdependence between the beginning and the end of the word. I'm finishing up the data prep and will publish it shortly, but I can already tell you that the numbers show they aren't just gluing pieces together at random. There is an underlying grammar dictating strict rules. Can I count on your help to "stress-test" and try to break my data with your analytical rigor?
Mauro Wrote:Well, no. A language being agglutinative is one thing, while having productive word reduplication is a different thing. Neither implies the other one.
You're certainly the expert here, and I apologize for my "layman's" inaccuracies... from a strictly linguistic standpoint, the two things aren't necessarily linked. I misspoke. I didn't mean that agglutination causes reduplication. My comparison with Indonesian was only meant to show that both of these "anomalies" (suffix chains and doubled words) coexist perfectly fine in non-European languages. And they work seamlessly without having to drag in the usual excuse of a scribal error or a "distracted monk". There is conditional logic behind it... And if you ask me why Indonesian specifically, besides the fact that both indices—the VMS one and the Indonesian one—match up, my answer is that it's a natural language that uses reduplication as a standard grammatical rule (for instance, to form plurals: anak = child, anak-anak = children, or to intensify verbs). In European languages, repeating two identical words in a row is either a mistake or a statistical rarity (the rate is virtually zero). Instead, Indonesian scientifically proves that in a real, fully functional language, a reduplication rate around 2x higher than chance is completely normal and structured.
What do you think?
Alfredo -
RE: One funny word on f114r.30
Jorge_Stolfi > 08-07-2026, 03:46 PM
(08-07-2026, 11:17 AM)Vuk88 Wrote: You are not allowed to view links. Register or Login to view.
Jorge_Stolfi Wrote:What do you mean by 1-to-1 compression? Treating each of my "elements" as a single character?
This is the exact snippet of the compression function I used:
But in my model each bench or gallows element may have an integral e modifier. Here is my element parsing code in gawk:
(The "@" above is actually standing for an "i" in brackets. This "wonderful" forum software thinks that is italic markup, even when it is inside code fragment...)Code:#! /usr/bin/gawk -f
# Last edited on 2026-01-16 10:01:35 by stolfi
# To be included in gawk programs
function parse_word_into_elements(wd, we,punct,ok,el) {
# Parses {wd} into valid elements surrounding them by "{}"
# Glyphs that cannot be parsed are wrapped in "{}" with
# a "!" prefix
we = ""
while (wd != "") {
ok = 1; punct = 0
if (match(wd, /^[-,.«=»]/)) { punct = 1 }
else if (match(wd, /^[<][%$][>]/)) { punct = 1 }
else if (match(wd, /^[qoaydrl]/)) { }
# Resolving @e ambiguities:
else if (match(wd, /^[cs]h[e][e]/)) { RSTART = 1; RLENGTH = 2 }
else if (match(wd, /^ee[e][e]/)) {RSTART = 1; RLENGTH = 2 }
else if (match(wd, /^[kt][e][e]/)) { RSTART = 1; RLENGTH = 1 }
else if (match(wd, /^[c][kt]h[e][e]/)) { RSTART = 1; RLENGTH = 3 }
else if (match(wd, /^[c][pfwz]h[e][e]/)) { RSTART = 1; RLENGTH = 3 }
# Non-ambiguous cases:
else if (match(wd, /^[cs]h[e]?/)) { }
else if (match(wd, /^ee[e]?/)) { }
else if (match(wd, /^[kt][e]?/)) { }
else if (match(wd, /^[pfwz]/)) { }
else if (match(wd, /^[c][kt]h[e]?/)) { }
else if (match(wd, /^[c][pfwz]h[e]?/)) { }
else if (match(wd, /^[s]/)) { }
else if (match(wd, /^@?@?@?n/)) { }
else if (match(wd, /^@?@?m/)) { }
else if (match(wd, /^@@?[r]/)) { }
else { ok = 0; RSTART = 1; RLENGTH = 1 }
if (RSTART != 1) { prog_error("RSTART") }
el = substr(wd, RSTART, RLENGTH)
wd = substr(wd, RSTART + RLENGTH)
if (ok) {
if (punct) {
we = ( we el )
} else {
we = ( we "{" el "}" )
}
} else {
we = ( we "!{" el "}" )
}
}
return we
}
The 'match' function of gawk sets START and RLENGTH to the start index (from 1) and length of the matched string. Thus if 'wd' starts with "q", "o", "d", etc the call match(wd, /^[qoaydrl]/) will set both to 1. If 'wd' starts with "chee" or "shee", the line 'if (match(wd, /^[cs]h[e][e]/)) { RSTART = 1; RLENGTH = 2 }' will select only the "ch" or "sh" and leave the "ee" for the next element. If it starts with "che" but not "chee", the following lines will select the whole "che".
Quote:If I understood correctly... from what I read, a language like Vietnamese builds its short words using strict and unchangeable slots in this way: [Initial Consonant] + [Vowel Nucleus] + [Tone Tag].
It depends on the spelling system used. This is a sample of Vietnamese in VIQR, a spelling system that was used for email, bboards, etc., when those applications would probably break anything that was not plain ASCII:- DDu+'c Ye^su Kito^ sinh ra the^' na`y: Maria, me. Nga`i dda~ ddi'nh ho^n vo+'i
- Yuse; tru+o+'c khi o^ng ba` pho^'i ho+.p cu`ng nhau, thi` xa?y ra la` ba` dda~ co'
- thai do tu+. Tha'nh Tha^`n.
Quote:If we apply this to the VMS, maybe your final characters like "e" or "y" are simply filling that specific final 'Tone Tag' or 'Grammatical Particle' slot of the syllable. Is this the kind of architecture you have in mind?
I guess that the VMS e modifier looks more like a script convention chosen by the Author to encode more sounds than the basic set of glyphs would allow. Like the "h" in English is combined with "c", "s", and "p" to make the unrelated sounds "ch", "sh", and "ph". Or the "e" in German is combined with "a", "o", and "u" to make the distinct sounds "ae" = "ä", "oe" = "ö", and "ue" = "ü".
Since the Voynichese script was created from scratch, it would make sense for the Author to use a modifier symbol or accent to represent the voiced version of an unvoiced consonant. Like the "dakuten" ("voiced mark") accent turns unvoiced con Japanese Hiragana か = ka into が = ga. Thus maybe k = /p/, t = /f/, ch = /s/, etc, but ke = /b/, te = /v/, che = /z/, etc.
Quote:it's possible that the final combination "de" (even though it's statistically very rare, just like some specific Asian grammatical particles are) could be explained on a morphological level, without having to assume it's a distracted error?
First, treating uncommon cases as errors is actually healthy, because it lets one see the important rules and phenomena more clearly in the statistics. The uncommon cases can be analyzed later, after the common ones have been figured out.
Maybe Voynichese has inflections (or postpositions, like the Japanese -は and -を, or morphological suffixes like the -儿 of Mandarin). But I would expect them to be syllables, at least some of the time; in that case they would not fit in the word structure models. And all of those examples occur quite frequently in prose texts.
I think de is too rare to be an alternative for the last slot of the syllable. In monosyllabic languages, the number of valid syllables is usually small compared to the size of the lexicon, thus a large fraction of them have meaning. Therefore every valid alternative for each slot gets used in several distinct syllables, so it is unlikely that some alternative will occur only 49 times in ~10'000 syllables.
Moreover, the dy alternative for that slot is very common; why would de be avoided there?
Considering that my "a priori" probablility of errors is rather high, Bayes tells me that de is most likely a blind dy that ran after that farmer's wife with the carving knife.
All the best, --stolfi - DDu+'c Ye^su Kito^ sinh ra the^' na`y: Maria, me. Nga`i dda~ ddi'nh ho^n vo+'i
-
RE: One funny word on f114r.30
Jorge_Stolfi > 08-07-2026, 03:56 PM
(08-07-2026, 01:26 PM)Mauro Wrote: You are not allowed to view links. Register or Login to view.Actually, prefixes, infixes and suffixes, everything is possible. But one also needs roots! and this is the problem with Voynichese: after removing prefixes and suffixes most of the times nothing remains.
I don't think that the Voynichese words have morphological prefixes or suffixes (inflections or attached function words like the Arabic article, Japanese postpositions, Italian oblique pronouns, etc.).
However, in defense of that claim: just because a syllable or letter can be a suffix or prefix, it does not mean that it is always one of those things. Not every final "-mi", "-le", "-re", etc in Italian is a morpheme. So if you strip everything that could be a prefix or suffix from an Italian word, often there will be nothing left.
And in agglutinative languages the set of possible suffixes and prefixes is very large.
All the best, --stolfi -
RE: One funny word on f114r.30
Mauro > 08-07-2026, 04:43 PM
(08-07-2026, 02:46 PM)Vuk88 Wrote: You are not allowed to view links. Register or Login to view.
Mauro Wrote:Actually, prefixes, infixes and suffixes, everything is possible. But one also needs roots! and this is the problem with Voynichese: after removing prefixes and suffixes most of the times nothing remains.
...That's only true if we keep looking at it through the lens of someone desperately searching for a spoken European language. But what if we try to look at it as a compressed data structure or a cipher? In such a system, a "root" doesn't necessarily need to be 3 or 4 letters long to trigger a word with complete meaning. Sometimes a single character in the middle (like a gallows) is enough to act as a pointer or a category index. Why do we have to fixate solely on classical phonetic linguistics? I really don't get this "therapeutic stubbornness", as if someone were forcing us to.
Why do you say that? I even made the example of a philosophical language, which surely would not be a spoken European language and doesn't even need to have a phonetics. It does not matter which language you use or create ex-novo: you need roots to have something on which prefixes and suffixes can work. -
RE: One funny word on f114r.30
Mauro > 08-07-2026, 04:46 PM
(08-07-2026, 03:56 PM)Jorge_Stolfi Wrote: You are not allowed to view links. Register or Login to view.
(08-07-2026, 01:26 PM)Mauro Wrote: You are not allowed to view links. Register or Login to view.Actually, prefixes, infixes and suffixes, everything is possible. But one also needs roots! and this is the problem with Voynichese: after removing prefixes and suffixes most of the times nothing remains.
I don't think that the Voynichese words have morphological prefixes or suffixes (inflections or attached function words like the Arabic article, Japanese postpositions, Italian oblique pronouns, etc.).
My position is not so sharp: it's "I don't know"
. When I talked about prefixes infixes and suffixes I was referring to agglutinative (natural) languages in general, which can use all the three forms, not to the VMS (which I don't even know if it's a language or not).