08-07-2026, 03:46 PM
(This post was last modified: 08-07-2026, 04:15 PM by Jorge_Stolfi.)
(08-07-2026, 11:17 AM)Vuk88 wrote: You are not allowed to view links. Register or Login to view.Jorge_Stolfi wrote:What do you mean by 1-to-1 compression? Treating each of my "elements" as a single character?This is the exact snippet of the compression function I used:
But in my model each bench or gallows element may have an integral e modifier. Here is my element parsing code in gawk:
Code:
#! /usr/bin/gawk -f
# Last edited on 2026-01-16 10:01:35 by stolfi
# To be included in gawk programs
function parse_word_into_elements(wd, we,punct,ok,el) {
# Parses {wd} into valid elements surrounding them by "{}"
# Glyphs that cannot be parsed are wrapped in "{}" with
# a "!" prefix
we = ""
while (wd != "") {
ok = 1; punct = 0
if (match(wd, /^[-,.«=»]/)) { punct = 1 }
else if (match(wd, /^[<][%$][>]/)) { punct = 1 }
else if (match(wd, /^[qoaydrl]/)) { }
# Resolving @e ambiguities:
else if (match(wd, /^[cs]h[e][e]/)) { RSTART = 1; RLENGTH = 2 }
else if (match(wd, /^ee[e][e]/)) {RSTART = 1; RLENGTH = 2 }
else if (match(wd, /^[kt][e][e]/)) { RSTART = 1; RLENGTH = 1 }
else if (match(wd, /^[c][kt]h[e][e]/)) { RSTART = 1; RLENGTH = 3 }
else if (match(wd, /^[c][pfwz]h[e][e]/)) { RSTART = 1; RLENGTH = 3 }
# Non-ambiguous cases:
else if (match(wd, /^[cs]h[e]?/)) { }
else if (match(wd, /^ee[e]?/)) { }
else if (match(wd, /^[kt][e]?/)) { }
else if (match(wd, /^[pfwz]/)) { }
else if (match(wd, /^[c][kt]h[e]?/)) { }
else if (match(wd, /^[c][pfwz]h[e]?/)) { }
else if (match(wd, /^[s]/)) { }
else if (match(wd, /^@?@?@?n/)) { }
else if (match(wd, /^@?@?m/)) { }
else if (match(wd, /^@@?[r]/)) { }
else { ok = 0; RSTART = 1; RLENGTH = 1 }
if (RSTART != 1) { prog_error("RSTART") }
el = substr(wd, RSTART, RLENGTH)
wd = substr(wd, RSTART + RLENGTH)
if (ok) {
if (punct) {
we = ( we el )
} else {
we = ( we "{" el "}" )
}
} else {
we = ( we "!{" el "}" )
}
}
return we
} The 'match' function of gawk sets START and RLENGTH to the start index (from 1) and length of the matched string. Thus if 'wd' starts with "q", "o", "d", etc the call match(wd, /^[qoaydrl]/) will set both to 1. If 'wd' starts with "chee" or "shee", the line 'if (match(wd, /^[cs]h[e][e]/)) { RSTART = 1; RLENGTH = 2 }' will select only the "ch" or "sh" and leave the "ee" for the next element. If it starts with "che" but not "chee", the following lines will select the whole "che".
Quote:If I understood correctly... from what I read, a language like Vietnamese builds its short words using strict and unchangeable slots in this way: [Initial Consonant] + [Vowel Nucleus] + [Tone Tag].
It depends on the spelling system used. This is a sample of Vietnamese in VIQR, a spelling system that was used for email, bboards, etc., when those applications would probably break anything that was not plain ASCII:
- DDu+'c Ye^su Kito^ sinh ra the^' na`y: Maria, me. Nga`i dda~ ddi'nh ho^n vo+'i
- Yuse; tru+o+'c khi o^ng ba` pho^'i ho+.p cu`ng nhau, thi` xa?y ra la` ba` dda~ co'
- thai do tu+. Tha'nh Tha^`n.
Quote:If we apply this to the VMS, maybe your final characters like "e" or "y" are simply filling that specific final 'Tone Tag' or 'Grammatical Particle' slot of the syllable. Is this the kind of architecture you have in mind?
I guess that the VMS e modifier looks more like a script convention chosen by the Author to encode more sounds than the basic set of glyphs would allow. Like the "h" in English is combined with "c", "s", and "p" to make the unrelated sounds "ch", "sh", and "ph". Or the "e" in German is combined with "a", "o", and "u" to make the distinct sounds "ae" = "ä", "oe" = "ö", and "ue" = "ü".
Since the Voynichese script was created from scratch, it would make sense for the Author to use a modifier symbol or accent to represent the voiced version of an unvoiced consonant. Like the "dakuten" ("voiced mark") accent turns unvoiced con Japanese Hiragana か = ka into が = ga. Thus maybe k = /p/, t = /f/, ch = /s/, etc, but ke = /b/, te = /v/, che = /z/, etc.
Quote:it's possible that the final combination "de" (even though it's statistically very rare, just like some specific Asian grammatical particles are) could be explained on a morphological level, without having to assume it's a distracted error?
First, treating uncommon cases as errors is actually healthy, because it lets one see the important rules and phenomena more clearly in the statistics. The uncommon cases can be analyzed later, after the common ones have been figured out.
Maybe Voynichese has inflections (or postpositions, like the Japanese -は and -を, or morphological suffixes like the -儿 of Mandarin). But I would expect them to be syllables, at least some of the time; in that case they would not fit in the word structure models. And all of those examples occur quite frequently in prose texts.
I think de is too rare to be an alternative for the last slot of the syllable. In monosyllabic languages, the number of valid syllables is usually small compared to the size of the lexicon, thus a large fraction of them have meaning. Therefore every valid alternative for each slot gets used in several distinct syllables, so it is unlikely that some alternative will occur only 49 times in ~10'000 syllables.
Moreover, the dy alternative for that slot is very common; why would de be avoided there?
Considering that my "a priori" probablility of errors is rather high, Bayes tells me that de is most likely a blind dy that ran after that farmer's wife with the carving knife.
All the best, --stolfi