The Voynich-Ms as a concatenation of abbreviations

[MarcoP]

Let us setup a thought experiment where an educated person created their own syllabary shorthand. If glyphs can stand for both common words/phrases and syllables, then we start to explain away a lot of the repetition we see in the text.

I don't see how a syllabary shorthand should result in the 1% word repetitions we see in the Voynich ms. Lindemann and Bowern (You are not allowed to view links. Register or Login to view., 2021) compared historical texts in modern "normalized" transcriptions and more faithful transcriptions that actually included the abbreviations used by scribes. They found that "abbreviated Latin has higher conditional character entropy than unabbreviated Latin": since Voynichese has a particular low conditional entropy (see also You are not allowed to view links. Register or Login to view.), it diverges from ordinary texts in the opposite way than actual Medieval abbreviated texts. If anything, the increased entropy of actual abbreviated texts suggests fewer repetitions, rather than more of them.


Moving from actual evidence to speculation, one can imagine a very "lossy" abbreviation system where several plain-text words are mapped into the same shorthand word. An extreme example is only keeping the first and last character of each word, so that 'the', 'tree' and 'tease' are all abbreviated as 'te'.
Applying this system to Mattioli (Latin) and the Genesis from King James Bible (English) results in a rate of repeated words similar to Voynichese in the case of Mattioli: ~1%. The "abbreviated" English Genesis stops at about half that rate.

But one should also note that repetitions in the Voynich manuscript exceed the number observed in a randomly scrambled version of the text: i.e. these repetitions do not appear to be accidental. In the case of the possible occurrence of "the tree" encoded as "te te", the pattern is purely accidental: there is nothing that makes the occurrence of two t*e words particularly likely. On the contrary, in the case of Voynichese, there is something that makes identical (and  very similar) words appear consecutively. This plot shows actual repetitions (Y) vs repetitions in the scrambled text (X). Voynichese samples appear above the y=x line.

   

Finally, this extreme shorthand also results in a very low MATTR. With a window of 1000 words, Voynichese shows a value close to 0.5, lower than Latin but higher than English, perfectly compatible with several European languages. But the extreme abbreviation that can result in Voynichese-like repetitions also reduces MATTR1000 to below 0.2, quite below Voynichese and actual written texts.

   

I know that this lossy shorthand is not what Helmut is proposing: it is just my guess at something that might seriously increase repetition (though not above the rate of a scrambled text). I am very curious to understand how the system discussed in this thread is supposed to produce the observed rate of repetition without significantly reducing MATTR.