More rigorously testing the hoax hypothesis

[MarcoP]

My own brief thoughts have been along thought experiments like the following:

1) Take a known plain text, which can be quite long
2) Create a list of words that appear in this text, according to decreasing frequency
3) Put the list of Voynichese words sorted by descreasing frequency next to it
4) Substitute the plain text words by Voynichese words according to this table .

This results in a text using valid Voynichese words that has a meaning.

Now the question: how can we detect that this has meaning?  ( = Question 1)

A possible approach to this problem was described by Tony C. Smith & Ian H. Witten ("Language inference from function words", 1993, mentioned You are not allowed to view links. Register or Login to view.). The paper is old enough to still be understandable (nothing as powerful and obscure as GPT). Basically, the 1% most frequent word types in a longer text are most likely function words. Function words belong to closed classes, i.e. their number is finite and small (new pronouns, articles, conjunctions etc. only appear through the centuries, in the context of the historical shift from a language to the other). One can start from here and iteratively cluster word types into grammatical clusters (e.g. functionWords-> pronouns-> singularPronouns-> masculineSingularPronouns-> accusativeMasculineSingularPronoun [=him]) and at the end maybe define a possible mapping to words in a source language and give a positive answer to Question 1.

I know that most people on this forum are aware of this, but maybe it is worth mentioning - the basics of Smith-Witten's method are enough to show that Voynichese does not conform to Rene's thought experiment:

1. Each section has its own set of 1% most frequent words; the change in lexicon appears to be a continuous drift and the extreme Currier-A and Currier-B sections (HerbalA and Bio/Q13) only show a minimal intersection in the sets of most frequent words (see You are not allowed to view links. Register or Login to view.), while in all languages function words tend to be uniform in different texts.
2. In all sections, many of the most frequent words often appear consecutively (e.g. "daiin daiin" in HerbalA, "chedy chedy" in BioQ13) and this is not the case for any European language.


Another way to tackle this subject is MATTR (a method amply discussed by Koen and more recently by Luke Lindemann at the Malta conference). The though experiment does not affect MATTR (just like a simple substitution does not affect conditional character entropy). A text created through this method will perfectly match MATTR values from the source language. But, if you take small MATTR windows, Voynichese does not fit much (see You are not allowed to view links. Register or Login to view.).