More rigorously testing the hoax hypothesis

[ReneZ]

This is a complex question, and I cannot do it justice with just a short post here.

Your (Daniel) question can be interpreted in two ways:
- how could the experiment you did be improved to be more rigorous
- how can one generally decide whether any given text is meaningful

I guess that you are open to both.
As the paper already points out, a main shortcoming of the experiment was the limited length of the generated texts, which is of course more or less inevitable.
 
However, it is an important aspect that the Voynich MS text is very long, and it is consistent in many respects. It is not certain, but I consider it extremely likely, that the alphabet (or the writing system as a whole) was designed specifically for this book. This then shows that there was a singificant amount of planning and intention behind it.
Thus, if we cannot yet decide that it has meaning, I dare say that it is the product of intention, whatever that is worth.

A linguist from the earlier days of the internet, Jacquest Guy, has given this matter considerable thought, and he came to the conclusion, as several posters above, that we cannot decide if a text has meaning.

My own brief thoughts have been along thought experiments like the following:

1) Take a known plain text, which can be quite long
2) Create a list of words that appear in this text, according to decreasing frequency
3) Put the list of Voynichese words sorted by descreasing frequency next to it
4) Substitute the plain text words by Voynichese words according to this table .

This results in a text using valid Voynichese words that has a meaning.

Now the question: how can we detect that this has meaning?  ( = Question 1)

Next, we do the same, but after step 1, we modify the meaningful text by scrambling the words arbitrarily. As a result, this new text has become meaningless.

This process results in a text using valid Voynichese words that has no meaning.

So question 2 is: how can we detect that this text has no meaning?

And question 3 is probably the better one: what difference between the two texts can we quantify, such that we can detect meaning?

Of course, this is a specific type of meaningless text, which can be converted to a meaningful text by a specific 'unscrambling' of the words.

For me, one way of looking at this question is: is it possible to create a dictionary of Voynichese to "any-known-language", such that a word for word substitution results in a meaningful text?
This is what almost all would-be solvers are tacitly assuming. It implies that the word spaces are indeed word spaces, which is another big question, and which is generally (tacitly) assumed.
Anyway, if the answer to the above question is yes, then the text is meaningful. One can just create the dictionary, and the whole question about word structure and entropy is swept under the carpet.
If the answer is no, it is still possible that the text is meaninful, but it becomes even more difficult.

Back to the earlier thought experiment, instead of fully scrambling the original text, one could do a relatively small number of word swaps over short distances, and the text might still remain meaningful. One may then start to wonder at which point meaning disappears, and come to the (obvious) conclusion that the concept of "meaning" is quite vague, even subjective.