[split] Percentage of word types that occur more than once

[Emma May Smith]

Thanks to RobGea for the post with verifiable numbers.

It confirms that, at least for this particular statistic, the Voynich MS is within the range of existing reference texts.

The Pliny reference is very interesting and good potential point of comparison, but Latin is a heavily inflected language, which increases the number of unique words. As far as I know, all attempts to find inflection in Voynichese have resulted in failure however.

The number of one token words will vary depending on whether the vocabulary has been lemmatised or not. Which is why we see lower figures for English than for Latin, because one has more inflection, as you note. We're safer working with non-lemmatised vocabularies as we don't know about this feature in the Voynich text. You could simulate lemmatisation by, for example, counting all words beginning [q] as being variants of the same word with [q] removed.

But as Rene said, the number of words only occurring once isn't outside the bounds of natural language. Only if we could prove that the Voynich language was non-inflected might this raise a concern. I don't know how we could prove such a thing at our current state of knowledge.