Voynich Reconsidered

[kckluge]

I'm not asking rhetorically or as a "gotcha", but shouldn't you be normalizing your samples in terms of number of tokens in the text? As the number of tokens in a text increases, two things will happen: 1) additional types further out in the frequency tail will appear as new hapax, and 2) additional occurrences of prior hapax will appear, removing them as hapax. It's not obvious that those two effects should/do cancel each out. (An obvious test would be to compute the percentage of hapax types for the first N words in the Divine Comedy as a function of increasing N.)