12-05-2021, 05:58 PM
(12-05-2021, 12:06 PM)ReneZ wrote: You are not allowed to view links. Register or Login to view.Coming back to the opening post, it is also possible to plot, instead of the Hapax ratio, the absolute number vs. the number of word tokens. This is done below, first for word types vs. word tokens, and then for hapax vs. word tokens.
We see that for both quantities, the values decrease for increasing number of spaces, i.e. increasing nr. of word tokens.
This is counter-intuitive, and tells us something (but what?) about the Voynich text.
I think that the decrease in word-types and hapax when more spaces are considered is due to the presence of more short words. Long words are almost always hapax, while splitting a Voynichese word in two often results in two valid words.
This could be connected with the method that Wentian Li (Random Texts Exhibit Zipf’s-Law-Like Word Frequency Distribution, 1992) used to produce his Zipfian "random" text: instead of just considering the space as the other characters he had to "introduce a cutoff of the maximum possible word length L-max". Without the cutoff, you get a huge number of hapax.