Repetition of words

[Mauro]

If a language follows Zipf's law, the token/lexeme ratio in a sample cannot be a constant.  As the number N of tokens (word occurrences) in a sample increases, the number M of lexemes (distinct words) grows like K*sqrt(N).  More precisely like K*N**b where b is typically between 0.4 and 0.6.  This formula is known as You are not allowed to view links. Register or Login to view..

So, when comparing the VMS lexicon size to that of other languages, it is important to use samples with the same number of tokens.  

Assuming the exponent b is 0.5 for both languages, the interesting language parameter (independent of  sample size) is K = M/sqrt(N), not M/N.

Thank you, I didn't know about Heaps' law.