I understand that it may not be convincing to some, but the inverse power relation between rank and frequency is there. Anybody can compute it. Maybe looking at the plots in the paper cited above would have been more persuasive. The departure for the high ranking words, is also well known, the wikipedia page on this shows many plots which do not follow strictly for the highest rank words (I am pretty sure that Stolfi would love the Tibetan, Chinese and Vietnamese curved Zipf plots 
). You are not allowed to view links. Register or Login to view.
Dismissing the relation of log(rank) vs. log(frequency) based on a few points of raw data (i.e. not on the expected log(data)) does not seem right.
There is also something called the Zipf's-Mandelbrot law which aims to fit that high frequency range better.
Just to be clear, I do not know what the vms is, whether a language, code, a cipher or something else. I would consider any suggestions based on testable evidence, something that few people provide.
One more important thing is that any attempts to explain a 'solution' should explain that relation too.
Regards,
G.
). You are not allowed to view links. Register or Login to view.Dismissing the relation of log(rank) vs. log(frequency) based on a few points of raw data (i.e. not on the expected log(data)) does not seem right.
There is also something called the Zipf's-Mandelbrot law which aims to fit that high frequency range better.
Just to be clear, I do not know what the vms is, whether a language, code, a cipher or something else. I would consider any suggestions based on testable evidence, something that few people provide.
One more important thing is that any attempts to explain a 'solution' should explain that relation too.
Regards,
G.