(14-09-2019, 05:14 PM)Koen G wrote: You are not allowed to view links. Register or Login to view.Anton: what would you say is the minimum amount of words required for the calculations to be worth something?
As I said yesterday in I don't remember which statistics thread, I think at least 2 orders of magnitude more than the vocabulary size. But, once again, you can explore that by taking a large text of, say, 100000 words, and calculate entropy over a fragment thereof, say, of first 10000 words (note that its vocabulary would probably be less than that of the whole text). Then take a larger fragment of, say, 20000 words. And so on. And observe how the values change.