17-11-2025, 12:16 PM
Quote:In short, the non-filler words will not follow Zipf's law.
If most words are fillers then the remaining texts would be really short.
For example a whole plant page could contain a text:
Yarrow is good for heart and wounds. Drink hot with wine
or even
yarrow good heart wounds drink hot wine
Such short texts may not obey statistical laws working for longer texts

