05-07-2022, 09:04 PM
(05-07-2022, 08:26 PM)Bernd wrote: You are not allowed to view links. Register or Login to view.Large differences in sample text size are certainly a problem but it can be addressed by subsampling.
You can split each section into chunks of ~1000 vords each (maybe rounded to full pages) to match text length to the smallest and 2nd smallest ones (Astro and Zodiac).
Yes, this is a valid approach statistically, but of course it weakens the statistics of the larger sections.
It also removes the meaning of the idea of a 'corpus'.

