29-04-2020, 11:56 PM
Thanks, Marco!
I did not respond earlier because I got a new PC and I had to get all the java stuff installed.
For Herbal A in EVA, my conditional entropy (h2) goes from 2.1 to 2.3. You seem to get different values, where the difference between space and no space is much smaller. I redownloaded the file (Tahahashi) then pre-processed by eliminating punctuation and line-endings. Still my results are similar...
Results for a few randomly selected texts of similar size:
I did not respond earlier because I got a new PC and I had to get all the java stuff installed.
For Herbal A in EVA, my conditional entropy (h2) goes from 2.1 to 2.3. You seem to get different values, where the difference between space and no space is much smaller. I redownloaded the file (Tahahashi) then pre-processed by eliminating punctuation and line-endings. Still my results are similar...
Results for a few randomly selected texts of similar size: