20-04-2022, 11:09 PM
I must admit that I don’t like math and calculations, it seems to me quite boring, so I usually don’t delve too much into statistics, into various tests.
Nevertheless, to find the problem, I ran some experiments with the text, and the results were disappointing. I am sure that you have done all the same manipulations before me, but I hope that I will add something new. I need a little more time to note all my observations, but in general, I can conclude that it is impossible to increase entropy of the VMS text to the necessary level just reconstructing glyphs or ligatures, either excluding spaces and changing borders of the tokens.
Nevertheless, to find the problem, I ran some experiments with the text, and the results were disappointing. I am sure that you have done all the same manipulations before me, but I hope that I will add something new. I need a little more time to note all my observations, but in general, I can conclude that it is impossible to increase entropy of the VMS text to the necessary level just reconstructing glyphs or ligatures, either excluding spaces and changing borders of the tokens.