Spaces and Entropy

[Koen G]

But do I read this graph correctly that in EVA h1 slightly decreases when removing spaces?

I had not calculated the h1 difference yet, but it is a bit all over the place, and usually very small. Latin texts are generally also reduced in h1 when spaces are removed. The increase in h1 ranges from -1% to 5%.  The median value is just a 1% increase in h1 upon removing spaces.

I find this hard to grasp intuitively, but it must have something to do with the balance between glyph frequencies, and to what extent the removal of spaces upsets this balance. Maybe someone else can give a better explanation for this.


I don't think we can use this data to determine whether something can be a language or not. After all, we have no control group. Maybe all kinds of different data sets behave this way when spaces are removed.