Have we ruled out simple substitution unwisely?
#14
(09-02-2026, 07:46 PM)Koen G wrote: You are not allowed to view links. Register or Login to view.What I learned from this is that EVA does make things look worse than they are, but they are still very bad.

Koen, first, changing the "spelling system" can indeed raise the entropy per character to close the theoretical limit of log_2(n) where n is the number of distinct letters you are allowed to use in the new spelling.  If the new spelling is (say) capital letters plus the digits 0-9, then you could get close to 5 bits/character.  This is one of the main results of information theory.  

There are algorithms for finding such efficient "spelling systems" given a large enough sample text.  That is basically how text compressors work.  

But the new spelling system produced by those algorithms can be ugly.  Then, second, you should try the "elements" that I proposed some time ago: You are not allowed to view links. Register or Login to view.  Those were guessed from looking at the Voynichese words so I claim that they are "natural".  They still don't take into account the possible role of a/o/y as additional modifiers (like the e modifier but always after it) .

Howver, third, you should not waste time with character (or digraph) entropy, precisely because it is largely determined by the spelling system, not by the language.  You should look instead at word entropy, because that is not affected by the spelling system -- as long as it is consistent (the same word is always spelled the same way) and does require splitting or joining words compared to some reference spelling.

But even then, word statistics are a property of the text, not of the language.  It the text is very repetitive (like those Alchemist Herbals) then the word entropy will be low too.

All the best, --stolfi
Reply


Messages In This Thread
RE: Have we ruled out simple substitution unwisely? - by Jorge_Stolfi - 09-02-2026, 09:58 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  A simple substitution experiment ReneZ 19 10,083 18-09-2025, 10:54 AM
Last Post: ReneZ

Forum Jump:


Users browsing this thread: 1 Guest(s)