Have we ruled out simple substitution unwisely?

[eggyk]

Indeed, the point of the thread is to ask whether there is a potential flaw shared by all of the popular transcription alphabets that causes unusual entropy results, instead of unusual entropy being an inherent trait of the VMS text.

I would recommend anyone interested in this to fist read my blog post here: You are not allowed to view links. Register or Login to view. .

I also wondered what the impact of EVA was on entropy, the impact of the transliteration file used etc. So I drove it to the extreme by gradually replacing common EVA pairs with new letters. Some findings:

• The impact of the initial transliteration file (TT vs ZL) is low. Don't stress about this.
  
• Benched gallows are a big problem for anyone undertaking something like this. You need to make a choice about how to treat them, and that has an impact. No single choice is the obvious correct one.
  
• Q13 is a huge outlier (in 2020, the extent of the difference was news to many people)
  
• EVA lowers entropy in obvious ways, like by splitting the bench and chopping up "in"-clusters. However, fixing these does not make Voynichese normal by any definition of the word. 
  
• In Herbal A, I obtained the best results by "desplitting" the following EVA clusters, in that order: qok, chol, chor, che, chy, ol, cho, or, qot, ar, eey, al, qo. This frankly ridiculous exercise leaves you with barely viable h2 stats and extremely short words.
  
• The same can be done for other sections, with some different choices. 
  

What I learned from this is that EVA does make things look worse than they are, but they are still very bad. The fact that a cluster like [edy] wreaks havoc on your conditional character entropy has nothing to do with the transliteration system, and has everything to do with the Voynichese system.

Now, this was over 5 years ago, maybe someone has a better approach. To me it also looks like the proposals in the opening post wouldn't necessarily lead to an increase of entropy (it's not always intuitive). Such ideas remain speculation until you actually test them.

I appreciate the response and the work done in that post. I remember reading it when I first joined the forum but it's worth me going through it again. 

The fact that EVA does lower the entropy and making changes has a measurable effect on the entropy is something I may persue. As I mentioned in the OP, there may be aspects of the EVA (or other transcription alphabets) that significantly decrease entropy that haven't been addressed. Or there could be tweaks to EVA that can be made that may significantly increase the entropy. I think being as exhaustive as humanly possible is worthwhile here. 

Out of curiosity, is there a widely accepted tool to calculate and plot the entropy values of text? I remember seeing a website link somewhere here.