Bit-level Geometry: A deterministic discovery of Voynichese building blocks

[TheEnglishKiwi]

I have a couple questions.

How do you consider spaces? As separators between words, or as any other character?

How many bits are required to store the VMS when you compress it using your Minimum Description Length algorithm (dictionary + text)?

Hi there, great questions!

1. The engine is blind to words. It encodes everything into raw UTF-8 bytes which then gets fed into the engine as a continuous stream of 1's and 0's. It doesn't "see" a space, it just sees the bit pattern for the space character (00100000) like it would any other character. It then lets the math decide if that space forms a structural boundary or not. The tool then uses spaces, tabs and newlines to figure out where those blocks sit. That way it can categorise if a block tends appears at the start, middle or end of traditional word grouping.

2. The raw, uncompressed EVA transcription is 1,862,208 bits or 232,776 bytes. It depends on which Lens you run through the engine (standard MDL formula) so here's each of them:

• 8-bit: 903,682 bits (~51.5% reduction)
  
• 16-bit: 723,282 bits (~61.2% reduction)
  
• 24-bit: 688,640 bits (~63.2% reduction) (winner)
  
• 32-bit: 782,379 bits (~57.9% reduction)
  

This is an intriguing result for me. 24-bit (3 characters) is the mathematical sweet spot which matches what Voynich researchers have long suspected. It suggests the text is built out of tight, highly constrained prefixes, roots and suffixes (e.g. "qok", "che", "dai").

Cheers!