Deconstructing the VMS : A positionless ledger (part 1)

[Mauro]

How is this different from a slot grammar? If treated as a slot grammar, how does this ledger score against the existing slot grammars in Mauro's metric? Most likely this question is about the full version of the ledger with gallows, etc, because the metric measures the efficiency in reconstructing the full text, which requires a nearly 100% coverage.

Ok, couldn't sleep last nite so I dug into your question. I was vaguely aware of Mauro's work but hadn't really examined it.  So here's some preliminary results as I understand his work:

This ledger is basically a conditional slot grammar rather than a conventional slot grammar: the the next letter in a word depends on the glyph immediately before it. I had to force it into a finite slot-style implementation so that Mauro's coverage/efficiency metric could be applied. 

Every slot grammar can become a 'conditional slot grammar', i.e. by adding transition probabilities. Indeed I did exactly that when I wrote my asemic pseudo-Voynich generator. A slot grammar becomes a Markov chain (and with enough states and transitions a Markov chain can arbitrarily approximate any text in any language).

With gallows retained it covered 100% of the 8,424 RF1a-n types tested (the transcription Maro used) and generated about 7.82 billion possible forms, giving an efficiency of 1.08×10⁻⁶. For comparison, Mauro's EXTENDED-12 covers 88.45% while generating 6.12 billion forms, with efficiency 1.16×10⁻⁶. So the two have almost identical efficiency, but the ledger reaches full coverage. 

However, my metric does not use efficiency, because it's unsound, as I demonstrated (as is the "F1 score"). It uses Nbits/Ncoverage, the amount of compression which is achieved by treating the VMS as being generated by the slot grammar. This metric is, instead, sound. To get this metric you first need to parse the VMS text into 'chunks' (ie.: 'qok', 'aiin', 'd') using your ledger (I don't know how), then use the chunks dictionary to compress the text (one symbol for each 'chunk'; I used variable-length encoding for the symbols, but fixed-length should work too and is less anachronistic).

Stripping gallows (which I will explain my reasonings for this in the next post) makes a huge difference. The ledger still gives 100% coverage of the stripped vocabulary, but the generated space falls from 7.82 billion to about 337 million. Efficiency improves about 15-fold, to 1.65×10⁻⁵. Mauro’s EXTENDED-12 is 1.16×10⁻⁶ at 88.45% coverage.

So the stripped ledger has full coverage and roughly 14 times Mauro’s efficiency.

But here you're comparing apples and oranges. The coverage of the full text with your ledger (without gallows) is much lower than the coverage of my EXTENDED-12 (which, by the way, is an obsolete grammar since much improved upon). Or, alternatively, you should compare with other grammars where gallows have been stripped too.


In case you want to know more about my metric, you can read here:
You are not allowed to view links. Register or Login to view.