[Article] update to Zattera's slot machine

[Mauro]

I'm not much into Python, so I cannot check the code, I'm sorry.

I haven't yet understood which are the final 3 slot grammars which are compared, Model A, B and C. Can you post their stucture? Ie. Slot1 = ['q', ch', sh'] Slot2 = [..], something like that?

All 3 models share the same structure: 
Slot1 = ['q', 'ch', 'sh', 'cth', 'ckh', 'cph', 'cfh', '']
Slot2 = ['e', '']
Slot3 = ['e', '']
Slot4 = ['o', 'a', '']
Slot5 = ['i', 'ii', 'iii', 'iiii', '']
Slot6 = ['l', 'r', 'd', 'n', 'm', 's', 't', 'k', 'p', 'f', '']
Slot7 = ['y', '']

Here are the main differences:

• A: all slot combinations treated as chunks
  
• B (Zattera, one loop): slot sequence as a single-pass automaton (one pass through the 7-slot sequence, then training removes edges to improve F1 as Zattera did, Output words are paths through the slot chain)
  
• C (Zattera, 2 loops): slot sequence as a two-pass automaton 
  

I'm a little confused.

Model A: I'm sorry but I don't understand what you mean by "all slot combinations treated as chunks"

Model B: why do you call it 'Zattera'? It's completely different from Zattera's. My tests:

1 repetition along the slot grammar

Transliteration file used: RF1a-n-x7 Full cleaned
Grammar name: PROVA Voynich Ninja Labyrinthinesecurity, max repeats = 1
Grammar notes: You are not allowed to view links. Register or Login to view.

Slot 1: q ch sh cth ckh cph cfh
Slot 2: e
Slot 3: e
Slot 4: o a
Slot 5: iiii iii ii i
Slot 6: l d k r s t p f n m
Slot 7: y

Word types coverage (excluding word types with rare characters): 0,06015976

Total number of bits required for the Huffmann codes dictionary: 14175 (character set = 16 characters)
Total number of bits required for the compressed text (all tokens): 80300
Total number of bits required (all tokens): 94475

Nbits_tokens/Coverage metric: 1570401,83852722

It has an extremely low coverage (ie. it cannot find 'daiin').


With 2 repetitions (Model C)

Transliteration file used: RF1a-n-x7 Full cleaned
Grammar name: PROVA Voynich Ninja Labyrinthinesecurity, max repeats = 2
Grammar notes: You are not allowed to view links. Register or Login to view.

Slot 1: q ch sh cth ckh cph cfh
Slot 2: e
Slot 3: e
Slot 4: o a
Slot 5: iiii iii ii i
Slot 6: l d k r s t p f n m
Slot 7: y

Word types coverage (excluding word types with rare characters): 0,4767848

Total number of bits required for the Huffmann codes dictionary: 20678 (character set = 16 characters)
Total number of bits required for the compressed text (all tokens): 329808
Total number of bits required (all tokens): 350486

Nbits_tokens/Coverage metric: 735103,093436505

Coverage stays low, ~46%

So I etsted with 5 repetitions:

Transliteration file used: RF1a-n-x7 Full cleaned
Grammar name: PROVA Voynich Ninja Labyrinthinesecurity, max repeats = 5
Grammar notes: You are not allowed to view links. Register or Login to view.

Slot 1: q ch sh cth ckh cph cfh
Slot 2: e
Slot 3: e
Slot 4: o a
Slot 5: iiii iii ii i
Slot 6: l d k r s t p f n m
Slot 7: y

Word types coverage (excluding word types with rare characters): 0,9978782

Total number of bits required for the Huffmann codes dictionary: 21870 (character set = 16 characters)
Total number of bits required for the compressed text (all tokens): 467105
Total number of bits required (all tokens): 488975

Nbits_tokens/Coverage metric: 490014,71623591

Coverage gets quite good: ~100%. Nbits_tokens is 488975, and Nbits_tokens/Coverage is 490014, quite good values. Loop-Lay was slightly better, at Nbits = 486946 and Nbits/Coverage = 487859. The best grammar I ever found (unpublished in detail, just posted in a thread) improves a little more:

Transliteration file used: RF1a-n-x7 Full cleaned
Grammar name: LOOP_chshy_481947_Nbitstokens_vs_Coverage, max repeats = 5
Grammar notes: Best Nbits_token/Coverage found up to 02/02/2025

Slot 1: ch sh y
Slot 2: eee ee e q a
Slot 3: o
Slot 4: iii ii i d
Slot 5: l k r s t p f cth ckh cph cfh n m y

Word types coverage (excluding word types with rare characters): 0,998003

Total number of bits required for the Huffmann codes dictionary: 15314 (character set = 16 characters)
Total number of bits required for the compressed text (all tokens): 465671
Total number of bits required (all tokens): 480985

Nbits_tokens/Coverage metric: 481947,446167254