A family of grammars for Voynichese

[Mauro]

Thank you for your effort! Probably, I'm missing some context or some important background idea, I still don't understand how and why this metric works. To me it seems that any grammar with 100% coverage will reduce to the 1*WS grammar after chunkification, because it should be possible to match any word to the grammar, producing a single chunk.

Yes, by repeated chunkification every grammar (with or without 100% cover) is ultimately reduced to the 1*WS trivial. But this does not mean the method does not work. Example:

We want to evaluate grammar X.

We chunkify grammar X: X' = Chunkify(X)
We calculate Nchunktypes(X'): the lower this number is, the better the original grammar X is.


Now we can go on and evaluate also grammar X'

We chunkify grammar X': X'' = Chunkify(X')
We calculate Nchunktypes(X''): if it's lower than Nchunktypes(X') then well, we have found an even better grammar than X: X'. If it's higher, we can stop here, but if we go on...

... ultimately we find the 1*WS trivial, at the bottom of the scoring list.


If X = the trivial LL*WS grammar the process is very fast:

We chunkify grammar LL*WS: LL*WS' = Chunkify(LL*WS) = 1*WS (!)

And we have already hit the bottom.