A family of grammars for Voynichese

[Mauro]

You are perfectly right: the pattern-matching algorithm is 'greedy', it will stop at the first match it finds. Actually the algorithm is more sophisticated and I could go on and find all the possible alternative matches too, but I think this would not be much useful. It's one of those things which would be very easy to explain face-to-face just by scribbling on a piece of paper, but it's not easy to explain in written form. I try my best.

It does not matter because, after all, the grammar structure univocally determines the chunk structure (even if there many be many possible alternative 'chunkings'). We could easily re-arrange the grammar to get the alternative chunkings, for instance in LOOP-4 all the 'consonants' [d, k, l, r...] etc. are at the end, so a word such as 'daiin' is chunked as [d] + [aiin]. But I could just put that slot at the beginning, and 'daiin' would become [daii] + [n].

So, what discriminates between one of the two alternatives forms of LOOP-4? It's the total number of chunks, Nchunktypes, found by chunkifying each grammar: I actually made a test with the 'consonants' at the beginning (just past the [q, ch, sh, y] slot) and it finds 609 Nchunktypes, so it's slightly worse than the standard LOOP-4 at 606 (but a very viable grammar nonetheless, I guess).  I also tested with 'consonants' both at the beginning and the end and it resulted in ~675 chunks (I don't remember exactly).

About 'separable' words: I don't think the concept is even meaningful in the context of slot grammars: just duplicate any grammar and it will find all the separable words it had been missing before. Seen from another point of view: it's not that defining a word as 'separable' excuses the original grammar for not having found it

After reading this reply I realized that the (loop) grammar matching mechanics are more complicated than I thought, which probably explains my persistent confusion. I cannot really understand your results so far, but I'm looking forward to new updates, especially those a bit less technical in nature 

Heheh yeah, I tried to simplify the mechanics as much as possible for my presentation, but indeed the repeating loop is one of the problems. Another problem is that the character strings inside the slots have variable lenghts, and another one is the strings can overlap, ie. "lk" overlaps both "l" and "k", which give rise to many possible different chunkification paths (ie.: is 'lkaiin' lk+aiin or l+k+aiin?).

But don't worry, it all works as intended, and I think the best proof of it (beyond the Excel tables I posted) is that it can then generate a pseudo-Voynich text which resembles a lot the original (indeed the main purpose of writing pseudo-Voynich was to verify that everything worked properly, and it's also cool xD).

So I'm sorry if you are yet confused, I very much understand my reasonings are easy to follow for me, but not that much for people who never heard about them (and expecially if they're not used to write code, and in my convoluted English). So I thank you (and all others) a lot for taking the time to go through my writings and try to understand. I actually made a lot of progress by reasoning on what you (and others) have said, ie. the key idea of the 1*WS grammar came to my mind after reading an answer here.

I really don't know what else I coud report, unfortunately. My grammar project looks to be 'finished' at the moment:

1. I think I found a way to soundly score grammars which was previously lacking (efficiency, the previous standard, being unsound, as I demonstrated with 1*WS). This is actually my main result.
   
2. And I think the LOOP grammar, or a variation of it, can possibly be the best one and actually be the 'true' structure underlying Voynich words, given it scores so well against all others on the Nchunktypes metric. At the very least, it's the best grammar available at this time.
   

But any questions you may have, just ask!

Now of course I still have to get my two results to gain acceptance, and this another hard part , and maybe someone will come out and point to a fatal flaw in my reasonings, bursting everything. But I'm rather happy anyway! If something more about the Voynich can be learned by building on these results remains to be seen, but I think that the problem of elucidating the structure of Voynich words is essentialy solved at this point, or on the brink of it.