(03-01-2025, 06:13 PM)oshfdk wrote: You are not allowed to view links. Register or Login to view.This is great, I misunderstood the formula then. The numbers shown for the naive grammars do roughly correspond to what I get by just gzipping the transliteration (minus the metadata, line numbers, etc), I get something like 6.5e5 bits.
Yes, with pkzip it's about 68K. My algorithm compresses the text to ~59K with LOOP-Lay-oa, but this excludes the spaces, it only compresses the word tokens.
(03-01-2025, 06:13 PM)oshfdk wrote: You are not allowed to view links. Register or Login to view.I think your metric looks very good for practical assessment of these grammars.
Thank you!
(03-01-2025, 06:13 PM)oshfdk wrote: You are not allowed to view links. Register or Login to view.Would be interesting to see which process, beyond the grammar compression, would results in the smallest lossless output.
What I want to do next is to try to find a better grammar than LOOP-Lay-oa. The number of possibilities is enormous but maybe I can use a Monte Carlo algorithm to sample the grammars space. Will see what happens.