n-grams while ignoring spaces?

[nablator]

Their experiment sounds interesting: you make a long string of characters with spaces removed, and test at which intervals entropy goes up. But would this be testable at all?

I don't think this would work well, because a frequent bigram like yq should signal a probable word break. Infrequent bigrams are not the only bigrams where word breaks should occur.

Parsing space-less Voynichese into "chunks" must follow some simple rule that can be easily applied by a human (not a complex grammar), such as partial ordering in a series of "slots" as Massimiliano Zattera suggested, but without the ambiguities.

You are not allowed to view links. Register or Login to view.