[Article] A Glyph Is Not a Letter, a Token Is Not a Word, a Space Is Not a Space

[pfeaster]

In Patrick’s words:

Voynichese is encoded continuously in a way consistent with transitional probability matrices, but glyphs can sometimes affect transitions beyond the ones that follow immediately after them.

Could this be an argument in favor of the idea that, in a system like EVA, we are chopping up minimal units? Especially in cases where EVA-e and EVA-i are involved. If "eeed" behaves in a predictable way, then why should we assume that every stroke is a glyph?

That's certainly worth considering, and the line Marco quoted was actually one of three possibilities I'd put forward simultaneously, the other two being:

• Voynichese is encoded continuously in a way consistent with transitional probability matrices, but certain bigrams, such as ok or qo, behave as distinct and inseparable elements rather than as combinations of their parts.
  
• Voynichese has a significant “word structure” after all, or at least a cyclical structure, such that the probability of k or r following after o varies based on which slot in the cycle o itself fills.
  

The reason I was doubtful about the bigram (or n-gram) interpretation was that some of the strongest patterns I'd found involved consistent statistical deflections between specific glyphs and the glyphs that came two positions later.  For instance, the glyph two positions after n is more likely to be k or t than it "should" be, regardless of whether the intervening glyph is y or o.  If ny and no or ok and ot were independent unitary bigrams, I would have expected them to behave differently from one another. As it is, n*k and n*t seem to be favored as patterns regardless of what the intervening * is.

On the other hand, the authors of this new paper do conclude based on their study that we're likely to be breaking up minimal units ("recurrent multi-symbol units"), so it looks like they're thinking in the same terms you are.

Beyond that, I wholeheartedly agree with their more fundamental argument:

 An EVA glyph should be called a glyph until a model demonstrates that it functions as a letter. A ZL token should be called a token or surface form until a model demonstrates that it functions as a word. A transcribed blank should be called a separator until its boundary function is established. Treating the three observed categories as letters, words, and uniform word spaces builds an untested decipherment into the data.

To translate this into Ninjaese: what they call a "ZL token" here corresponds to what some of us colloquially call a "vord" around these parts, and what they call a "transcribed blank" is what we usually call a "space" (which I'd consider neutral, since it still involves space whether or not it's a word separator).