[Article] A Glyph Is Not a Letter, a Token Is Not a Word, a Space Is Not a Space

[rikforto]

There are passages that are obscure, verbose and give that AI-feel. An example is "Currier strata" - "Currier languages" may be misleading, but here "strata" sounds like arbitrary LLM pseudo-scientific jargon.
It’s a pity, because, from what I understand, the contents make sense.

It is a little hard to tell exactly how much is hallucination, jargon, and what I suspect is machine translation---which is getting quite good but struggles with these technical passages---but yes, some of their conclusions do seem AI inserted. Firmly inside my wheelhouse is this:

The letter and word readings are not the only unit hypotheses. Stolfi’s long-standing proposal that each token is one syllable of a tonal, isolating language of the East Asian type, written in an invented phonetic script (You are not allowed to view links. Register or Login to view.; You are not allowed to view links. Register or Login to view.), makes token-level predictions that can be checked against pinyin renderings of a genre-matched classical Chinese herbal (Bencao Beiyao, 1694) and a narrative (Romance of the Three Kingdoms) at matched token counts (Appendix You are not allowed to view links. Register or Login to view., Table You are not allowed to view links. Register or Login to view.). Two comparisons carry the weight. Shuffle-corrected adjacent order is 5.2–6.4% of capped entropy for the pinyin syllable streams, including the herbal, against −0.5% and +0.4% for Currier A and B and 0.79% pooled; and at a matched sample of about 10,700 tokens the pinyin streams use 335–749 syllable types with 10–20% hapax, whereas Currier A uses 3,343 types with 72% hapax. A syllabary is a closed inventory whose syllables carry sequential structure; Voynich tokens are neither closed nor sequentially constrained. Two further checks—the merge-scale profile of the pinyin letter stream and the calibrated attack against a pinyin-letter trigram model—point the same way but are less specific. You are not allowed to view links. Register or Login to view. already noted that Voynich letter statistics resemble pinyin; the order and closure comparisons show that the resemblance stops at the token level. The test covers Mandarin only; languages with larger syllabaries would narrow the inventory gap but not the order gap.

(Emphasis mine)

This is dense enough, flows just well enough, and is ancillary enough that I just raised an eyebrow at it, but at the same time, I don't see how this is supported by the tests they did. (The first person is not an accident; I don't see it, but that is a statement about me deliberately.) For Mandarin in particular we know the syllable inventory and their more varied sample has covered only half the field, meaning it is nowhere near measuring closure directly. [Edit: At the same time, arguing Voynichese syllables are open in a paper that favorably cites Stolfi's word formation models---which implicitly assert closure, or something like it---seems inconsistent to me. Okay, no, I just didn't absorb---and still don't really understand if I'm being scrupulously honest---A.6. Some of this is that I'm in over my head on their analysis and I don't mind admitting it, but also the paragraph is so murky that I am not sure what questions I'm supposed to be asking to figure out what they've tested and what their analysis means.]

It is hard to tell what is going on here, which is a problem in and of itself, but if pressed, yes, my suspicion was AI contamination. But I could not find a problem with their data claims---though I still stress they did not measure a readable Chinese text!---nor a smoking gun for impropriety. The paper needs revised and clarified before peer reviewed publication is all I'll say.