[Article] A Glyph Is Not a Letter, a Token Is Not a Word, a Space Is Not a Space
#19
Sorry to interrupt the conversation about uncertain spaces, which is certainly more interesting.

I read (much of) the paper and I had a hard time with it. This is partly due to my ignorance and partly to objectively complex measures. But at the end I read this and I think it also plays a role:

Rozanova and Temerev wrote:Acknowledgements

Large language models (Claude Fable 5, Anthropic; GPT 5.6 Sol, OpenAI) were used to assist with data analysis code and with editing the manuscript. All analyses, results, and interpretations were designed, checked, and approved by the authors, who take full responsibility for the content.

There are passages that are obscure, verbose and give that AI-feel. An example is "Currier strata" - "Currier languages" may be misleading, but here "strata" sounds like arbitrary LLM pseudo-scientific jargon.
It’s a pity, because, from what I understand, the contents make sense.

Something that seems unclear to me: they say that Linneus’ Species Plantarum has a higher value of “adjacent tokens within one edit of each other” than the VMS (measured as a ratio against counts in shuffled versions of the text). I cannot find that text in their github repository:
You are not allowed to view links. Register or Login to view.

But since they reference Project Gutenberg, I guess they could have used this:
You are not allowed to view links. Register or Login to view.

Entries look like this:

Quote:CROCUS.

[_sativus._]

1. Crocus spatha univalvi radicali, corollæ tubo longissimo.

  Crocus floribus fructui impositis: tubo longissimo. _Roy. lugdb. 41._
  _Hort. ups. 15._ _Mat. med. 27._

  Crocus flore fructui imposito. _Hort. cliff. 18._

[_officinalis._]

α. Crocus autumnalis sativus. _Moris. hist. 2. p. 335. s. 4. t. 2.
  f. 1._

  Crocus sativus. _Bauh. pin. 65._

[_vernus._]

β. Crocus vernus latifolius. I-XI. & I-VI. _Bauh. pin. 65. 66._

  Habitat in Alpibus _Helveticis_, _Pyrenæis_, _Lusitanicis_,
  _Tracicis_. ♃

I had a look at a few entries, but they do not seem to feature consecutive similar words. I suspect the authors counted short “words” like “s. 4. t. 2.” as four consecutive similar words?
[+] 5 users say Thank You to MarcoP for this post
Reply


Messages In This Thread
RE: A Glyph Is Not a Letter, a Token Is Not a Word, a Space Is Not a Space - by MarcoP - 22-08-2026, 05:26 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  [Article] Glyph combinations across word breaks in the Voynich Manuscript Emma May Smith 36 43,928 28-10-2024, 07:10 PM
Last Post: Barbrey

Forum Jump:


Users browsing this thread: 1 Guest(s)