Mapping Voynich connections through rare tokens

[quimqu]

If I strip the gallows from those hapax tokens, that decline flattens out.

That is interesting, although I would probably be careful interpreting the gallows result too deeply for now.

If removing gallows causes many forms to collapse into already existing tokens, then part of the flattening may simply come from reducing the vocabulary space artificially. The Voynich token space already has extremely small Levenshtein distances between many words, so removing one high-information element like gallows can easily create merges with forms that already exist elsewhere in the manuscript.

My own objective was a bit different anyway. I was trying to see whether low-frequency tokens create bridges between folios and sections. That is why I did not use hapax legomena initially: a single occurrence cannot really create network structure or repeated connections between pages.