GPT Models Fail to Find Language Structure in the Voynich Manuscript

[quimqu]

Did the Latin text use scribal abbreviations (and if so, how were these tokenized?) or was it first expanded into plaintext? Many folks on this site have pointed out the similarities between Voynichese letters and Latin scribal abbreviations, and I'd be very curious about whether or not a Latin text with the sigla tokenized as distinct characters behaves like Voynichese. This would effectively be a verbose substitution cipher, but one leaning strongly on historical convention.

Great work on this- I'm planning on tinkering with something similar in order to try to do embedding space alignment across Currier hands. I'd love to look at your repository, if it's public!!!

Hello, the public plain texts I worked with were first expanded into full plaintext, so the original abbreviations are not present in the text. This should make Latin easier for GPT-style transformers to understand, but even with that expansion, the results in Latin are still not very good. I agree with Koen that Latin was a 'dead' language and must have been heavily influenced in many ways by natural languages in use at the time — including errors and other irregularities.