GPT Models Fail to Find Language Structure in the Voynich Manuscript

[Jorge_Stolfi]

In fact, I trained several models with different block sizes (from 1 to 32). Block size tells how many tokens are used for learning and predicting. As expected, with block size 1 the results are better — the model just echoes bigrams. But with larger blocks (e.g., 8 or more), it needs to rely on broader context, and that’s where Voynichese starts to diverge from natural languages.

I would be surprised if the models with block size 3 and larger end up using more than the last two words.

Here’s a sample of matching bigrams and trigrams for block size = 8, for several texts including Voynichese:

You are not allowed to view links. Register or Login to view.

Google Drive says I don't have permission to read that link.

From the Voynichese output that you quoted, it seems that many word spaces are missing.  I suppose they were marked with comma ("possible space") instead of period in the original file. (Or, sadly,not marked at all: it looks like the Scribe sometimes ran words together by accident.)  Would you consider trying again with periods and commas treated as word separators?

All the best, --jorge