(09-06-2025, 01:02 PM)quimqu wrote: You are not allowed to view links. Register or Login to view.Yes, you got it mostly right! GPT models learn patterns and rules (like a kind of “grammar”) from the Voynich text. They use this knowledge to predict or generate new words. Then, by comparing these generated words to the actual Voynich words, the model calculates a “perplexity” score, which shows how well it understands the text. Lower perplexity means better understanding. This score is also used during training to improve the model step by step.
I'm not sure if it's nitpicking or not, but I don't think the low perplexity here can be called "understanding" in any sense. The perplexity is just the (anti-)confidence score when predicting new tokens. I can train a model on the following sequence: "You are not allowed to view links. Register or Login to view." and the perplexity will be extremely low, because it's very easy to predict next tokens from past tokens. This doesn't mean the model would have any idea about the semantics of this phrase or could correctly predict anything in "Chicago cats Denver dogs bully bully Minnesota mice".