25-06-2025, 12:37 PM
Quote:But then it's strange that the perplexity goes up with longer ngrams, it shouldn't.
I guess at some moment you hit end of the word effect.
So when you have let's say 5 characters and you are predicting the 6th one, you will be often predicting a first character in a new word based on a previous word. In such case all the patterns like "ccc" or "aiin" appearing in the Voynich words have no use.


![[+]](https://www.voynich.ninja/images/collapse_collapsed.png)