I did a new experiment today, not with the model, but trying to focus on generation of similar words. Starting with daiin, the most used word in the MS.
I started from a simple version of the similar-token generation idea. If words like daiin are produced by making small variations of nearby forms, then its closest variants, the words at Levenshtein distance ≤ 1, should also tend to appear in similar surroundings. So I took the immediate neighbours of daiin in form, and I compared the words that come before and after them. I also allowed context to continue across line breaks when the paragraph continues, so the comparison was not artificially cut by the layout.
The first result was negative for a naïve version of the idea. The exact neighbouring words of these variants are usually not very similar to the exact neighbouring words of daiin. In other words, just because two words look almost the same does not mean they sit in the same local context.
Then I relaxed the test. Instead of asking whether the neighbours are exactly the same, I asked whether the neighbours are themselves also close variants, again with Levenshtein ≤ 1. This is a fairer test for a model based on repeated local variation. Under this looser comparison, the overlap rises a lot. So the variants of daiin do not usually share the same exact neighbours, but they often share neighbours from the same broad family of similar forms.
That is the important point. A free and almost random variation model would suggest that close variants should circulate in almost the same environments. What I see is more constrained than that. The system does seem to keep words inside a broader field of similar contexts, but not in a loose or fully interchangeable way. Some variants are clearly closer to daiin than others, and they are closer in different directions.
In particular, dain stands out more on the previous-word side, while aiin stands out more on the next-word side. So even inside the immediate neighbourhood of daiin, the closest forms are not behaving in exactly the same way. They are partly related, but they are not simple drop-in substitutes.
A simpler way to say it:
Changing a word into a very similar one does not give you the same neighbours.
But it often gives you neighbours that also look similar.
So the system is not freely swapping words. It stays inside small groups of related forms.
I started from a simple version of the similar-token generation idea. If words like daiin are produced by making small variations of nearby forms, then its closest variants, the words at Levenshtein distance ≤ 1, should also tend to appear in similar surroundings. So I took the immediate neighbours of daiin in form, and I compared the words that come before and after them. I also allowed context to continue across line breaks when the paragraph continues, so the comparison was not artificially cut by the layout.
The first result was negative for a naïve version of the idea. The exact neighbouring words of these variants are usually not very similar to the exact neighbouring words of daiin. In other words, just because two words look almost the same does not mean they sit in the same local context.
Then I relaxed the test. Instead of asking whether the neighbours are exactly the same, I asked whether the neighbours are themselves also close variants, again with Levenshtein ≤ 1. This is a fairer test for a model based on repeated local variation. Under this looser comparison, the overlap rises a lot. So the variants of daiin do not usually share the same exact neighbours, but they often share neighbours from the same broad family of similar forms.
That is the important point. A free and almost random variation model would suggest that close variants should circulate in almost the same environments. What I see is more constrained than that. The system does seem to keep words inside a broader field of similar contexts, but not in a loose or fully interchangeable way. Some variants are clearly closer to daiin than others, and they are closer in different directions.
In particular, dain stands out more on the previous-word side, while aiin stands out more on the next-word side. So even inside the immediate neighbourhood of daiin, the closest forms are not behaving in exactly the same way. They are partly related, but they are not simple drop-in substitutes.
| Comparison | Main result | Interpretation |
|---|---|---|
| Exact neighbours of daiin vs Lev<=1 variants | Usually low overlap | Very similar word forms do not usually share the same exact local context |
| Fuzzy neighbours, allowing Lev<=1 also for the context words | Overlap rises strongly | The contexts are not identical, but they often belong to the same broader family of similar forms |
| dain vs daiin | Closer on the previous-word side | dain seems more tied to similar left contexts |
| aiin vs daiin | Closer on the next-word side | aiin seems more tied to similar right contexts |
| Overall reading | Partial support, not full support | The text may use local families of related forms, but not as free random substitutions |
A simpler way to say it:
Changing a word into a very similar one does not give you the same neighbours.
But it often gives you neighbours that also look similar.
So the system is not freely swapping words. It stays inside small groups of related forms.