[Article] A Glyph Is Not a Letter, a Token Is Not a Word, a Space Is Not a Space

[Koen G]

So if I understand one if their conclusions correctly, the final letter of a word (I'll keep saying it) is abnormally good at predicting the first letter of the next word. We would expect this property for whole words, like "you" is often followed by "are", which the VM does not do. So the VM displays an abnormally high predictability between edge glyphs, and near-zero token-to-token predictability. (I know people have researched this in the past, but I haven't learned much about it yet.)

So I wonder: can we figure out if edge glyph phenomena are caused by selection, or by alteration? Will a certain end glyph attract an existing word with a specific initial glyph (selection) or will it change the initial glyph of the "planned" next word (alteration)? Can this even be calculated?