@ Stolfi,
2. These diagrams do not show a first-order Markov chain. Each percentage value refers to the entire sequence of glyphs tracked up to that point (oqyde—with a “folger”), not just the last glyph.
So the arrow from q to y asks: How often does y follow the sequence oq? And, of course, not: How often does y generally follow q?
The search takes place within the lines of the glyph stream, with spaces effectively skipped. In principle, this also captures the sought-after sequence within longer tokens—which, of course, is very rare after oq. But a standalone word is not assumed.
The displayed spaces are counted separately and represented by the gray bars. “Space 99%” means: In approximately 99% of cases, a space follows the sequence in question—regardless of which glyph follows next. The figure therefore does not refer exclusively to the continuation with y but is backward-looking.
The graph simply describes the observed continuations in the text—I can also display them with any other glyphs that follow a given glyphcluster (this is a very flexible program that works with any actual glyph sequence, both forward and backward).
2. These diagrams do not show a first-order Markov chain. Each percentage value refers to the entire sequence of glyphs tracked up to that point (oqyde—with a “folger”), not just the last glyph.
So the arrow from q to y asks: How often does y follow the sequence oq? And, of course, not: How often does y generally follow q?
The search takes place within the lines of the glyph stream, with spaces effectively skipped. In principle, this also captures the sought-after sequence within longer tokens—which, of course, is very rare after oq. But a standalone word is not assumed.
The displayed spaces are counted separately and represented by the gray bars. “Space 99%” means: In approximately 99% of cases, a space follows the sequence in question—regardless of which glyph follows next. The figure therefore does not refer exclusively to the continuation with y but is backward-looking.
The graph simply describes the observed continuations in the text—I can also display them with any other glyphs that follow a given glyphcluster (this is a very flexible program that works with any actual glyph sequence, both forward and backward).