About the generation of similar words

[nablator]

About the bursts: before this post, the burst detection followed a fairly natural idea: the burst starts with the first word and the following are considered variations of it. This is simple and works well for segmenting the text, but it has one major problem: it assumes that the first word is the origin of the others. And this does not have to be true.

This is an important fact to take into account: lines were not always written sequentially from top to bottom as any normal text would be. There are many instances of gallows intrusions where the text visibly curves upward to avoid a big gallows glyph on the next line. (This is a big indication of something fishy going on by the way.) So the earlier written words on the same page don't have to be on a line above or to the left on the same line.