Heap's Law

[Torsten]

Now, if Torsten's algorithm produces something close to f(x)=0.5*x, it could be that it copies half of the words from the already generated text (and these words obviously do not add to Heap's total of unique words) and randomly generates the other words (similarly to the blue line representing the rnd26 file). While copying already generated words is an excellent way to approximate Heap's law, one must also find a way to appropriately constrain the generation of new words.

The generated text only contains words copied from each other.

The idea behind the auto-copy algorithm is that the scribe was generating the text by copying words already written. But to copy a word doesn't mean that a source word 'chol' is copied as 'chol'. Instead the words are modified. For instance instead of 'chol' the scribe could write for instance words like 'shol', 'chor' or 'cheol'. While doing so the scribe would normally copy words he was able to see. This is in my eyes the explanation for the observation that similar words co-occur on the same pages. For instance on a page containing many instances of 'chol' you will also find many instances of similar words. See for instance all the instances of 'chol' and 'chor' on page You are not allowed to view links. Register or Login to view. (see You are not allowed to view links. Register or Login to view.). 

For each page the scribe could prefer other words. For instance on page f3r many words with a 'm'-glyph can be found (see You are not allowed to view links. Register or Login to view.). If you search for new words on page You are not allowed to view links. Register or Login to view. you will find for instance the words 'sheoldam', 'tsheoarom' and 'pcheoldom'. This three words are similar to each other and they occur only once within the VMS. Moreover 'tsheoarom' and 'pcheoldom' are used as paragraph initial words. There are only seven paragraph initial words using a final 'm'-glyph in the whole VMS. This is the way new words occur within the VMS.