26-02-2020, 05:59 PM
(This post was last modified: 27-02-2020, 12:49 PM by Torsten.
Edit Reason: f42r added
)
The word tokens in the VMs are not randomly distributed. There "exists an inherent relation between word similarity and context: when we look at three most frequent words on each page, for more than half of the pages two of three will differ in only one detail" (You are not allowed to view links. Register or Login to view., p. 3). "All pages containing at least some lines of text do have in common that pairs of frequently used words with high mutual similarity appear" (Timm & Schinner 2019, p. 3).
This increases the chance that frequently used word tokens on a page occur duplicated or next to similar ones. There are even sequences with four or more similar word tokens in succession:
<f1r.P3.15> kol.chol.chol.kor.chal
<f15v.P.5> otchor.chor.chor.ytchor
<f27v.P.1> fochof.chof.cho.sho
<f42r.P3.20-21> sho.chol.chol.chal-shol.chol.chol.shol
<f75r.P.38> qokeedy.qokeedy.qokedy.qokedy.qokeedy
[The three most frequently used word tokens on folio You are not allowed to view links. Register or Login to view. are <daiin> (7), <chol> (7), and <dain> (6). The most frequently used word tokens on folio f15r are <chor> (5), <chol> (4), and <daiin> (4). On folio You are not allowed to view links. Register or Login to view. <sho> (3), <dchy> (3), and <shy> (2) are the most frequently used word tokens. The most frequently used word tokens on folio You are not allowed to view links. Register or Login to view. are <shol> (11), <chol> (8), and <daiin> (5). The most frequently used word tokens on folio You are not allowed to view links. Register or Login to view. are <qokain> (22), <shedy> (16), <qokeedy> (14) and <qokedy> (14).]
This increases the chance that frequently used word tokens on a page occur duplicated or next to similar ones. There are even sequences with four or more similar word tokens in succession:
<f1r.P3.15> kol.chol.chol.kor.chal
<f15v.P.5> otchor.chor.chor.ytchor
<f27v.P.1> fochof.chof.cho.sho
<f42r.P3.20-21> sho.chol.chol.chal-shol.chol.chol.shol
<f75r.P.38> qokeedy.qokeedy.qokedy.qokedy.qokeedy
[The three most frequently used word tokens on folio You are not allowed to view links. Register or Login to view. are <daiin> (7), <chol> (7), and <dain> (6). The most frequently used word tokens on folio f15r are <chor> (5), <chol> (4), and <daiin> (4). On folio You are not allowed to view links. Register or Login to view. <sho> (3), <dchy> (3), and <shy> (2) are the most frequently used word tokens. The most frequently used word tokens on folio You are not allowed to view links. Register or Login to view. are <shol> (11), <chol> (8), and <daiin> (5). The most frequently used word tokens on folio You are not allowed to view links. Register or Login to view. are <qokain> (22), <shedy> (16), <qokeedy> (14) and <qokedy> (14).]

