26-05-2019, 09:55 PM
(26-05-2019, 09:12 PM)Torsten wrote: You are not allowed to view links. Register or Login to view.(26-05-2019, 01:00 PM)Anton wrote: You are not allowed to view links. Register or Login to view.Quote:I'm most interested in this claim:
Quote:when we look at the three most frequent words on each page, for more than half of the pages two of three will differ in only one detail.
That's a very interesting angle. In particular, it is interesting in which detail they do differ, and what is their relation to most frequent words on the whole. If the page-frequent words are the same as the corpus-frequent words, then that would be a bit trivial.
There is much more said in section 2. For instance, we also say that "a token dominating one page might be rare or missing on the next one".
See for instance the pages You are not allowed to view links. Register or Login to view. and f1v. On page You are not allowed to view links. Register or Login to view. the most frequent tokens are [daiin] and [dain] whereas on page You are not allowed to view links. Register or Login to view. only one instance of [daiin] exists:
You are not allowed to view links. Register or Login to view. daiin (7) / dain (6)
You are not allowed to view links. Register or Login to view. chol (5) / shol (4) / ... / daiin (1)
The occurrences of [daiin] for each page of the first quire are: 7 1 3 5 2 1 5 4 2 2 5 5 4. Considering that You are not allowed to view links. Register or Login to view. has more words than any of the other pages, I don't see how this is anomalous. Indeed, You are not allowed to view links. Register or Login to view. has 5 [daiin] over eight lines compared with You are not allowed to view links. Register or Login to view. having 7 [daiin] over 24 lines. Yet you give the signature word pair for You are not allowed to view links. Register or Login to view. as [chol, chor], which also have 5 tokens, simply because that page doesn't have any strong matches to [daiin].
Why did the writer choose to copy some words more than others on any given page? Why does the choice of words and word pairs seem unpredictable aside from their overall frequency? Why did the writer choose to write [daiin] a lot, but [dain] rather less, and [daiir] or [daiiin] a lot less?
The page f89r2 has a massive 19 tokens of [daiin], yet zero [dain], one [daiiin], and zero [daiir]. He wrote [daiin] three times in a row! Linguistic theories are expected to account for this, but how can you? Did the writer not catch what he was writing? Was he suffering from monotony? Did he forget that he could spice things up a bit by dropping or adding an [i]? Was he simply insane? Blind? Experiencing dementia?

