A mathematical approach to double words, conditional logic and the missing pages

[Vuk88]

I too have difficulty understanding your claim.

But also when you are doing an analysis on repeat words then it would help to enumerate the repeats. I have done this for you, for quires 13 and 20. See attached.

In particular you will see:

• In quire 13 the frequency of word pairs that are repeats is 1.1%. For quire 20 it is 0.6%. Roughly a halving.
  
• In quire 13  ol is the top word ( 299, 4.2% of words ). The number of  ol ol repeats is 12 and this is nearly the same as the expected number if words were distributed randomly ( 299 x 4.2% = 13 ).  qokedy repeats 11 times, 4 would be expected.  qokeedy, observed 9, expected 3.
  
• In quire 20  aiin is the top word ( 322, 2.73% ) but it does not repeat. 9 would be expected.  qokeedy, observed 11, expected 1.  qokeey, observed 9, expected 2.  ar, observed 8, expected 4.
  

We do know that there is a difference in the words and language of the sections of the manuscript and that this will probably affect the frequency of repeats. In particular quire 13 has a smaller corpus of words [ see You are not allowed to view links. Register or Login to view. ]. Moreover, the number of repeats over and above what could be expected is so low that I doubt that any analysis will lead to any definite conclusion.

Hi @dashstofsk,
Thank you for the interest shown in my work and for the detailed PDF. 
The expected vs. observed frequency calculations in your document are mathematically correct, however, there is a fundamental misunderstanding regarding what the Machine Learning model is actually measuring. Your analysis counts self-repeats (how often ol is immediately followed by ol). My Logistic Regression model does not measure self-repeats.
As @asteckley accurately noted (in post #15), the algorithm operates at the page level. It measures whether the presence of a specific word anywhere on a page correlates with the appearance of any doublet on that same page.
When the model identifies aiin as an "Inhibitor", it means that on pages where aiin is heavily used, the author (or authors) actively suppressed the use of doublets entirely, conversely, "Triggers" act as page-level syntactic markers, predicting that a doublet sequence is likely to occur nearby.
Interestingly, your own data highlights this underlying syntax even at the self-repeat level. You noted that qokedy self-repeats 11 times instead of the expected 4. Furthermore, your data shows that aiin (with 322 occurrences) should statistically produce 9 self-repeats, but it produces 0.
If doublets were pure random statistical noise, aiin would have formed its 9 expected repeats. Its total absence further proves the existence of a syntactic suppression rule.
While your analysis shows that self-repeats of frequent words like ol are likely statistical noise, this does not affect my central correlation that the surrounding page vocabulary mathematically predicts the doublet phenomenon.
To understand exactly why certain words trigger this behavior, I am designing a second phase of the study to drop down to a sub-lexical level. But before jumping to hasty conclusions or introducing new variables, I need to further stress-test the underlying logic upon which the current algorithms were built, and that takes time.
Regards,
Alfredo