(28-01-2025, 10:26 AM)008348dc760f858fd668476b75fb6f wrote: You are not allowed to view links. Register or Login to view.I wrangled a python script that goes through each folio, takes each individual word and counts how many times it appears in any other folio. To balance the fact that different folios will have a different number of words I divide the final score by the number of words in the comparison page. The idea is that we are balancing the count against the event opportunity (although I admit this is likely not a *clean* scoring method).
I'm not sure if the result is not largely just the geometric mean of the number of word tokens on both pages.
The manuscript is dominated by words like "daiin", "chol", etc., and, as far as I understand, the relative frequencies of these words are mostly determined by the "hand" (A/B, etc). So, unless there is some filter that removes the effect from top words, I would expect the chart to mostly show the (weighted) number of daiin's on page 1 times the number of daiin's on page 2.