Of bookshelves and statistics

[quimqu]

So my question is: what are @LisaFaginDavis's, @npcompl33t's and @quimqu's studies telling us? Are they finding a "Stolfi's bookshelf" or a "Mauro's bookshelf"? I guess demonstrating the condition of indipendence of the variables is quite important, am I right?

@Mauro good question. I think it can be answered case by case.

The big arc (npcompl33t’s order of the whole manuscript) is Stolfi’s bookshelf. It sorts the pages by hand and section, like sorting books by publisher. My generator has no order at all between its sheets and gives almost the same arc.

Q13 isn’t even a bookshelf. His own runs flip a coin for almost every pair of bifolia. All the books have the same height.

Q20 is the interesting one, so I went looking for the hidden variable. I think I found it:

• Language B has two ways of writing. The herbal B pages use words like daiin, aiin, otaiin. The biological pages (Q13) use qokeey, qokeedy, shedy, and repeat them a lot.
  
• Q20 mixes both, sheet by sheet. Sheet 105-114 is written almost like herbal B, sheets 103-116 and 108-111 almost like the biological section, the others in between. Each sheet keeps its style from the first line to the last.
  
• I trained word models on the rest of the manuscript, without Q20, and scored each Q20 page. “How biological is this page” follows npcompl33t’s order at 0.93 (1 = perfect).
  
• It isn’t page length. With every page cut to the same number of words, nothing changes.
  

That’s why Layfield and Davis’s LSA and npcompl33t’s curve give the same Q20 order: both measure which words a page uses, so both see the same thing. Height measured with two rulers.

So for Q20 my guess is: Stolfi’s bookshelf, but now we know what the height is. The sheets are written in two styles of language B. To make it your bookshelf we would need something that isn’t words and moves the same way: the ink, the hand, the stars in the margin. The text alone can’t tell us if the change is time, subject or exemplar.