Chronological Path Stability/Uncertainty Analysis

[Jorge_Stolfi]

This is a follow-up analysis to my You are not allowed to view links. Register or Login to view. All of the code for this work can be found in the You are not allowed to view links. Register or Login to view..

Thanks for posting these results!

I still disagree that the ordering of sections based on any textual statistics can be interpreted as chronological order.  But let's leave that discussion for other posts.  Here is what those plots tell me about the clustering of pages and bifolios, and your splitting of the usual sections (defined by image topic and A/B language) into sub-sections:

1. I see no evidence in those plots that Herbal 1a and 1b should be separate sub-sections.   
   
2. Ditto for Herbal 3a and 3b. 
   
3. Ditto for Stars 5a and 6.
   
4. Stars 3c and 4c look like they belong together too.
   
5. There are some single pages that look like they don't belong to their assigned sub-sections: maybe two pages each in 1.5a and 2a, one in 3b, two in 5a.  
   

Stars S-3c and S-4c look very different in those plots. But here is what I think is probably happening (in a suitable projection of R^256 to R^2:
   
In this proposed interpretation, subsections S-3c and S-4c are just two parts of the same cluster that happens to be elongated in a direction perpendicular to the direction d that the curve fitting algorithm usually chooses as the independent variable t (the horizontal axis on those plots).  Presumably that is the direction of "maximum extent" of the whole dataset as given by PCA or some similar method.  

Because of that choice for d, the fitted curves, like C1 and C2, will generally follow similar paths.  Apparently they will go though or near 4c mostly at the same parameter value t0 = 0.85.  But the curves may happen to fall on two main classes that have different mean tangent directions at t0.  Then, depending of the curve's class, the 3c part of that cloud may project onto the curve either before or after t0 (green and red arrows).

The bimodal plot of B-4a may have a similar explanation.  That cloud presumably lies close to the t0 point too, but a bit further off the average curve.  Then, depending on the curve's class, it may project onto the curve either before or after t0.

The two classes of curves may be due to either the choice of d or the output of the curve fitting algorithm being a discontinuous function of whatever parameter is  varied to obtain different cures.  For example, imagine three equal clouds of data points at the corners of a pointy isosceles triangle.  Maybe the algorithm would have a tendency to pick a direction d along either of the two long sides, rather than along the longest median.

Sections 5a and 6 look almost identical on the second plot. The faint smudge to the right of 6 may be due to the same cause for 3c and 4c proposed above.  Namely 5a and 6 are just two parts of a singe cloud that is elongated in such a direction that for some curves the 6 part projects at t = 0.75, instead of t = 0.6; whereas 5a, being closer to the curve, always projects around t = 0.6.

As for item 5, the marked difference between those "oddball" pages and the rest of their assigned sub-sections may be an argument against your thesis.  Presumably they are pages with no images, or with "unclassified" images, that got placed in those sub-sections only because they were in the same bifolios as other classifiable pages -- is that correct?  If so, then either
<A> Those pages have different statistics because they were written long after the other pages on the same bifolios, or
<B> Those pages were written at the same time as the rest of their bifolios, but have different statistics because they are about different topics structure.
I don't see why <A> should be more likely than <B>.

All the best, --stolfi