Chronological Path Stability/Uncertainty Analysis

[Jorge_Stolfi]

Genuinely empty columns may be present, but i suspect that many of the stripes you are looking at actually do have some values in them. There are several relevant properties of the implementation, none of which are bugs. 

• The chart bins positions into 100 interval
  

That may be it?  In the plot image you posted, the t scale 0-1 is ~520 pixels.  So it may be a "beating" between the bin spacing and the pixel spacing.
   
Looking at the magnified image, in fact I see that the white streaks are not perfectly white, just very faint.

Anyway, I was confused by those streaks into thinking that, for each curve, all the pages of a section projected on the same dark vertical streak, which seems to be ~0.01 wide in the t axis.  

I understand now that this is not the case, right?  for each curve, the page t's are scattered over a range of ~0.10 or more, with their positions shifting depending on the curve because of the starting point variation.  Moreover, their t-order can change depending on the curve, especially for a F1 curve versus an F2 curve. Is this correct?

I did in fact do this. I looked at circular text because you explicitly asked about it. i also looked into f86v4, somehow this seems to have been turned into f84 v in your response, so maybe wires got crossed somewhere on this.

Oops!  I don't know when and why the switch happened, but I was puzzled too.  

So 

• f84v is an ordinary inlier in the Bio section?
  
• f86v4 is a Cosmo-like page that you placed at the end of you H-3b section because of statistics, but which is still an outlier there?
  

I did quite an in-depth analysis on this in the OP on this thread, so its frustrating that you seem to have not actually read that.

Sorry, I think I wrote that there was a lot of stuff to take in... an many of the details of method and nomenclature I am still trying to understand.

In your [novel] example, ordering the pages would reverse the authors decision to shuffle two distinct chronologies together, and would recover their true state prior to being shuffled. I am not claiming to have recovered reading order. I am claiming to have recovered production order.

In my example both the order of writing and the story time switch back and forth between the two settings.  Like between a band of robbers on the run, and the investigators who are trying to find and catch them. So ordering the pages by similarity would have been just wrong.

This is similar to what is happening with Herbal A and Herbal B.

In that case it is quite possible (but not certain!) that Herbal-A and Herbal-B were produced in distinct epochs separated by some interval.

But it is still not certain that the statistical differences are directly due to passage of time.  Again, if the differences turn out to be due to a "spelling reform", that would almost surely be the case.  But they may also be due to other causes, like two different classes of plants which somehow imply very different vocabularies.  

Or they are recordings of plant lore from two distinct herbalists, who spoke different dialects.  That is a common situation when anthropologists record local legends, linguists record languages or dialects, journalists record war veteran stories, etc.  Then Herbal-A and Herbal-B could have been created in parallel -- some plants from Mary today, some from Nancy tomorrow, some more from Mary the next day...

Clusters and a gradient are not mutually exclusive. No-where do i claim a single 2d plot places them into a linear arrangement. In-fact, i make the point several times in the original horseshoe thread that this is not the case and is expected.

But if they do not lie in an approximately straight line, what is the meaning of "gradient"? 

If you take any 5 clusters in R^256 space, you can always find a projection onto some R^6 subspace such that their centroids lie on equally spaced points along an arbitrary parabolic curve.  Then, on any projection from that R^6 to R^2, they will seem to be equally spaced along a parabolic curve. 

Granted, you are choosing the R^6 subspace and the projection by some fixed criterion (the MDS one). Not by computing a projection that would have that specific result.  Then finding that the projected centroids lie close to a parabolic curve would indeed be a somewhat non-trivial fact.  But still not as impressive as if they fell close to a straight line.

But also, if you have 5 clusters in R^256, you can always find a polynomial curve of degree 4 that goes through all their centroids. In fact, for any specified order of the 5 clusters, there is a direction d in R^256 (actually MANY directions) such that the centroids project on that axis in the specified order; and then one can always find a curve of degree 4 in R^256 that goes through the centroids in that order.  An then the same will happen on any 2D projection of the points and the curve;

So, if the centroids of your sub-sections do not lie on or close to a straight line, but you sill claim that there is a "gradient" along some snaking and twisting curve, you need to explain whether and when a bunch of centroids would not show a gradient, and how significant your "gradient" actually is.

All the best, --stolfi