Chronological Path Stability/Uncertainty Analysis

[npcompl33t]

Your analogy is somewhat misleading, in a way that may be affecting many of your interpretations.  You say: 

• Kansas City and Springfield are slightly ambiguous, going from OKC -> Springfield -> Kansas City is roughly the same distance as OKC->Kansas City -> Springfield. Its clear they both occupy the same 2 slots, but the relative ordering is uncertain
  

My city example was answering a much narrower question you had posted, I am not claiming it is a perfect analogy to what was happening. My central point was uncertainty about pairwise orderings does not logically imply uncertainty about every ordering, I was responding directly to some of your narrower questions to illustrate what was happening.


You are right that similar projected positions do not establish 2 pages are close in the feature space. In fact, this analysis demonstrates narrow position ranges can coexist with substantial perpendicular variation. 

You are conflating 3 diffrent quantities:
- transition similarity between two pages
- their distance from the fitted curve
- their separation along that curve

The instability charts show the third quantity. Your analogy also has a critical omission. The fitted routes respond to the observations, they are not simply independently selected highways onto which unrelated cities are projected. 

Let v be the dominating direction of the current fitted curves.   In those plots, the projection of v run mostly left to right.  What would happen to the fitted curves if you stretched the data along a direction u in R^256 that is perpendicular to v?  Wouldn't the new fitted curves run mostly in the direction u? Wouldn't that completely scramble the t order of the sections?

The projections run mostly left to right because I have the same dimension on the X axis in each plot, MDS1. MDS1 captures the most obvious temporal gradient, [/quote]

You're right. I misunderstood you. Unequally stretching can change distances and not the fitted route. However, it is important to distinguish arbitrary distortion from plausible alternative measurement choices. Simply showing a sufficiently extreme stretch changes the answer does not invalidate the analysis. Showing a modest or defensible change alters the broad ordering would. That is why i undertook this analysis, i tested variants of these assumptions, and found that in general the overall path is stable, with certain relative orderings subsequently identified as uncertain. 

You are also correct, I should have said MDS1 is the leading coordinate of the distance derived embedding.



 

Please look again at You are not allowed to view links. Register or Login to view..  Those white vertical streaks, across each color swab, are real, quite obvious, and obviously a bug.  From what I understand, as the curves vary, the t-value of each page should vary smoothly over its range -- and the t-values of different pages should not change in absolute lockstep.  


The streaks may not be a plotting bug, as I guessed before (they are not aligned from section to section), but they must be the result of a spurious quantization of the t-values occurring somewhere.

Genuinely empty columns may be present, but i suspect that many of the stripes you are looking at actually do have some values in them. There are several relevant properties of the implementation, none of which are bugs. 

• It projects continously ont line segments, it does not round to nearest nodes
  
• Many pages project exactly onto the same polyline corder or endpoint. Distinct observations can share a position and move together
  
• The chart bins positions into 100 intervals
  
• The ensemble consists of separately optimized fits, including distinct starting routs and penalties. It is not a continuous sweep guaranteed to fill every position. 
  
• If you give me a specific 'streak' to look into, i can and i will report exactly what is going on. Is there one you would prefer?
  

Here is another Free Advice (you can heed it without registering, but only for this month, hurry!) You should consider only text that consists of full parags.  Excluding labels, titles, radial lines, circular text, etc. And mini-parags like those on f84v, f67r2, f68v3.  

I did in fact do this. I looked at circular text because you explicitly asked about it. i also looked into f86v4, somehow this seems to have been turned into f84 v in your response, so maybe wires got crossed somewhere on this.

They could...  Or they could have broken the current ordering... 

They absolutely would not, and if you actually read my analysis, i tested exactly the impact they would have, including wild assumptions placing them far away from their current centroid. It breaks nothing, its position changes the preferred ordering of 3b/3a. I did quite an in-depth analysis on this in the OP on this thread, so its frustrating that you seem to have not actually read that.

But, even there, the computed ordering may not be the reading order.  In a novel where the story mostly switches back and forth between two locations with mostly different characters, ordering the pages by lexeme sharing would put all the pages of location A together before those of location B, or the other way around.  

you are correct, deliberately shuffling these would break the gradient. I am not claiming to have recovered reading order. I am claiming to have recovered production order. In your example, ordering the pages would reverse the authors decision to shuffle two distinct chronologies together, and would recover their true state prior to being shuffled. This is similar to what is happening with Herbal A and Herbal B. 

I see 4-5 well-separated clusters with are not linearly arranged in any of the 2D projections I have seen, including those I made 20+ years ago.  

Those clusters can always be ordered by projecting along some axis, straight or curved; but there is no justification for interpreting that order as chronological.

Clusters and a gradient are not mutually exclusive. No-where do i claim a single 2d plot places them into a linear arrangement. In-fact, i make the point several times in the original horseshoe thread that this is not the case and is expected.