Chronological Path Stability/Uncertainty Analysis

[Jorge_Stolfi]

I think you might be misunderstanding the plots. The plots are showing where certain sections may appear across many runs. They are not saying that these sections are shuffled in together in some runs. On the contrary, the sections are grouped together and the entire section shifts.

Indeed I am confused.  If each Golf sub-section is treated (or behaves) as a single object, why are the lines in H-1.5a staggered rather than aligned, as in H-1a and H-1b?

And what about those "anomalous" lines, like the one at the bottom of H-3b?

Your data is in R^256, but I think that this model in R^2 could reproduce your plot, including your "move together" information:
   
Your Golf sub-sections are the magenta smudges.  In this sketch, I tried to account for the following points:

1. The fact that all pages of each sub-section plot at about the same t value on each curve says that the diameter of each cloud (say, the RMS of the JS distance between pages in that cloud) is small compared to the overall span of the curve.
   
2. I tried to place each cloud so that its projection onto the curve (by Euclidean distance on that plot) is at the mean t value shown in your plot.  
   
3. The distance of each cloud to the curve is very roughly proporional to the spread of its t values on the plot, assuming that most of this spread is due to change in the slope of the curve rather than the curve itself moving in the general t direction.
   
4. The cloud pairs 1a+1b, 3c+4c, 3a+3b, 5a+6 were drawn as single clouds because they cannot be split based on the plot.  Maybe, as you say, they can be split in other dimensions; but those dimensions apparently do not affect their projection on any of the curves.
   
5. If the curves were straight line segments, the t value of the projection of a page would vary smoothly (indeed, linearly)  with the parameters of the curve.  But if the curves are not straight, the nearest t value may suffer a large jump for small changes in those parameters.  That is my theory for why the plot of 4a clusters at two distinct t values, ~0.67 and ~0.87.  In my sketch, as the curve changes from C1 to C2, at some moment the nearest points for cloud 4a jump from the right arm of C1 (at t = ~0.87) to the left arm of C2 (at t = ~0.67) without going through intermediate values.
   
6. A similar situation may explain why clouds 3c and 4c seem to have complementary ranges of t values.  Specifically, I conjecture that the cloud 3c+4c is elongated in the direction perpendicular to the curve.  Your cloud 3c is the half of that joint cloud that is closer to the curve, and it projects to a continuous range of t values as the curves move.  Your could 4c is the other half of that joint cloud; being farther away from the curves, its t values jump discontinuously between ~0.70 and ~0.90 as the curves move.
   
7. The anomalous page at the end of your section 3b woud be an outlier (not shown), located near 3b in my sketch but further away from the curve.
   

What do you think of this model?

All the best, --stolfi

[Jorge_Stolfi]

(in my opinion, the talk about much later painters, and thus irrelevant colors, is pure speculation).

But the claim that colors are later than the binding has at least two crumbs of evidence: 

• the green pigment over the page number on f42r, attested by Rene and others from microscope examinatons;
  
• the blue paint spots at the upper left corner of f40r, that look like they are continuous with the blue "hearts" of You are not allowed to view links. Register or Login to view. but are in fact not aligned and do not match their color.
  

Granted, both of these bits of evidence can be disputed.  But the opposite theory -- that the color paints are original -- has no evidence at all, not even historical precedent.

Note that this "color paints are original" is actually two claims:

C1. The color paints were applied while the VMS was still in possession of the Author, and under his supervision.

C2. The Scribe applied colored paints to each page (or bifolio) as soon as he finished drawing the outlines, before proceeding to the nexp page (bifolio).

C1 would be sufficient for one to consider colors when trying to identify the plants.   But even if C1 is assumed, C2 seems rather unlikely.  But one needs C2 in order to use colors as the clue for the order of creation of the bifolios.

An intermediate hypothesis would be 

C1.5.The Scribe created the bifolios in batches.  For each batch, the Scribe drew the outlines of figures and text on all pages, and then he applied color paints to all those pages.

With C1.5 one could use colors at least to guess the order of creation among sections.

All the best, --stolfi

[npcompl33t]

Indeed I am confused.  If each Golf sub-section is treated (or behaves) as a single object, why are the lines in H-1.5a staggered rather than aligned, as in H-1a and H-1b?

And what about those "anomalous" lines, like the one at the bottom of H-3b?

excellent questions. 1.5a is staggered because its pages more consistently fall into sequence across all of the model runs. In a sequence, the most easily identifiable ordered segments are those in the center, because they contain many overlapping sections before and after. That is why the 1.5a and 2a are so clearly offset. 1.5a in particular appears to be a period of high change, connecting 1a/1b to 2a, which is why i have named it 1.5a / Transitional Herbal A.


The anomalous lines are indeed anomalous. specifically one that jumps forward is f86v4. Looking at the MDS plot here shows what is happening -- this page is on the far end of all the other 3b sections, much closer to the 2a. Below i've highlighted it on the MDS chart.

   


It is extremely anomalous, appearing very far offset from its bifolia, the largest offset by far, across any bifolia, as shown below.

   


I absolutely think this needs more research to untangle f86v4, it could be key to discovering the ordering of these sections.

Your data is in R^256, but I think that this model in R^2 could reproduce your plot, including your "move together" information:

Your Golf sub-sections are the magenta smudges.  In this sketch, I tried to account for the following points:

1. The fact that all pages of each sub-section plot at about the same t value on each curve says that the diameter of each cloud (say, the RMS of the JS distance between pages in that cloud) is small compared to the overall span of the curve.
   

Not quite. It means that those two sections are estimated to fall within the same t slot, and are probably adjacent in the time. It doesn't mean that the differences are small, just that the ordering is ambiguous depending on how you want to draw the line. They can still have substantial distance between them, they are still placed sequentially, just in some cases the line goes left to 3a first and in others the line goes right to 3b first. 

Another example, Houston TX, Austin TX, and Dallas TX all form a triangle. if you start in a random city, and drive to another random city, then drive to the last, and only record your distances, it will be hard to determine the exact order without knowing which city you started in. Houston -> austin ->dallas is almost the same set of distances as houston->dallas->austin. IF you drive to 4 cities, and the 4th is Oklahoma city, i can infer dallas is probably the 3rd city, because it lies halfway in between austin and houston and oklahoma city. However it is still ambiguous if you start the path in austin or in houston.  

I show exactly how this is occuring with the Stars/Bio ordering iYou are not allowed to view links. Register or Login to view.

I tried to place each cloud so that its projection onto the curve (by Euclidean distance on that plot) is at the mean t value shown in your plot.  

I just finished up work adding this as a published view to my website for the clusters, i will share it below.

1. The distance of each cloud to the curve is very roughly proporional to the spread of its t values on the plot, assuming that most of this spread is due to change in the slope of the curve rather than the curve itself moving in the general t direction.
   

No, the spread of the cloud has nothing to do with the distance from each to the curve.  In my city example above, austin and houston would both have overlapping clouds. Some simulations have you starting in austin, some in houston.

1. The cloud pairs 1a+1b, 3c+4c, 3a+3b, 5a+6 were drawn as single clouds because they cannot be split based on the plot.  Maybe, as you say, they can be split in other dimensions; but those dimensions apparently do not affect their projection on any of the curves.
   

No, it has to do with the ambiguity as in my city example, austin and houston would be drawn as a single cloud pair. You can still clearly split one apart from the other in geographic space.

If the curves were straight line segments, the t value of the projection of a page would vary smoothly (indeed, linearly)  with the parameters of the curve.  But if the curves are not straight, the nearest t value may suffer a large jump for small changes in those parameters.  That is my theory for why the plot of 4a clusters at two distinct t values, ~0.67 and ~0.87.  In my sketch, as the curve changes from C1 to C2, at some moment the nearest points for cloud 4a jump from the right arm of C1 (at t = ~0.87) to the left arm of C2 (at t = ~0.67) without going through intermediate values.

This is only ture in the non ambiguous section. The distances between dallas and oklahoma city would be like 1.5a and 2a on the chart -- there is really only one ordering that makes sense with them, so they fall into distinct bands with linear increasing t bands. Ambiguous sections do not. The point of this analysis was to identify which sections are ambiguous, rather than simply saying the path is houston->austin->dallas->OKC then calling it a day. This analysis shows that really houston->austin or austin->houston are both valid paths.

[*]The anomalous page at the end of your section 3b woud be an outlier (not shown), located near 3b in my sketch but further away from the curve.

This is accurate, You are not allowed to view links. Register or Login to view. is quite far from the curve, the farthest such outlier by far. Its assignment to Hb is listed as tentative, the MDS plots above i think show why it was placed there and why it is ambiguous.

[npcompl33t]

You are not allowed to view links. Register or Login to view.. You can play around with the sorting mechanisms, and view the individual page/bifolia level variation across all the runs from the above charts.

I believe this ordering is more robust than my original chronology, and it better communicates the uncertainty and ambiguity around specific page/bifolia ordering. 

I hope this level of testing and information around the stability helps to improve confidence in the overall broad chronology, while explicitly showing sections that exhibit more ambiguity or uncertainty.

[npcompl33t]

@jorge_stolfi -- this should help clear things up. This plots all of the ensemble lines at once. You can see they aren't randomly dispersed, they cluster around a few equilibria. Overlapping segments on the chart above mean that there are two possible equilibriums, most lines route through any one area in just 1 or 2 ways, despite individual random variance in the actual paths.

   

   


PS: As a data person that loves charts,  i think these are some really cool charts. And I think there is actually a ton of information in these the other charts missed, with respect to stable ordeerings. i think i probably need to spend some time studying these.


EDIT: Replaced charts with one that shows order as a color scale so you can see the directionality.

[Jorge_Stolfi]

The fact that all pages of each sub-section plot at about the same t value on each curve says that the diameter of each cloud (say, the RMS of the JS distance between pages in that cloud) is small compared to the overall span of the curve.

Not quite. It means that those two sections are estimated to fall within the same t slot, and are probably adjacent in the time.

In that item I was referring to the pages of one cloud, not to two clouds.

Let's fix on sub-section H-1a,.  For each curve C, all pages p of cloud 1a projected onto the curve at positions t(p,C) very close together around some mean t value tbar©; say 0.05±0.01. (Isn't that so?) (Except for an outlier about 1/3 the way down the list; let's ignore that).  We can replace that curve C by a straight line segment which is tangent to the curve at that mean value tbar©,.  The projections of each page p on that line segment will be practically the same,and thus all within 0.05±0.01.

The fact that all pages p have projections tbar(p,C) = ~0.05±0.01 says that the extent of the cloud H-1a along the direction of that axis is about ±0.01 times some length L that is approximately the length of the whole curve.

For the plot in question you used dozens of curves; that is, you projected each page on dozens of different axes.  For every curve C'  (IIUC) the projections t(p,C') of all pages p were close together, only around some different value tbar(C',1a) that varied between ~0.00 and ~0.20 depending on C'.  Which means that, along all those directions in R^256, the extent of the cloud H-1a is equally small, namely about ±0.01 times L.

That is why I drew H-1a as a small smudge on my sketch.  Maybe H-1a is much longer in other directions of R^256, but that elongation does not show up in your plots.

And the same applies  to every other sub-section except 1.5a.  The fact that, for each curve C, the pages of the sub-section S project to a narrow range of t values tbar(C,S)±0.01 says that the extent of that sub-section S, along any of the tangent directions of those curves, is  ±0.01 times L.

Sub-section H-1.5a is a bit different, because for each curve C the projections t(p,C) of the points in that cloud are spread over a longer range, say tbar(C,1.5a)±0.03.  That is why I drew that section as a smudge that is elongated in the direction of the curves.

The distance of each cloud to the curve is very roughly proportional to the spread of its t values on the plot, assuming that most of this spread is due to change in the slope of the curve rather than the curve itself moving in the general t direction.

No, the spread of the cloud has nothing to do with the distance from each to the curve.

Ah, thanks.  Your nice plot clarifies things a lot:
You are not allowed to view links. Register or Login to view.
This plot shows that indeed the t positions shift by up to 0.20 depending on the curve C mainly because the curve itself shifts -- not because the direction of projection changes.

But another important information I get from this plot is that the set of all the curves is partitioned into two compact and well-separated families, F1 and F2.  Thus the theory of discontinuous projection that I had guessed to explain the split plot of B-4a and S-4c is unnecessary.  The two parts of each plot are just the t values for the curves of F1 and for those of F2.

The cloud pairs 1a+1b, 3c+4c, 3a+3b, 5a+6 were drawn as single clouds because they cannot be split based on the plot.  Maybe, as you say, they can be split in other dimensions; but those dimensions apparently do not affect their projection on any of the curves.

No, it has to do with the ambiguity as in my city example ...

Do you mean that, for curves from F1, subsection H-1a projects around t = 0.05 and H-1b projects around t = 2.05; while for F2 curves, it is the reverse -- H-1a projects around t = 2.05 and H-1b projects around t = 0.05?

Or is the case that, for F1 curves, both subsections project near  t =0.05, while for F2 curves both project around t = 0.25?

If it is the latter, that is evidence that, in that sub-pace at least, the two sections are practically coincident.

This is accurate, You are not allowed to view links. Register or Login to view. is quite far from the curve, the farthest such outlier by far. Its assignment to Hb is listed as tentative, the MDS plots above i think show why it was placed there and why it is ambiguous. I absolutely think this needs more research to untangle f86v4, it could be key to discovering the ordering of these sections.

You should consider at least possible (if not expected) that some (if not all) of the statistical differences between pages and sections are due to topic, style, etc -- not to time. Then You are not allowed to view links. Register or Login to view. and other "hard to place" pages will be no big mystery. 

Page You are not allowed to view links. Register or Login to view. explains that girls bathing on blue water pools should watch out for aardvarks.  They come out of their burrows (where they hide from aasvogels) on warm afternoons to mate and gorge on aaliis berries.  The text warns that they should not be confused with aardwolves.  It explains that there are four types of aardvark, the blue aardvark, the teal aardvark, the red aardvark, and the pink aardvark (the colors being purely theoretical, not actual).  Each type can be scared away by wearing a shower-cap of the proper color and reciting "by Aaron and Aalar, I command AAALL aardvarks to stay AAAWAY! Aaah! Aaah! Aaah!"   The page is an outlier only because of all those "aa".

Seriously, that page obviously does not belong to Herbal, A or B.  It should be put into Bio, even if it is an outlier there too.

Being an outlier means that it has some rather unusual digraph frequencies.  If you want to understand its puzzle, you should identify the anomalous digraphs, then look for the words that contain them.  You may find that its anomaly is due to a single "unusual' word, or maybe a few such words, that are used quite a number of times on that page, and nowhere else.  (I hereby propose the term "aardword" for that concept.    )

By the way, you should just ignore the pages that cannot be clearly classified into any of the major sections sections by their image contents, including f1r, f57v, f58r, f58v, f76r, f85r1, f85r2, f86v6, f86v5, f86v4, f86v3, You are not allowed to view links. Register or Login to view. from line 31 on, and of course f116v.  I guess that you want to include those pages because you see them as intermediate stages between the main sections.  But you should check that afterwards, if and when you think you figured out the order of the main sections.  Until then, including those pages in the analysis will only distract and confuse.

All the best, --stolfi

[npcompl33t]

@Jorge_Stolfi I'm going to try to respond to all of your points and questions, but let me know if I miss anything. I appreciate your continued dialog about this.

First, a small correction, you refer to the above charts as showing 'dozens' of curves, they show 1,600 curves, so several orders of magnitude off.

But another important information I get from this plot is that the set of all the curves is partitioned into two compact and well-separated families, F1 and F2.  Thus the theory of discontinuous projection that I had guessed to explain the split plot of B-4a and S-4c is unnecessary.  The two parts of each plot are just the t values for the curves of F1 and for those of F2.

This is exactly correct, I'm glad the plots helped here. Collapsing to two curves F1 and F2 may be a bit of an oversimplification but I think for the purposes of our discussion it is fine. Just to be clear though, the saved fits contain combinations of:

• 1a before/after 1b
  
• bio 4a before/after stars 4c
  
• Herbal B 3a before/after Herbal B 3b
  

So really it can't be reduced to just two fixed global sequences.

Do you mean that, for curves from F1, subsection H-1a projects around t = 0.05 and H-1b projects around t = 2.05; while for F2 curves, it is the reverse -- H-1a projects around t = 2.05 and H-1b projects around t = 0.05?

Or is the case that, for F1 curves, both subsections project near  t =0.05, while for F2 curves both project around t = 0.25?

If it is the latter, that is evidence that, in that sub-pace at least, the two sections are practically coincident.

Your first example is correct: when H-1a projects around t-0.05, H-1b projects around t-2.05, and for the F2 curves it is the reverse. Neither F1 nor F2 shuffles H-1a and H-1b together. So F1 or F2 curve places 1a and 1b as occupying the same position.

In that item I was referring to the pages of one cloud, not to two clouds.

Let's fix on sub-section H-1a,.  For each curve C, all pages p of cloud 1a projected onto the curve at positions t(p,C) very close together around some mean t value tbar©; say 0.05±0.01. (Isn't that so?) (Except for an outlier about 1/3 the way down the list; let's ignore that).  We can replace that curve C by a straight line segment which is tangent to the curve at that mean value tbar©,.  The projections of each page p on that line segment will be practically the same,and thus all within 0.05±0.01.

The fact that all pages p have projections tbar(p,C) = ~0.05±0.01 says that the extent of the cloud H-1a along the direction of that axis is about ±0.01 times some length L that is approximately the length of the whole curve.

For the plot in question you used dozens of curves; that is, you projected each page on dozens of different axes.  For every curve C'  (IIUC) the projections t(p,C') of all pages p were close together, only around some different value tbar(C',1a) that varied between ~0.00 and ~0.20 depending on C'.  Which means that, along all those directions in R^256, the extent of the cloud H-1a is equally small, namely about ±0.01 times L.

That is why I drew H-1a as a small smudge on my sketch.  Maybe H-1a is much longer in other directions of R^256, but that elongation does not show up in your plots.

And the same applies  to every other sub-section except 1.5a.  The fact that, for each curve C, the pages of the sub-section S project to a narrow range of t values tbar(C,S)±0.01 says that the extent of that sub-section S, along any of the tangent directions of those curves, is  ±0.01 times L.

Sub-section H-1.5a is a bit different, because for each curve C the projections t(p,C) of the points in that cloud are spread over a longer range, say tbar(C,1.5a)±0.03.  That is why I drew that section as a smudge that is elongated in the direction of the curves.

This is reasonable for locally straight projections away from endpoints. However,  projections onto a curved or endpoint limited path can complicate the straight-line approximation. Many projections land at curve bends, where there is no single tangent. That limits how literally we can apply the straight-line approximation.

For individual pages, within a single fit, the typical central 80% of positions spans approximately 10.5% of the curve for 1a and 11.0% for 1b. For pooled bifolia, it is lower, between 6-8% roughly.

1.5a is not unique elongated, its typical within-fit width is about 7% for pages and 6% for bifolia.

Ultimately the above plots only capture the 1st 3 dimensions of a 256d space, so its natural they don't show all or even the majority of the distances. I'll share the MDS5 plots below, as they capture another dimension specifically useful for evaluating the end orderings.

Page You are not allowed to view links. Register or Login to view. explains that girls bathing on blue water pools should watch out for aardvarks.  They come out of their burrows (where they hide from aasvogels) on warm afternoons to mate and gorge on aaliis berries.  The text warns that they should not be confused with aardwolves.  It explains that there are four types of aardvark, the blue aardvark, the teal aardvark, the red aardvark, and the pink aardvark (the colors being purely theoretical, not actual).  Each type can be scared away by wearing a shower-cap of the proper color and reciting "by Aaron and Aalar, I command AAALL aardvarks to stay AAAWAY! Aaah! Aaah! Aaah!"   The page is an outlier only because of all those "aa".

Being an outlier means that it has some rather unusual digraph frequencies.  If you want to understand its puzzle, you should identify the anomalous digraphs, then look for the words that contain them.  You may find that its anomaly is due to a single "unusual' word, or maybe a few such words, that are used quite a number of times on that page, and nowhere else.  (I hereby propose the term "aardword" for that concept.    )

I specifically tested this, removing the 3 largest word types on an already text-light page only removes a few % points of the divergence. A handful of repeated words does not explain most of its difference. The differences come from different word construction, specifically use of the bigram ol (apart from the single ol words, removing those does not change this), along with eo, initial ch, and o-t. 

You should consider at least possible (if not expected) that some (if not all) of the statistical differences between pages and sections are due to topic, style, etc -- not to time. Then You are not allowed to view links. Register or Login to view. and other "hard to place" pages will be no big mystery. 

Topic can produce distinct clusters. What it cannot produce is a gradient. For example, there is no reason that arranging topics in the order i've listed would result in chol's counts gradually decreasing at each step. On a broad scale, the exact amount of 1a vocabulary retained from step to step declines gradually at each step. Topic cannot explain this. Style can, if it is a function of chronology or time.

Seriously, that page obviously does not belong to Herbal, A or B.  It should be put into Bio, even if it is an outlier there too.

By the way, you should just ignore the pages that cannot be clearly classified into any of the major sections sections by their image contents, including f1r, f57v, f58r, f58v, f76r, f85r1, f85r2, f86v6, f86v5, f86v4, f86v3, You are not allowed to view links. Register or Login to view. from line 31 on, and of course f116v.  I guess that you want to include those pages because you see them as intermediate stages between the main sections.  But you should check that afterwards, if and when you think you figured out the order of the main sections.  Until then, including those pages in the analysis will only distract and confuse.

These were all checked, and not every page was placed into a section. That is why there is an 'unclassified' label on the initial charts i shared. Each page was placed into a group only if it made sense, the relative certainty was established for every individual assignment. It was placed in 3b, because that is legitimately its best match, in the MDS plots i shared it genuinely looks like it is a part of that grouping.

Again, you are also only viewing the top 3 dimensions in a 256d space, you do not have the information to make that call from just the above plots alone. I've shared the full 256d MDS space if you want to take a deeper look. 

Ignoring pages that seem to cause contradictions is exactly what can lead to warped perspectives about the MS as a whole, I do not think sweeping away the difficult pages under the rug will lead to a better understanding of the MS.

[npcompl33t]

Here are the MDS1 vs 5 and MDS1 vs 6 plots. These show how the ambiguity related to the ending sections arises. They almost form a coherent circle of points. The exact ordering of these segments depends on where you enter the circle. This makes sense, if the points all form a ring, you can be reasonably sure of the ordering but starting at different points on the circle results in different sets of orders.

If Phileas Fogg began his journey in New York as opposed to London, suddenly it is on opposite sides of the path from San Francisco, even though the two are spatially adjacent. This ordering swap does not mean that there is no difference between San Francisco, New York, and London. It does not imply any kind of inconsistency with respect to there being a path connecting all of them.