Proposed Manuscript Chronology

[npcompl33t]

This may actually explain the odd behavior at the end of the graph, with respect to stars and Bio. You are not allowed to view links. Register or Login to view.. 

In the egyptian pottery example, you would expect pottery styles to all evolve over time. However, if for some reason, some of the potters get separated, then their styles would start to diverge from each other. Assuming the entire sequence to be a single path would actually cause issues once the split occurs.

The above chronology has scribes 2,3 and 5 all working at the same time during the 3a Herbal B section. This is the only section where 3 people are active at the same time. 

   

If instead there is a fork, and 2 LFD hands diverge from one another in style, that would lead to poor ordering of the final herbal/stars sequence.


This type of branching path is also compatible with the data, as follows: 

   


This variation would obviously have substantial implications for interpreting the manuscript. The chart above shows this is a possibility, it does not prove that this configuration is better than the sequential ordering of these sections. It is an attempt to explain the odd nonlinear drift from the horseshoe at the Stars/Bio end. I will try to dig into these more and develop a more comprehensive perspective around these possibilities.

[Jorge_Stolfi]

Respectfully, again this is not accurate. if a guttman horseshoe is present, it means there is a dominant, sequential order underlying the data. Guttman Horseshoes appear (not just in the VMS) when a sequential ordering dominates the other features in the data.

And I respectfully insist that this does not make sense.  Again, what the plots show is five fairly well-separated roundish clouds of points, originally in a space of dimension d much greater than 4 but projected onto some 2D subspace thereof.  

Again, there will be directions in that d-space where the centroids appear in the order 1-2-3-4-5, but other directions where they appear in the order 3-5-2-4-1 or any other order.  

You may use some criterion like PCA to chose a particular direction in d-space, and that will define an order for the centroids, namely by projection on an axis parallel to that direction.  But that order is not meaningful.   You can see this if you consider a linear transformation of the data that is not an isometry: that can completely change the PCA direction, and hence the apparent order of the clouds.

Once you have picked a direction, you can always fit a quadratic (or higher polynomial) curve to the centroids as a function of position along that direction.  But that "horseshoe" is no more significant than the chosen direction itself.  Again, it depends totally on the coordinate system you use, and on how you pick the direction that determines the independent variable of the fit.

Think of picking any three countries in Europe and plotting the coordinates of their major cities as points on a 2D plot. You get three clouds of points.  If you pick a direction by PCA or any other method, you will get an ordering of those three countries, and a "Guttman horseshoe" from that data.  But obviously that order will not be significant.

All the best, --stolfi

[npcompl33t]

Respectfully, again this is not accurate. if a guttman horseshoe is present, it means there is a dominant, sequential order underlying the data. Guttman Horseshoes appear (not just in the VMS) when a sequential ordering dominates the other features in the data.

And I respectfully insist that this does not make sense.  Again, what the plots show is five fairly well-separated roundish clouds of points, originally in a space of dimension d much greater than 4 but projected onto some 2D subspace thereof.  

Again, there will be directions in that d-space where the centroids appear in the order 1-2-3-4-5, but other directions where they appear in the order 3-5-2-4-1 or any other order.  

You may use some criterion like PCA to chose a particular direction in d-space, and that will define an order for the centroids, namely by projection on an axis parallel to that direction.  But that order is not meaningful.   You can see this if you consider a linear transformation of the data that is not an isometry: that can completely change the PCA direction, and hence the apparent order of the clouds.

Once you have picked a direction, you can always fit a quadratic (or higher polynomial) curve to the centroids as a function of position along that direction.  But that "horseshoe" is no more significant than the chosen direction itself.  Again, it depends totally on the coordinate system you use, and on how you pick the direction that determines the independent variable of the fit.

Think of picking any three countries in Europe and plotting the coordinates of their major cities as points on a 2D plot. You get three clouds of points.  If you pick a direction by PCA or any other method, you will get an ordering of those three countries, and a "Guttman horseshoe" from that data.  But obviously that order will not be significant.

All the best, --stolfi

You are correct that typical PCA direction is not necessarily meaningful. However, that is not what this is showing. For a horesshoe to mechanically appear, a single linear ordering, must be the most significant part of the feature space. Otherwise it would not  appear. By appearing, it means there is a meaningful order.

Typially, this is something like temperature, or time. I'm sorry but you are wrong here.

Furthermore, the clusters only appear isolated because the voids are in areas where the missing bifolios would naturally slot. They are between pharma, and f58. F58 is known to be missing 3 bifolio. The missing quires, the foliation would put them exactly in the voids. There is an obvious linear sequence in the data here.

The voids ive circled below, are f58, and before and after pharma. If you filled these in, it would not appear to be isolated clusters. They appeaer isolated because we are missing data.

[npcompl33t]

In the Egyptian pot example, if a rival pharaoh destroyed all of a previous pharaoh's pots, a gap would appear in the data. We would be able to infer we were missing pots from a certain timeframe, because of a discontinuous region. 

This currently is where i would estimate the missing bifolio to go, based on the missing voids in the proposed path. I'll dig into the data and see if there is anything else that might help here. These mostly either line up with non-missing pages in the same bifolio, or based on nearby section foliation. 

There is still a decently sized void In between Stars/Herbal B and Bio/ other stars, that no obvious missing section would fit into -- its possible the missing stars pages would go there.


MDS1 vs MDS2
   


MDSI vs MDS3
   

[RadioFM]

@You are not allowed to view links. Register or Login to view. Thanks for the long response. I'll chew on it and come back when I have time to go through that and the ecology papers properly.

---

@Stolfi I agree the order depends on the axes on which the data is projected on. If the data follows a sequence-like pattern (like horseshoe) in some direction, then surely that sequence pattern will be apparent in at least 1 PC (PCA1) if using the Euclidean norm, but I think the reciprocal might be true as well: if a horseshoe pattern emerges, where the clusters are sequential in PCA1, then in that axis (PCA1) the data must be sequence-like distributed, and what's more I suspect a partial order may be preserved to some extent between data points.

Of course, interpreting that sequence and its meaning depends on what PCA1 even stands for, and whether we could canonically ascribe it to "time" is a whole different matter. Because the plot is based on pairwise divergence, PCA1 probably measures a pondered "distance" to many folia, as in "It's 20% similar to f33v's glyph transitions, 5% similar to f66r's, 13.5% similar to 107v, ···", which is hard to interpret. On that topic, @npcompl33t Is the PCA data open-sourced? Could we take a peak on the MDS axes and their coordinates to try and interpret that? Sorry if you've mentioned this in your main thread, I'm having a hard time catching up on threads lately.

[MarcoP]

• Proposed: Bio represents a comparatively stable production phase, with its internal differences mostly unrelated to progression.
  
• Alternative: Bio contains a stronger internal progression that the sequence follows.
  

Thank you for the detailed reply and for visually comparing the two different models!

Of course, the fact that the Proposed seriation places f116 very near to the end of the sequence is consistent with both the binding of the MS and the almost-blank page at f116v.

On the other hand, people have suggested a progression in the Bio section, in particular based on illustration subject and layout: Marginal-Pipes (Q13A) vs Central-Pools (Q13B). See for instance:

• You are not allowed to view links. Register or Login to view.
  
• You are not allowed to view links. Register or Login to view.
  

It would be interesting to see if the Alternative seriation matches the image split between Q13 pages:

Q13a – medical – biological – Galenic

76r/76v – bifolio 2 > medical
80r/80v – bifolio 5 – flipped > medical
77r/77v – bifolio 3 > biological
82r/82v – bifolio 3 > Galenic
79r/79v – bifolio 5 – flipped > Galenic
83r/83v -bifolio 2 > Galenic

Q13b – Balneological

84r/84v – bifolio 1 – flipped > balneological
78r/78v – bifolio 4 > balneological
81r/81v – bifolio 4 > balneological
75r/75v – bifolio 1 – flipped > balneological

There are a couple clues that say Q13b was written after Q13a

[npcompl33t]

@npcompl33t Is the PCA data open-sourced? Could we take a peak on the MDS axes and their coordinates to try and interpret that? Sorry if you've mentioned this in your main thread, I'm having a hard time catching up on threads lately.

• I actually have created You are not allowed to view links. Register or Login to view.! 
  
• I also had open sourced You are not allowed to view links. Register or Login to view.
  

However, i did not directly publish the underlying transition or direct MDS data. So i went ahead and did that.

• You are not allowed to view links. Register or Login to view..
  

I want to be clear though, I am not claiming PCA1,MDS1, or any single dimension = time. 

In fact, when a Guttman Horseshoe appears, it means you can not simply ascribe any one single MDS or PCA dimension to the dominant sequential ordering. The linear path moves through multiple PCA dimensions. This is why the Guttman horseshoe is often considered a confound to PCA analysis, because it means there is a single dominant factor that is not directly captured by any one PCA dimension. The point of fitting the curve to the data is to recover this dominant sequential ordering.

In the Flinders Petrie example, he recorded different aspects about pottery. You can imagine styles changing, perhaps necks slightly elongate for a few years, then start to slowly widen, then maybe gain fluted edges. The PCA captures all of these differences but cannot put them all on a single dimension. When David Kendall did MDS on Flinders' pot data, he noticed the same horseshoe. A single dimension did not correspond to the pottery chronology. He was able to recover the full chronology, by tracing the path across multiple dimensions, and that is what I am doing here. Kendall's work on horseshoes are literally textbook, You are not allowed to view links. Register or Login to view.. 


Another example, here is a 3D plot of the You are not allowed to view links. Register or Login to view.. You can see the horseshoe appearing. This means there is a dominant sequential ordering underlying the data. This sequence is not directly captured by any MDS dimension. It can be recovered though, which is what the plot on the right is.

   


So if a horseshoe appears, it means there is a dominant, single, continuous gradient driving the data. My hypothesis here is that it is time, as it was in Kendall's example. That is my hypothesis, and you might think it represents something else, but a single dominant gradient does exist.

[npcompl33t]

On the other hand, people have suggested a progression in the Bio section, in particular based on illustration subject and layout: Marginal-Pipes (Q13A) vs Central-Pools (Q13B). See for instance:

• You are not allowed to view links. Register or Login to view.
  
• You are not allowed to view links. Register or Login to view.
  

It would be interesting to see if the Alternative seriation matches the image split between Q13 pages:

I just ran this, under the alternative proposal, 7 of 7 Q13b pages follow every Q13a page. The single exception is f81r, which precedes You are not allowed to view links. Register or Login to view. and f77v. Those are the only two reversed comparisons among the 96 possible cross-group page pairs.

This distinction is captured in the same MDS5 dimension that seems to capture most of the variation across the stars-S, stars-T, and bio sections, MDS1 barely contains any discriminatory behavior here. 

The chart below shows how these sections split in both proposed/alternative paths. Note that the two groups are distinguished in the overall MDS space. So regardless of which path is used, there does seem to be a distinction between the two groups. 

   

   

[npcompl33t]

I have tested a lot of historical comparanda, and while the above method can recover their reading order fairly well, virtually none of them  have guttman horseshoes. 

Could scribal shifts cause a horse-shoe?

Here is what the Black Book of Basingwerk looks like. This has a well known orthographic shift in it, the scribe is replaced half-way through, and the new scribe writes in a much more modern style -- an extremely good parallel to test the scribes impact. lets look at the MDS. Very obviously not a horseshoe, so the voynich horseshoe does not seem to be simply due to an abrupt shift in scribes.

   


 

What could cause a legitimate horse-shoe in text?

I did find an actual Guttman Horseshoe, in Napiers casebooks. These are astronomical/medical logbook, that covered 37 years. Across the entire set of casebooks, there is no arch. But for the first few years he was recording, he did not have a consistent formula for how he recorded cases. For example:

• yeres old becomes a bare y or y. around 1600; 
  
• the Latin weekday marker die before 
  
• the planet sign is universal in 1597–1601 and gone by 1603; 
  
• the hourmarker is h or h. in 1597–1599, hor. takes over in 1600–1604, and h. wins out again from about 1605 and stays for thirty years.
  

After about 1603, his method stabilizes and no longer changes. It turns out, the data before 1603, has a horseshoe shape, and tracing the path recovers the chronology of the reports. After 1603, once his method stabilizes, the horseshoe collapses and nothing can recover the chronology.

So this is showing exactly the theory -- small random walk-esque drift along a time axis, exactly like the egyptian pottery style example. Once the style is locked, the random drift ceases and the horseshoe vanishes.  

   



How long of a time frame does the sequence represent?

I also calculated an expected time for the entire manuscript sequence, for both the alternate and proposed route. i did this by looking at the densest region, where pages were produced the fastest, and assuming the time between those points represents the minimum time production time. Then assuming a scribe can write 25 lines per hour, 2 hours for herbal, 4 hours for a zodiac, 3 for a bio page. These are rough estimates based on some studies i could find.

These produce estimates between 200 days and 240 days for the entire manuscript sequence across all scribes. This is just a rough estimate that could probably be better refined. 

   

[Jorge_Stolfi]

Furthermore, the clusters only appear isolated because the voids are in areas where the missing bifolios would naturally slot. They are between pharma, and f58. F58 is known to be missing 3 bifolio. The missing quires, the foliation would put them exactly in the voids. There is an obvious linear sequence in the data here.

I will address the "Guttman horseshoe" issue in a latter post.  Here I must disagree with the above statement.

The "voids" are not significant because they are determined by the outliers, ignoring the bulk of the points.  That is not how one infers whether a set of data points is "clustered" or "continuous".

A more reliable way to tell whether a set of points on the plane is "clustered" is to replace each point by a roundish fuzzy smudge and add the smudges.  Make the smudges just big enough for most of them to overlap their neighbors. 

Then, if the points come from a simple distribution -- like a 2D Gaussian -- you should see a single dark smudge surrounded by a few scattered small smudges, like this:
   
If you instead see something like this
   
you should realize that a single 2D Gaussian distribution is not an adequate model for the data.  A better model would be: a point can be any of three classes, and each class has its own 2D Gaussian distribution, with its own centroid and principal vectors.

You claim that the data points extracted from the VMS pages fit instead a model where, instead of N discrete classes, there is a 2D probability distribution D(t) that depends on a single parameter t, such that for each fixed t you get (say) a 2D Gaussian with centroid C(t), but C(t) varies continuously and smoothly along some curved path as the points are generated.  

If that were the case, the procedure above should give an image like this:
   
But, even without actually building the image, we can see that this is not what we would get.  Each major section would produce a concentrated lump, much like the second image above.  

So the page statistics do not suggest or support that (1) there is an intrinsic order of the pages or (2) there is a gradual transition between pages of different sections.

They do not suggest or support claim (1) even within each section. 

If you arrange the books of a library in a single row, sorted by height, you will get a fairly smooth variation of height from one end to the other.  That of course does not imply that those books are the product of a single process with a continuously varying parameter.   That conclusion is unwarranted even if you do that for each shelf separately.

All the best, --stolfi