Proposed Manuscript Chronology

[Jorge_Stolfi]

You are not allowed to view links. Register or Login to view.about performing MDS on Flinders' tomb pottery data. I have a feeling Stolfi would have argued with Kendall over whether his data actually shows a horse-shoe, his is not as clear as voynich's, even though Kendall objectively reconstructs the correct tomb chronology.

I do argue with Kendall.  He did not reconstruct the tombs' chronology.

   

This is the data in that plot:
   
There is no alignment. Not along a straight line, not along a parabola, not even along an arc of a circle.  The "horseshoe" that Kendall claims "stands out well" is simply not there. It is an illusion that can seen only by those who really want to see it; the canonical example of pareidolia. 

If Kendall can see a horsehoe there, with a bit more optimism he surely will see also a horse. A flying horse, why not.  And Lady Godiva with flaming hair riding a flying horse with a ten-foot lightsaber in one hand and juggling three alligators with the other and ...

What that plot shows is only four well separated clusters of points. Three are fairly compact, one more diffuse.
   
Are those clusters pareidolia too?  I did pick them by eye, but they could be found objectively by clustering algorithms.  Technically, the data points (at least on the plane) can be modeled as independent samples of some smooth probability distribution; and this model will be far more likely if the distribution is the sum of four 2D Gaussian (indicated by the ellipses) than by single Gaussian.

One may say that the four clusters are arranged along a circle.  But they can also be said to be corners of a square, or the tips of four-pointed star.  And that, in fact, is the most un-aligned arrangement possible for four things on a plane.  

Moreover, as I pointed out before, if the original data has four or more attributes (coordinates) for each data point, then we can always choose a projection that puts the four clusters in any arrangement that we want -- in a square, or along a parabola, or even aligned along a line.

Clusters A, B, and D are extended in directions roughly tangent to that alleged circle, but the elongation of C is  extended at an angle to it.

Even if we fit a curve through the centroids of the four clusters, there is no obvious starting point or direction of travel for it.  We could start at any of the four clusters and go either clockwise or counterclockwise.  That is, we can fit four different "horseshoes" to that plot, and get eight possible orderings for the clusters.  Nothing in that plot makes any of these eight possible orderings significantly more "correct" than any other. 

Moreover, even if we know by other means that the the data represents burials made at the same location in four well-separated epochs, we cannot conclude that each of those clusters corresponds to one of those epochs.  In the extreme case, the clusters may correspond to four castes of people, each with its distinctive burial practices, with some burials of all four castes happening in each epoch.

And if the data came from burials at four or more different locations, we cannot even assume or conclude from the plot that their dates fell into four separate epochs.  They could be overlapping in time, or the same location X may have been occupied in two epochs bracketing the times of location Y.  

And even if we assume that each cluster corresponds to one of four separate epochs, there is no justification to assume that any order that we derive from the positions of those clusters reflects the chronological order of those epochs.  It may well be that the earliest inhabitants, the Albanians, had the general style A. Then the place was taken over by the Chaldeans whose burials were of style C. They were followed by the Basques with style B, then by the Dorians with style D.

The large cluster D can be better modeled as three more compact clusters D1,D2,D3 sampled from Gaussian distributions, plus two pairs of points (gray) that may be considered two additional mini-clusters:
   
This probability distribution may be more likely than the four-Gaussians one, if one gives enough probability a priori that there may be more than four classes of tombs.  In this model, clusters D1 and D2 are fairly roundish, while D3 is elongated in the radial direction, almost perpendicular to the circle.  

Anyway, again, there is no justification for assuming that these six clusters represent six separate epochs.  Much less that any of the eight orderings of the clusters round the center corresponds to the chronological order of those six epochs.

And it turns out that the chronological order of the burials, as estimated by the archaeologists, jumps around quite a bit in this plot:
   
While the geometric ordering B-C-D-A  of the clusters roughly matched the archaeologists chrono order of the burials,  there were significant discrepancies, like the point at far left which the archaeologists dated as contemporaneous with those of the C cluster.  Were they wrong? Or is their date correct, and the burial just happened to be in anomalous style?

Another big discrepancy is that the archaologists chrono order jumps back and forth between cluster A and cluster D3, which geometrically occurs well before A along the hypothetical "circle".

More generally, within each of the six clusters there is no visible correlation between ordering along the "gradient" (direction of maximum elongation) of the cluster (or CCW order seen from the origin) and the archaeologists chronological order.  The exception is D2, where the two orders roughly coincide, with only a few exceptions.

That plot may be misleading because the scale of each data coordinate must be arbitrary (like measuring the width of a pot by the radius instead of the diameter).  Changing the scales and units of the data coordinates independently can stretch the MDS1xMSD2 plot in an arbitrary direction by an arbitrary amount.  But stretching it in the NE direction, for example would make it seem "obvious" that the "horseshoe" should start at B and end at C, ordering the four main clusters as B-A-D-C.

So what can be justifiably say about that example?  

• The fourof the geometric clusters -- B, C, D1, and D2 -- indeed seem to approximately correspond to four epochs;
  
• On the other hand, the two geometric clusters A and D3 seem to comprise a single epoch.
  
• The six-cluster model could be a fairly reliable way to assign an undated tomb to one of the five epochs.
  
• The chronological order of the five epochs cannot be deduced from the plot. Not even that it is one of the eight "circular" orders.
  
• The geometric position of each burial within a cluster is generally not correlated with its date. 
  

This example only reinforced my belief that the "Guttman horseshoe" is just another instance of "scientific superstition": a claim, concept, or method that is wrong, nonsensical, or not justified by real data, but gets universally accepted within a discipline just because everybody in that field accepts it. Like the dozens of key patterns of stock technical analysis, the F-score in information retrieval, the entire field of "meta-heuristics", the "Turing machines are fundamental" and "polynomial is faster than exponential" of theoretical computer scientists, the dozens of types of neurons and "neural avalanches" in computational neuroscience, the discrete bonds and "orbitals" or chemistry...

All the best, --stolfi