Proposed Manuscript Chronology

[Jorge_Stolfi]

The horseshoe does NOT manifest as a result of two significant factors, one monotonically increasing and the other increasing/ decreasing, as you stated.

My statement was not about the causes of the "horseshoe", it was just an objective description of what it is.  Being an arc of a curve, there is a direction along which it is monotonic, and another direction, more or less orthogonal to it, along which it it increases then decreases.

Any statement about the causes and meaning of such "horseshoe" cannot come from the data by itself.  They will depend on external assumptions.

These are four clusters of points in R^4.  Is there a horseshoe in this dataset?
   

Before the "horseshoe" stuff, archaeologists used to deduce the chronological order of different sites by building a table where each row was one site and each column represented an attribute (like a particular feature of the pottery style).  They would then try to rearrange the rows so as to make consecutive rows as similar as possible.  

But that method (like the more modern computer-based ones) relied on the assumption that the styles changed gradually over time across the various sites, more or less synchronically.  And they often knew that progress went from more primitive to more sophisticated -- like, from plain pottery to painted to glazed, with returns to more primitive forms being unlikely.  

Without those assumptions, the final order of the table could be the reverse of chronological order.  Or it could be that the first and last rows were the most ancient sites, with two completely different styles, and the middle row was the result of the two populations merging and settling on a style that shared features of both.  O the other way around, namely the guys in the middle row split into two populations that diverged.  Or a zillion other possible scenarios.  And if the columns represented types of pollen or isotopes of copper, the assumption of continuous change with time would not be warranted.

It happens because of distance saturation, combined with multidimensional scaling. It is an artefact of the MDS process, which is why hundreds of papers have been written about correcting it.  When calculating distances between objects, particularly when certain features are not present across all observations, distances get warped. Points at the end appear more similar than they actually are, because they each lack features the other possesses.   

For example, in voynich, the bigram ED is completely absent in Early Herbal A compared to Bio, where it is prevalent. ... pharma,, 1a, and bio, all appear equally distant.

Yes, if you take three points in any space with two or more dimensions, with high probability they will form a triangle.   

If you try to fit a curve through them, by any method, with high probability it will be an arc of a curve.

But the fact that one can fit a curve through the three points is not remarkable, and that curve does not necessarily mean something.  In particular, the order it implies for the three points need not match any meaningful order for the things represented by those points.  

Because 1a and bio cannot be compared on certain features, due to one simply lacking them.

A probability distribution on n things is a point in the canonical (n-1)-dimensional simplex K_n of R^n, whose vertices are the cardinal vectors or R^n.   A point that is nonzero in only m coordinates lies on an (m-1)-dimensional face of K_n.  Thus if your data vectors have lots of zero coordinates, a curve that passes through or near a given set of k points in any specified order will probably have to have degree k-1 and will twist around like crazy, grazing the k faces. 

my conclusions incorrect

Your conclusions are unwarranted, independently of the "horsehoe" issue, because there is no reason to expect that the "temporal order" (whatever that means) of the sections, pages, or bifolios is in any way related to the arrangement of the page data points in 256-space, or any projection thereof. 

So one of the most studied and cited cases, with over 120 citations on google scholar, You are not allowed to view links. Register or Login to view., is not correct. And not only this, but the You are not allowed to view links. Register or Login to view., across many different disciplines, are all written by people collectively experiencing mass horse shoe delusion?

Yes.

There are easily 10 times more "scientific" papers and books on metaheuristics.  Ditto for the "Technical Analysis" of stock prices.  There must be thousands of papers that compare data retrieval or pattern recognition methods by comparing their F-scores -- and many even assume that any internal parameter or threshold should be adjusted to maximize the F-score.  Every textbook on data structures and algorithms begins by "demonstrating" that polynomial-time algorithms are much more efficient than exponential-time ones, and then goes on to compare algorithms by their asymptotic worst-case time alone.   And many theoretical computer scientists even define "fast" and "efficient" to mean "polynomial time"; and they will state without blinking that Turing's non-computability is the most important concept in computer science, that every computer professional absolutely must know, and that P vs NP is the most important open problem in that field.

There are many Big Money neuroscience labs that were created to study the human brain by doing computer simulation of artificial neuron networks with millions of virtual neurons connected in a random topology.  There are hundreds of papers that instead study such networks theoretically, under the assumption that the parameters are tuned at the critical value that causes massive avalanches of firings.  

Yet all of that is bullshit.  Techno-scientific superstition.  Built on assumptions that are not only unproven, but easily proved to be false.

So, no, I am not impressed by the 1200 papers on the "Guttman horseshoe".

All the best, --stolfi