"The Currier languages revisited" revisited

[kckluge]

dimensionality reduction using a heuristic PCA-like method rather than PCA

Why not using PCA? That would make for a more meaningful comparison with Rene's results

I was ("Applying PCA, the first two dimensions found capture 48% of the covariance in the 40-D data."). I should probably have done that as a table to avoid ambiguity. What Rene did is described on his page as follows:

The following procedure will not necessarily find this maximum, but it will find something near to the maximum.

1. Locate the bigram which has the most varying distribution over all pages. This means: locate the component of all vectors which has the largest rms about mean.
   
2. For all bigrams, find out their covariance with this bigram. This means, estimate a linear relationship (without offset) which best fits the value of each component vs. the value of the component found above. Save the slope of this linear relationship.
   
3. The vector containing all these slopes has the value 1 at the component found above, and values between -1 and 1 at all other components. Normalise this vector and the base vector for the prime direction has been found.
   

Now, the coefficient along this base vector may be computed for each page vector. The contribution of this base vector may then be subtracted from each of the page vector, meaning that the hypercloud collapses to a space with dimension one less than before.
After this last step, the procedure may be repeated, and it will automatically find the next most important base vector, which will be perpendicular to the first. This whole procedure may be repeated several times.