23-04-2017, 11:38 PM
(This post was last modified: 23-04-2017, 11:41 PM by DonaldFisk.)
(23-04-2017, 10:58 PM)nickpelling wrote: You are not allowed to view links. Register or Login to view.Looking more closely at your pages, does it not concern you that your clustering may be broken?
Many previous clustering studies have found a strong clustering effect between recto and verso sides of the same folio (with the exception of the nine rosette foldout folio, as Rene Z reminded me recently), yet this is apparently not as noticeable in your clustering, as far as I can see.
Clustering via k-means or any of the multitude of funky variants it spawned is only as good as the dimensions of data that go in, which suggests to me that you may not have got this step quite right. Just sayin'.
Are you referring to You are not allowed to view links. Register or Login to view.? It didn't use PCA. It looks like it used an ad-hoc approach.
I think my algorithms are correct. I used three standard ones, PCA, k-means and minimum spanning tree. The PCA plot in You are not allowed to view links. Register or Login to view. agrees with the one on Sarah Goslee's You are not allowed to view links. Register or Login to view.. I then considered separately the herbal, biological, and text pages, and modified the herbal clusters as I explained in You are not allowed to view links. Register or Login to view.. For clustering, I used the first two principal components.
In any case, clustering is very much a black art. It might be worth checking whether the folio mismatches are with nearby points assigned to different clusters (the so-called Copenhagen-Malmo problem), in which case there would be a good case for reassigning some points to different clusters.