Common vocabulary between pages

[MarcoP]

One problem is how to normalise the results. The text on pages can have many different lenghts.
Obviously, one can get more matches on longer pages.

With some googling, I found that You are not allowed to view links. Register or Login to view. was designed to address this problem.
It is quite simple and I expect it should work in our case:

The Jaccard coefficient measures similarity between finite sample sets, and is defined as the size of the intersection divided by the size of the union of the sample sets:

For any page A, longer pages B are more likely to get more matches, but a longer B will also result in a larger denominator (union i.e. total number of different types in A and B), compensating the larger nominator (intersection i.e. matches).