ReneZ > 01-09-2019, 05:49 AM
-JKP- > 01-09-2019, 07:42 AM
Koen G > 01-09-2019, 10:26 AM
(01-09-2019, 12:58 AM)nablator Wrote: You are not allowed to view links. Register or Login to view.With the right color gradient (I'm sure it can be improved) the pages with the most common vocabulary should be easy to find in this Excel sheet. It displays the percentage of common vocabulary between any two pages for ZL2.
Koen G > 01-09-2019, 10:33 AM
(01-09-2019, 05:49 AM)ReneZ Wrote: You are not allowed to view links. Register or Login to view.One problem is how to normalise the results. The text on pages can have many different lenghts.
Obviously, one can get more matches on longer pages.
nablator > 01-09-2019, 12:09 PM
(01-09-2019, 10:26 AM)Koen G Wrote: You are not allowed to view links. Register or Login to view.Nice! Is this like a zoomed-in version of Rene's graphs?
Quote:What if this exercise were done only with rare words? Then you would specifically look for themes common to certain pages.
Koen G > 01-09-2019, 12:20 PM
(01-09-2019, 12:09 PM)nablator Wrote: You are not allowed to view links. Register or Login to view.I would be very surprised if vocabulary (rare or not) has anything to do with themes and meaning. Many attempts to correlate vords with themes and illustrations and find grammatical structures would have showed better results than basically nothing. Currier was right: vords are not words.
MarcoP > 01-09-2019, 12:55 PM
(01-09-2019, 05:49 AM)ReneZ Wrote: You are not allowed to view links. Register or Login to view.One problem is how to normalise the results. The text on pages can have many different lenghts.
Obviously, one can get more matches on longer pages.
Quote:The Jaccard coefficient measures similarity between finite sample sets, and is defined as the size of the intersection divided by the size of the union of the sample sets:
nablator > 01-09-2019, 01:06 PM
(01-09-2019, 12:55 PM)MarcoP Wrote: You are not allowed to view links. Register or Login to view.With some googling, I found that You are not allowed to view links. Register or Login to view. was designed to address this problem.
It is quite simple and I expect it should work in our case:
RobGea > 01-09-2019, 01:08 PM
Koen G > 01-09-2019, 03:54 PM