About the generation of similar words

Index
About the generation of similar words
RE: About the generation of similar words

evandrunen > Yesterday, 06:45 PM

(Yesterday, 03:36 PM)quimqu Wrote: You are not allowed to view links. Register or Login to view.....
To test this, I stopped grouping tokens into “families” (that was too unstable) and instead looked only at direct pairs of similar tokens (Levenshtein = 1). Then I compared their contextual profiles. The result is quite strong: tokens that are formally similar have much more similar contexts than expected. Not slightly more. A lot more.
...

Could you explain what you mean by "context"? Sorry if I missed it.
RE: About the generation of similar words

quimqu > Yesterday, 07:41 PM

(Yesterday, 06:45 PM)evandrunen Wrote: You are not allowed to view links. Register or Login to view.Could you explain what you mean by "context"? Sorry if I missed it.

Context means the environment in which a token appears. It is not about meaning, but about surroundings. It includes which tokens tend to appear nearby, how similar they are, how dense or repetitive the local area is, and also where the token sits in the structure, like its position in the line or the section of the manuscript.
Next Oldest Next Newest