11-05-2023, 09:49 AM
(This post was last modified: 11-05-2023, 11:16 AM by Mark Knowles.)
I was wondering about the Levenshtein distance per letter. By this I mean taking the Levenshtein distance and dividing it by the word length, of the longest of the two words that you are comparing.
You would then calculate the mean Levenshtein distance per letter and sort accordingly. This would reduce the bias towards longer words. However this doesn't seem like a completely adequate solution.
I was thinking what makes me consider some words more distinctive than others. I think the following effect my thinking:
How unusual is the first letter of the word relative to other words?
How unusual is the second letter of the word relative to other words?
..
How unusual is the last letter of the word relative to other words?
(Are the letters at the beginning of the word more important than the end of the word?)
So many words start 'ok', so these should be flagged up.
Are the common relative position or sequence of letters important when comparing words?
The Levenshtein distance allows deletion, insertion and substitution. What about shifting? What about common substrings? Maybe one common substring shift could count as a single shift?
You would then calculate the mean Levenshtein distance per letter and sort accordingly. This would reduce the bias towards longer words. However this doesn't seem like a completely adequate solution.
I was thinking what makes me consider some words more distinctive than others. I think the following effect my thinking:
How unusual is the first letter of the word relative to other words?
How unusual is the second letter of the word relative to other words?
..
How unusual is the last letter of the word relative to other words?
(Are the letters at the beginning of the word more important than the end of the word?)
So many words start 'ok', so these should be flagged up.
Are the common relative position or sequence of letters important when comparing words?
The Levenshtein distance allows deletion, insertion and substitution. What about shifting? What about common substrings? Maybe one common substring shift could count as a single shift?

