29-06-2020, 11:27 PM
(29-06-2020, 07:18 PM)Alin_J wrote: You are not allowed to view links. Register or Login to view.
...
Regarding treating certain or uncertain spaces as word-separators, for the 101 transcription if you treat them as real separators you get 9905 different word-types but not treating them as real spaces (concatenating "word parts") you get 10641 different word-types, a significant number more. I have therefore subsequently treated uncertain spaces as real separators. My argument is that if these spaces would have been random mistakes/seemingly spaces they would have likely lead to more different word-types when treating them as separators, not less, because they would shop off many legit word-types into new ones. Instead, concatenating real word-types randomly should lead to more word-types which is what is also observed for the text.
I find this one of the most difficult parts of creating a VMS transcript, the evaluation of spaces. I don't think the half-spaces are random mistakes. They tend to break in the same places with reference to the glyphs. Not always, but much of the time.
And, as in the two examples, I posted above, the narrow spaces, even though they are clearly visible, are substantially narrower than the wide spaces on either side (and narrower than the other spaces on the line). What I can't tell is if this is an unconscious thought-process being revealed (the perception of certain clusters as separate from each other in some way), or if there is a deliberate creation of two kinds of spaces.