Confirmation that spaces are really spaces

[quimqu]

Out of curiosity, if you test a very simple rule that q always starts a word, n always ends a word and y ends a word unless this will create a single character word (.y.), what proportion of spaces would be correctly recovered? Just looking at the text it appears as if more than 50% of spaces can be explained by certain characters requiring a space before them, certain characters a space after them or, in the case of y, either before or after.

Well, this cannot be answered with the BPE model, but I have reused the final testing and checking logic. So I checked the rule against all the words that contain q, n or y (not single y), and the results are:

Results for q/n/y rule, restricted to words containing q, n or y (length > 1):
Number of selected words: 23577
Total word boundaries (start+end) in this subset: 47154
Correct word starts: 15341 (65.068 percent)
Correct word ends:  20939 (88.811 percent)
Overall boundary recall: 76.939 percent
Words with both boundaries correct: 13196 (55.970 percent)