A quick addendum:
If you incorporate these rules (and two others I haven't mentioned yet) and run them on the MHD corpus, you get a distribution of word lengths that comes very close to the VMS. On average, it's actually very close; it's just that the curve looks a little different... more 4/5 Glyph Token, less 1 Glyph Token. I'm currently trying to figure out why...
(But it's all still very rough, so take it with a grain of salt.)
If you incorporate these rules (and two others I haven't mentioned yet) and run them on the MHD corpus, you get a distribution of word lengths that comes very close to the VMS. On average, it's actually very close; it's just that the curve looks a little different... more 4/5 Glyph Token, less 1 Glyph Token. I'm currently trying to figure out why...
(But it's all still very rough, so take it with a grain of salt.)