Deconstructing the VMS : A positionless ledger (part 1)

[Dunsel]

I understand that.  My point is that the rules defined by your ledger are only a subset of the actual rules.  

Exactly.

Apparent actual rules include "there can be at most one gallows per word", "there can be at most three of {ch,sh,ee} per word", "there cannot be a d between two ch in the same word", "the in,iin,iiin can only occur at the end of a word", and many others.  

Does your ledger impose those restrictions? 

No, there are no rules in the ledger other than what letters can follow another letter. Just as I demonstrated with the Finnish ledger. The ledger only shows what letters are attested as following other letters in the midfix and suffix positions.

If the Scribe used your ledger as a "spell checker" while writing, could he write the words "iindy", "chdch", "okydoky", "chochochychy"?

If the ledger allows it then yes.  But, the scribe didn't just invent words out of thin air just as Finnish doesn't keep repeating umlauts forever.  The Finnish ledger shows that ÖÖ occurred in the observed text at least once.  It doesn't mean they said ÖÖÖÖÖÖÖÖÖÖ is a word.  There were other rules they used that restricted such words. If you consider copy and mutate then the ledger would only be needed to tell them that changing chody to chory would be a valid word because somewhere else, r followed an o as a midfix.

Anyway, when building the ledger you should exclude any word type that occurs only once in your sample.  Maybe exclude even any short word type that occurs only 2 or 3 times.  They are likely to be errors (by the Author, the Scribe, or the Transcriber).  

Of course.  In the OP, that ledger excludes all hapax and all gallows tokens. It's purpose was to build the ledger using core words that kept getting reused.  The 'rare' midfix transitions reflects that 2 or 3 times word count.

If you include those unique or almost-unique words, and they are errors, your ledger may end up allowing transitions that never occur in any correct word.  If you exclude them but they are correct, it probably will not make any difference, because the transitions in those words will probably occur in other words.

And you are correct. We all know that there are transcription errors. But how do you distinguish between what looks like an error and words like dydydy which apparently isn't an error?  In this initial build of the ledger the only 'safe' option is to include everything and then see where the outliers are.  If I arbitrarily set rules to exclude this or that, then I'd have objections that I'm not creating an accurate ledger. So, if I do include them, I shouldn't and if I don't include them, I should. How do I win that argument?