Word Entropy

[MichelleL11]

Conversely, doing something like ordering words' letters alphabetically would reduce character entropy and affect word entropy as well. (Although if something was done that involves shuffling letters, our efforts of decoding it are certainly doomed).

I don't see shuffling letters as a big problem, because I think that any pattern used, if used in some sort of consistency, can be figured out with enough time and computer power.  I think the Zodiac's 340 crack is an example of this -- because that's all it came down to in the end, a transposition, with a few irregularities thrown in, that took over 50 years and clever coding to figure out.

What I'm afraid of are letters gone, as this seems (especially if taken to extremes) like something that might be unrecoverable.

It also makes me nervous that this was medieval "standard practice" -- just dropping letters and substituting in inconsistent markers (granted, they were relatively consistent for a particular scribe but when you add in the "optional" spelling, etc. it can get pretty bad).  By relying on context and "tradition" as to what is abbreviated to support the sense, there is a line that is being run there.  I understand there are some highly abbreviated manuscript texts that will never be known for sure . . . 

So I do get concerned that without that grounding context (or similarity to traditional practice), just like what Koen's discusses above for his class notes, that things could get really, really problematic.

In my opinion, the Zodiac killer didn't realize how hard he was making his cipher.  This was due to ignorance.  In a similar way, I get concerned that the VM authors also did not know how difficult they were making things.  Now they have a bit more of an excuse, because they could have very well been making a cipher of this general type for the very first time (certainly not a claim the Zodiac killer can make!).  But by not understanding how difficult getting the sense back out would be -- the VM could prove, in the long run, to be also the only use of that cipher.

But let me emphasize, I'm not negative, just realistic, and definitely have more motivation to keep trying.Our "period-19"* data could be discovered tomorrow or maybe is even in this string -- just have to keep thinking.

[b]*see Zodiac 340 string for more details [/b]