+- The Voynich Ninja (https://www.voynich.ninja)
+-- Forum: Voynich Research (https://www.voynich.ninja/forum-27.html)
+--- Forum: News (https://www.voynich.ninja/forum-25.html)
+--- Thread: [Blog Post] Spatial Spread And Conditional Character Entropy Reduction (/thread-3713.html)
Pages:
1
2
Spatial Spread And Conditional Character Entropy Reduction - Anton - 03-01-2022
Occasionally I got some interesting ideas before the New Year and I took time to systematize them to a certain extent in a You are not allowed to view links. Register or Login to view..
There's a reasonable statement that if something does not behave like a flow of language then perhaps it is not a flow of language. Now, there's a question of how exactly to read it. Those who explore the "hoax" hypothesis judge that Voynichese is not a flow of language. I think that maybe Voynichese is not a flow of language instead.
I have some further ideas which it would be exciting to develop, but in terms of blog posting I wanted to pause at some point, lest it takes months or years.
Comments and suggestions most welcome.
RE: Spatial Spread And Conditional Character Entropy Reduction - MichelleL11 - 04-01-2022
(03-01-2022, 05:17 PM)Anton Wrote: You are not allowed to view links. Register or Login to view.Those who explore the "hoax" hypothesis judge that Voynichese is not a flow of language. I think that maybe Voynichese is not a flow of language instead.
Hi, Anton:
Thanks for this blog post. It was quite interesting and I appreciate the time and effort it takes to put abstract ideas like this to the test.
When I try to think about what the Voynich cipher could be, I am constantly reminded of the statement by Leon Battista Alberti in his You are not allowed to view links. Register or Login to view. that the structure of "letters, syllables, and words . . . and phrases" can be scrambled "so that to an outsider the words appear like the leaves of a tree blown about by the wind that have been raked into a pile and set there."
Although this publication is a bit late for the VM, it necessarily discusses ideas that are in existence and thus could conceivably be discussing cipher approaches that form a basis within the manuscript. It certainly describes something well different than ordinary substitution ciphers. I do wish he had provided more concrete examples (the single provided example is internally contradictory and discusses moving individual letters around in a pattern that superficially seems it would not alter entropy). Or perhaps if some other examples of such scrambling could be found within other writing than the VM -- but the search goes on.
To me, this is a good example of something that could be described in this way. I look forward to the directions you take this type of investigation.
RE: Spatial Spread And Conditional Character Entropy Reduction - Koen G - 04-01-2022
Yes, it's an interesting idea that I hadn't considered before. You kind of separate glyphs within a word in two sets, like when oil and water are mixed.
One issue is, if I'm not mistaken, that this would account for certain glyphs strongly preferring the company of others, but not necessarily for positional rigidity. For example, if [i] is part of the front set, why does is so often follow [a] and rarely if ever precede it? And if it is part of the second set, why do we always get [in] and not [ni]?
You could say that some of these glyphs are markers, but then which ones would it be? If "n" is a marker, these words would simply end in "ai" and the problem would remain the same.
Alternatively, you could say that there are more than two groups, but then there is a bigger issue. The more groups you introduce, the closer the system gets to sorting alphabetically and losing all positional information. This would require more markers, and hence even fewer glyphs that hold phonetic information.
RE: Spatial Spread And Conditional Character Entropy Reduction - Anton - 04-01-2022
What I was inspired by, was the so-called "litoreia".
Generally, litoreia is a collective term for the whole family of medieval Russian cryptographic techniques. There were "simple litoreia" techniques which were basically simple substitution ciphers. And there was "complex" or the so-called "wise" litoreia, wherein numerical values of Cyrillic alphabet letters were utilized.
With the wise litoreia, the whole alphabet was divided into four groups. The first group of letters had no numerical meanings, so they were translated directly as such into the ciphertext. The second, third and fourth groups were comprised of letters associated with meanings of units, tens and hundreds, respectively. A cipher token was associated with each of these three groups - for example, a dot for group 2, a cross for group 3 and a dash for group 4. Then each letter was encoded with the respective number of respective tokens. For example, letter "A" had the numeric meaning of 1, hence it was encoded with a dot. Letter "M" had the meaning of 40, so it would be encoded with four crosses, while letter "C" meant 200 and would be encoded with two dashes. Messages thus encoded are known in Russia since at least 14th century, if I am not mistaken. The most famous of those contains the disputable statement "all priests are thieves".
Now imagine that when writing the encrypted message you place all tokens of each letter vertically. Then you split this slip of paper in two halves along a horizontal line, leaving the upper set of tokens on one slip of paper, and the lower set - on the other slip. You then store the two slips in separate places for reliablility. What is thus produced are two portions of the ciphertext, each of which cannot be deciphered by itself, and which can only be deciphered if possessed together and matched against each other (unless perhaps a good deal of brute force is applied to a single portion in one's possession).
The principle is very elegant, but it is known to me from fiction (forum participants from the ex-USSR countries will guess the novel that I mean), and I do not know if it was actually known and used in real life. If it is a real-life principle then this is an excellent example of a "spatial spread".
RE: Spatial Spread And Conditional Character Entropy Reduction - Anton - 04-01-2022
@Koen
When it comes to Voynichese, I (currently) do not think that spatial spread is the only mechanism that is employed there, but I suspect that it may be part of the play - just because of all those strange effects like multipass, p's affinity to the paragraph-starting line, the somewhat "fuzzy" ordering of first letters of lines etc.
The notable property of spatial spread is that it reduces the average word-length, and then if it is combined with something that increases it (like e.g. a verbose cipher), then the perceived word length will be normalized. If both processes decrease H2, then H2 will be exceedingly low.
It is a very good comment of yours about the intra-bigram positional rigidity, in fact there is a very simple mechanism which would produce a similar effect, and this is what I am currently looking into
RE: Spatial Spread And Conditional Character Entropy Reduction - lelle - 05-01-2022
Thanks for the interesting read, Anton. I trust you're onto something here.
Keep up the good work!!
RE: Spatial Spread And Conditional Character Entropy Reduction - MichelleL11 - 10-01-2022
For Anton or anyone else interested in the possible integration of these ideas with the Polybius cipher:
Here is a recent article, You are not allowed to view links. Register or Login to view., Arroyo et al. (2020) that includes a lot of interesting adaptions of the Polybius square. The literature review starts in 1996, so it's all recent stuff -- but some of it is relatively straightforward and definitely could have been done without computers if the idea of the Polybius Square was known.
I had hoped there would be some discussion of the history but, alas, no -- but wanted to provide this reference and all the citations in it as possible inspirations for possible adaptions.
RE: Spatial Spread And Conditional Character Entropy Reduction - pfeaster - 10-01-2022
(10-01-2022, 02:15 AM)MichelleL11 Wrote: You are not allowed to view links. Register or Login to view.Here is a recent article, You are not allowed to view links. Register or Login to view., Arroyo et al. (2020) that includes a lot of interesting adaptions of the Polybius square.
Thanks much for sharing!
Just in the interests of completeness, I'd like to throw my own You are not allowed to view links. Register or Login to view. cipher into the mix as yet another adaptation of the Polybius Square (and one I came up with specifically as an experiment in trying to mimic Voynichese -- I think it's been brought up here in the past, but before I joined the forum). It uses the Polybius Square as a kind of gameboard, and instead of enciphering characters consistently by their coordinates, it enciphers the paths taken by a token around the board to get from one point to another. By repeating the same "moves" on a similar board, a reader can reconstruct the plaintext.
This approach has definite limitations when it comes to imitating Voynichese (which are more obvious to me now than they were then), but it was a fun experiment -- and, like some other examples given in that article, it *does* provide security against frequency analysis attacks.
RE: Spatial Spread And Conditional Character Entropy Reduction - nablator - 10-01-2022
(10-01-2022, 03:50 PM)pfeaster Wrote: You are not allowed to view links. Register or Login to view.It uses the Polybius Square as a kind of gameboard, and instead of enciphering characters consistently by their coordinates, it enciphers the paths taken by a token around the board to get from one point to another. By repeating the same "moves" on a similar board, a reader can reconstruct the plaintext.
Nice. I invented a "path" cipher too. Unlike the rail fence (zigzag) cipher the path is variable and many paths are possible.
Maybe it's time to reveal how I did my experiment back in 2017. I kept Voynichese glyphs for coordinates, but changed the "pick the letter at the cartesian coordinates given by the ciphertext" rule to something that may be called antipolybius: to decipher, pick Latin letters on the board at every cell on the zigzag path defined by the ciphertext stream *except* at the vertices. The game is played on a large, mostly empty board, with one or several Latin alphabets written on the cells. With highly optimized keys (choice of coordinates and board content) it generated relatively convincing simili-Voynichese: You are not allowed to view links. Register or Login to view.
Humans are good at path-finding so the enciphering game could be fun. Lots of variations are possible: find the shortest path despite different obstructions on each page, or: avoid straying off the beaten path as much as possible, etc.
But something was very wrong: almost half of the vocabulary found in the VM could not be used. I guess I should try and find special rules for [ch], [Sh] and gallows and reduce the size of the board accordingly.
RE: Spatial Spread And Conditional Character Entropy Reduction - pfeaster - 11-01-2022
(10-01-2022, 06:28 PM)nablator Wrote: You are not allowed to view links. Register or Login to view.I kept Voynichese glyphs for coordinates, but changed the "pick the letter at the cartesian coordinates given by the ciphertext" rule to something that may be called antipolybius: to decipher, pick Latin letters on the board at every cell on the zigzag path defined by the ciphertext stream *except* at the vertices.
Neat! So if I'm understanding this correctly, if we were to think of this as a game like checkers, you'd read any Latin letters on the squares the gamepiece jumped over each time to get to its new position?
I could see a nice competitive game arising out of this, where players would draw cards with words from a deck and compete to see who could encipher their word in the fewest moves. If you design it, maybe we can set up an online tournament.
This kind of cipher is great fun to try to design in general, and there seem to be a vast number of possibilities, all of which have the added advantage of seeming rather hard to crack -- not because they're particularly sophisticated compared to better-known cipher strategies, but only because they use an unexpected logic.
But whenever I pick up a history or survey of cipher techniques, it never seems to mention anything like this. Does anyone have an explanation as to why? I assume it must be for some reason other than that the Illuminati use it and have therefore suppressed all references to it. (One explanation could of course be that I'm just not reading the right stuff.)