+- The Voynich Ninja (https://www.voynich.ninja)
+-- Forum: Voynich Research (https://www.voynich.ninja/forum-27.html)
+--- Forum: Analysis of the text (https://www.voynich.ninja/forum-41.html)
+--- Thread: Character entropy of Voynichese (/thread-148.html)
RE: Character entropy of Voynichese - Anton - 05-01-2017
Surely, there are many observed layers of complexity in Voynichese, so mimicking was not my task. But this idea with the 2-dimensional array is something that sticks in my mind. And, if we treat Voynichese characters as "base shapes + tail modifiers" then this makes more vords consisting of even number of elements. Although there will be exceptions still.
RE: Character entropy of Voynichese - Davidsch - 20-02-2017
You are not allowed to view links. Register or Login to view.
with on position 75 and 76 the Voynich ms.
RE: Character entropy of Voynichese - Koen G - 20-02-2017
(20-02-2017, 04:48 PM)Davidsch Wrote: You are not allowed to view links. Register or Login to view.You are not allowed to view links. Register or Login to view.
with on position 75 and 76 the Voynich ms.
That's interesting to play around with. My first impulse was to sort descending by length, and obviously the VM is several orders of magnitude longer than most of them. There's only one that comes even close, the Copiale cipher which is an 18th century manuscript describing a secret society's rituals. The critical point here is that this is three centuries more recent than the VM.
The VM also has by far the lowest "multiplicity". "Lower multiplicity values indicate higher numbers of constraints on substitution ciphers, making them easier to solve." So by this logic, if the VM was a simple substitution cipher, it should be super easy to solve.
RE: Character entropy of Voynichese - Davidsch - 20-02-2017
haha...and the future will perhaps show that it indeed was easy...after all
RE: Character entropy of Voynichese - Anton - 13-12-2017
Back to the issue of "mimicking" the Voynich entropy characteristics.
The cipher that I suggested above yielded h1 = 3.42 and h2 = 2.13, while the corresponding plaintext has h1 = 4.07 and h2 = 3.10. So it lowers both h1 and h2 (bringing them lower than Voynich), but at the same time h1-h2 also lowers from 1.3 to approximately 1. While Voynich has h1-h2 = 1.44 (taking figures from Rene's site, don't remember right now what transcription they correspond to).
Also, this cipher results in considerably longer ciphertext, as compared to the plaintext. The length of the plaintext sample is 4794 characters, while the ciphertext is 8747 chars (+82%). That's because in the process of encryption each character (except spaces) is encoded with a sequence of two characters, which almost doubles the size. Actually this brings it closer to verbose ciphers, although it's not verbose in the strict sense of the word.
However, the cipher uses the considerably smaller alphabet as compared to the original: 12 characters instead of 28 ones.
I now invented another cipher, which is not verbose at all, but is just a substitution cipher (although a not-so-simple one, i.e not one-to-one).
Excluding apostrophes from the plaintext sample above (my Matlab enciphering script does not support anything other than letters and spaces), h1=4.06, h2=3.10.
Here's the corresponding ciphertext (the leading character is a space):
Code:
7!p11i6v8j5!w3!x9b2!q3c12t4!d4!t6!x5!t11!v8p11 7!p11n0!a8a13g7!p10!i4!e12!o0!z1n5j10!y6!o11!j4!v4!y3 9r3w4h7! 11!s10!j9!l3p2!d4!t6!f2!fk6!i4!w3!g1!h3!k7!g7!p10!x5!t11!v8p11n1s0!g7q2!a0!r6!d4!t6!y9!h10!s10!w4g6!o11!k2!d4!w9!p1!t7!a8a13j10t12p1!z1f5!s7! 7!q2p11i4!e12!o0!t11!w4i8!p11d11x4lz1f2!fk6!i4!w3! 7!k2!m6d4k10!k2!q10f2!fk6!i4!w3!d0!g1!y11a8a13w4g6!o11!h7! 11!t11!o11!d11b2u12s13y7q3h8t12n0!v9!u7j10t12z1i8!p11d10n5!r13d4s4m8!k6!g7q2!g7!z12!z1m11!j10t12a1! 12!n1t6!u7v9!a1g13 0!o3!u6m11!j10t12c3!i4!k6!f5!d4u12!q9!v10 5s7!y9!j1!q2o0!h0!o12!s7!d10d0!r0!x4v8!k2!q10x9b2!q3c12t4! 0!w4v9!j9f12p11x9b2o6!k6! 8!v2!g7!v4!f2!c11!x6!k6!j10t12d4u12!t12v4! 7!p11i4!e12!o0!z1x9i9! 12!l7!h0!x10j10t12f6!p1!p11f2! 5r0!p11p1!f6l7w4p1!d0!y7o11!n5j10r10w3!c11!v9! 7!k2!q10k3! 4! 5v4! 4!b2u6n0!w3!w9r13j10u6i6p2!t11k6!f6!c9a8 11!p11h7! 11!s10!j9!l3p2! 0!v8!k11!j1!s7!c1!f6k2n5!r13j10u6 7!p11g1!h3!o11!x9d4!w9!p1!t7!a8a13j10t12p1!z1i4!e12!o0!z1k2!d4!w9!p1!t7!a8a13j10t12p1!z1n5a1!m1!y6!r6!i9h3!j9!r13j10u6 7!p11n0!a8a13g7!p10!k2!j4!b11j9!l3t6!t12v8p11n5!d4!w4k2b13s1g6!w3!d11j10!v8!k6!s0!g6!o11! 7!p11t8tk6!a1g13d4!t6!t0!b11!j10t12a1!a13d1r2g7!p10!k2!j4!b11j9!l3t6!n5a1!m1!y6!r6!j10t11!o11!d11j10t12g7q2!l3t6!x9h8c12o13e3!x12v2!i9!u6 7!p11g1!r0!w3! 7!k2!q10 4!g7eo11!n5j10u6r10 5j10t12a1j9s10!j10!v8!k6!s0!g6!f2!x10g7!p10!q1v7a0!j1!n5!z1 7!p11o13 8!k8g7!p10!q1v7a0!j1!n5!z1m8!i4!f13 0!v8!h8c12o13e3!x12v2!h8c12o13e3!x12v2!d4!w4a13p11x9j10t12a1j9s10! 0!v8!f6!q2w3!k2b13s1g6!w3!n0!a8a13w4k2!i9!f3i4!d1 0!v8!j10n6!w4h7!q1u6!e2!l7j10u6n5!w4j4!b11f5!b2e0!k2!n5!l3t6!i8!p11y3 9r3w4p4!g4j1!z1j4!v4! 7!p11r2!c12!q2l7g7!p10! 7!n5j10g13p1!r9!s7! 4!b2a13w3!w9r13j10u6k2!e12!w8y4!p11j4!b11f5!b2e0!k2!n5!l3p2!g7!p10!q1v7a0!j1!n5!z1n5!d4!w4t1fs10!t11!q3e5f12!w3!d11j10!v8!k6!s0!g6!o11!b6j9!d4!w3!n5 0!h6!g13j10t12b2!c6s0!i4!d1g7!p10!r3v4!e9!a10!o11!v9!g6!w4 7!j9!k11i8!l3f12!b2! 5q3w4i5!g7!cf12g7u6t7!tk6!k11!r10w3!n5j10t12b2! 5q3b2! 5q3w4f5!y2!h10!g7!cf12d4!t6!d1s1e4t6!i4!b7c12 13i4!f13 0!t6!i4!b7c12 13i4!w3!n5b2!si4!a8f5!r6!e5f12!w3!n13y6!w3!x9a1!k6!x4k2!a1!a13f2!a8a13w4k2! 0!c3!o6!i4!b2!r4m4!a13v9!q3 0!t6!m8!i4!w3!t7!z1p1!f6!m12g7q2!g7!cf12g7!v4!f2!fk6!i4!w3!m8!i4!f13f6!m12g7!cf12g7!w9g7!p10! 7!p11q11e10 9z6b2!z2n1!v10w4k2!g7!w9w4n9! 11!g6!j9!l3t6!x9h8c12o13e3!x12v2!d4!w4 7!c12b6!x10b2s10!p11v10f13 0!o3!u6x4u10!j10r10w3!j4!v4!j1!p4!s0!w5a8a13d4!t6! 7!n5b2!z2n1!v10j10t12b2!r4m4!a13v9!q3k11!r3u9!a1p11c1! 8!d0!j10t12k11k6!z1q1v7!r13f12p11 7!j9!j10t12b2t11!d4!w4d11a1p11c12lr13j10t12i9!l7!g7!p10!f3x6c9!s0!g6!d1w4k2!k11!h0!e2!j10t12j10m5s4k2!q10x4u10!w4 11!b1p11g1!r0!w3! 7!k6!p11n5 0!b2t11!k11!n5!q9! 8!y1!q11d1w4p11!e2!d4k10!k2!q10a13p11x9j10t12g7c9!c1!r0!c3c12t7a13p11d11i9!p7! 0!sv7!r9!z6o11!b6! 7!d4!w3!w9h11!j10t12y7d4!w4 7!p11 11!b1p11b6k6!p11 7!p11f2!p11n5j10m5s4t6!x8z8!h7!w4n13x4g5p11k2! 0!i9ot7t8s0!j10t12y7d4!w4 7!p11x8r0!a13k11!k3!j10m5s4w4 7!p11f2!p11 7!p11m8!i4!f13j10r10w3! 4!a1!a13 8!d0!v4!k2!j10h0!w4n9!g6 11!p11i8!p11 7!p11q3!u8k6!w4k4w3!b6i4!j10t12b2t11!d4!w4c12lr13d4!w3!n5d4!n1!p1!x4q3j10u6f5!y2!h10!i9i4i4c1!v4! 7!j9!j10t12b2!r4m4!a13v9!q3f6!m12d4!w9!p4!i4!d1j10t12g7!g13i6v9!c3p1!d4!p2!x4v8p11u12j10t12b2t11!d4!w4t7!q3!q11k8!f12r13d4!t6!c11!v9!g7a13v3ml7! 9o12!s7!y2!z1 7!n5g7!p10!i6f12y6!k11!r3u9!d4!w9!p4!i4!d1j10t12g7!g13i6v9!f6j1n9n5p11 7!p11r3y5 12!p11t7!z1d0!x8v8!g7!t6!g6!z1d12q12!u12s10!a1!i8!e0!c9k11!m12d4!t6!b6s10!q9! 7!p11x8r0!a13j10m5s4w4 7!p11f2!p11g6!o11!c11!g7!t6!j4!v4! 7!n5g7!f12b2!z2n1!v10k11!p11b6!l 0!sc9!p11 7!j9!j10t12b2!r4m4!a13v9!q3b2w8!w4n0!w3!t7!u10! 0!s5!g7!p10! 7!p11b13ss10v10g7!g13i6v9!w4n1y7f6j1h3!z1d11g7!cf12j10r10t6! 7!p11q2!t12r0! 0!p4k3! 6!q9!k2!f6!q2w3!q2!t12v4!x5!i4!i8!l3w4 7!n5d4!w4n0!w3! 7!p11o12!y6!d4s4m8!k6!d4!w3!n5b2!b7d0!q3j10r10w3! 7!p11i4!b7c12 13i4!w3!f5!y2!h10!h8t8d1w4i6p2!e12!e4q3w4a13k11!r10w3!m8!i4!f13 0!y7h8q2 8!g5p11g6!o11!b6j9! 0!y7g7q2!b2!a4n1!x12c3!w5a8n13k2!j10t12b2l7j7!s10r13b2!r4m4!a13v9!q3d4!w3!n5d4!n1!q2 8!g5p11d11c3!l7!j10t12g7!g13i6v9!g7!p10!y6! 5t6! 8!n1!r6!a1q12!k10y6!j10t12y7d4!w4n0!i9!y4!q9!q3!u8k6!g7!t6! 7!p11f2!p11u12j10t12y7k11!k6!p11t1e2!g7c9!c1!v4!k2!g7! 7!k6!k11!p1!k7!d4!p10! 7!p11f2!p11y2!w4f2!fk6!i4!w3! 7!n5g7!g13i6v9!k11!r3u9!d4k10!p11c1!i4!h8q2 8!g5p11k9!w3! 7!j9!f6!p11!p2! 0!n0! 7!k6!b2!r4m4!a13v9!q3g7s4d4!t6!n13h1 5t6!i4!b7c12 13i4!w3! 7!y7p11v2!x8w4x9b2! 5q3w4t7!z1c1!a1!a13 8!d0!y7o11!g1!r0!w3!a13p11o12!t6!k2d3k2!p11m8!i4!f13j10r10w3!w9 5v4!v7!l9v4!t1e2!b2! 5q3w4i8!p11o12!lr13d4!n1!q2 8!g5p11j9 0!o3p11!n10!i9! 6!s12!s0!r13j10t12g7!g13i6v9!g7!p10!y6! 5t6!n5 0!t6!n9!g6 11!p11x9i9!y4!q9!m8!i4!w3!y6!i6v8!j10t12y7f6!m12a1p11m8!i4!f13k11!h0!e2! 0!q5!m12w4v7!l9v4!t1e2!b2! 5q3w4i8!p11o12!lr13a1!k6!x4k2! 0!t6!n9!g6 11!p11x9i9!y4!q9!e9!c9o11!c1!j10t12b2! 5q3b2l7j7!s10r13 0!w4r3y5 12!p11x9 0!s5!g7c9!c1!v4!k2!j10t12i9g6!s12!g7!p10!a13p11d11i9!p7!j10m5s4k2!q10 7!p11f2!p11f8!x3f5!y2!h10!c3!l7! 0!g7c9!c1!v4!l6 12!g7!w9j10u6 8!y1!c11!j10r10w3!m8!i4!w3!n5e5f12!w3!d1s1e4t6!d11g7!cf12j10h0!x6!j10t12y7f6!m12a1p11m8!i4!f13k11!h0!e2!f6!m12g7!v4!t7!z1n0!w3!v7!l9v4!t1e2!b2! 5q3w4i8!p11o12!lr13i9!d11e2!j9a8o12b6s10!q9!l7y6!q3r9!lz1v9!g6!w4 7!j9!j10t12z1i8!p11k4 5b11!r13a1z1e2!g6!d1 0!t6!n9!g6 11!p11n5j10t12g7!g13i6v9!g7!p10!n13v6s1s10!c9i8!g7c9!c1!v4!n9!i4 11!r9l5!j9!r9!g7c9!c1!v4! 7!y7p11n5!d4!w4e9!s1q9!n0!a8x10j10r10w3! 7!p11q1v7v10t7x9k11!r10w3!n5a1!r10v8p11 5r0!f12!g7q12!l7 8!v2! 0!v8!d4s4d4!w3!n5i9h3!j9!r13j10u6o12!f12!p11g6!o11!e0!s13k8!d4!w4g6!d4!n1!p1!x4q3d4!sz5!d4!t6!y9!j1!q2o0!p8!c3p1!g7!f12h8f12b13y6!w4 4!a1!g6!a1!a13 8!d0!v4!e2!g6!d1 0!w4c11!v9! 7!k2!q10n0!w3!n9!b1b9!z1q1r13s10!x4g5p11l6 12!j10t12b2!r4m4!a13v9!q3 0!w4 4!e5!n0!y2n5!d4!t6!r3v4!n9!g6 11!p11a13p11o12!nq2!b2!z2k8!r6!h8y7f2!k8!j10t12g7!g13i6v9!g7!p10!p11!e2!j10m5s4g7!w9a1!g6!g7!z12!z1n7l7w4p1!f6!f5p11q1x9!n0!y6!w4b6! 7!f6!p1!p11p1!f6l7w4i6w9!t11i4!d1d4!t6! 7!p11o12!y6!k11!n5!q9! 7!p11f2!p11 7!k6!p11n5j10!p4k2j9!h3!z1n0!j10!v8k6!x4k2!v2!d4!f13f6!q2!l3t6!n5c3!r3!k6!w9o11!m11!f6l11w4x9f6!q12!r10y11k8g7!z12!z1n5!w4o5!s0!i9p11!lz1r10x6!j10u6e12!b1p11g6!o11! 7!p11k2!n5!r9!s7!i6v8!n5!l3p2!g7!p10! 7!p11 7!c12y2f2!fk6!b2!b1q9!a8v9!j10t12y7f6!m12a1p11o12!y6!w4 7!r3g0!k11!t12y7j10t12y7d4!w4k2!t12y7q3j10!v8k6!x4k2!v2!a1g13d4!t6!g2 7!k6!a1!j9p11j4!v4!g6!b2!h11!k6!u7s7!v7y6!t2k6!b2!b1q9!w9y2r3y5 12!p11n5g7q2!b2l7!k6!x11!y11s0!s10!g7s4h8c12o13e3!x12v2!d4!t6!n5!w4y7f5!k11!p1!u9!i9!i4!y6!g7!p10! 7!p11e9!x6!d4!w4l7y6!q3r9!lz1 7!p11v9!j9f12p11x9a1!k6!x4k2!v2!g7!p10!g6!b2! 5q3j10u6v7!l9v4! 7!p11q1v7a0!j1!n5!z1x9 0!a1!k6!x4k2!b2! 5q3d4!w4d10n5!z1 7!p11q1v7a0!j1!n5!z1x9 0!t6!k2b13ss10v10b2! 5q3d4!w4b3c12j10t12h8c12o13e3!x12v2!g7!p10!n13s0!v7!r10s0!s10!b2! 5q3d4!w4c11!v9!b6k6!p11c1!u1!et6!b3c12 0!v8!j10!y11v2!j10t12c3!y7j9!k6!j10t12h8c12o13e3!x12v2!j10t12f6!p1!p11x12u10!r6!d4!w4 7!p11m8!i4!w3!d11g7!cf12This one uses a larger alphabet than the original (26 letters +1 space + 10 digits + exclamation mark = 38 characters instead of 27 ones). However the length of the ciphertext is just 6675 versus 4790, i.e. only 39% more than plain text.
The cipher raises h1 to 4.49, but lowers h2 to 2.87. Hence h1-h2 increases to 1.62 which is larger than Voynich.
Lets introduce an additional space after each "!" This would make the text more "readable". Although it increases its length to 7492 chars, that's still only plus 56% to the plain text.
Code:
7! p11i6v8j5! w3! x9b2! q3c12t4! d4! t6! x5! t11! v8p11 7! p11n0! a8a13g7! p10! i4! e12! o0! z1n5j10! y6! o11! j4! v4! y3 9r3w4h7! 11! s10! j9! l3p2! d4! t6! f2! fk6! i4! w3! g1! h3! k7! g7! p10! x5! t11! v8p11n1s0! g7q2! a0! r6! d4! t6! y9! h10! s10! w4g6! o11! k2! d4! w9! p1! t7! a8a13j10t12p1! z1f5! s7! 7! q2p11i4! e12! o0! t11! w4i8! p11d11x4lz1f2! fk6! i4! w3! 7! k2! m6d4k10! k2! q10f2! fk6! i4! w3! d0! g1! y11a8a13w4g6! o11! h7! 11! t11! o11! d11b2u12s13y7q3h8t12n0! v9! u7j10t12z1i8! p11d10n5! r13d4s4m8! k6! g7q2! g7! z12! z1m11! j10t12a1! 12! n1t6! u7v9! a1g13 0! o3! u6m11! j10t12c3! i4! k6! f5! d4u12! q9! v10 5s7! y9! j1! q2o0! h0! o12! s7! d10d0! r0! x4v8! k2! q10x9b2! q3c12t4! 0! w4v9! j9f12p11x9b2o6! k6! 8! v2! g7! v4! f2! c11! x6! k6! j10t12d4u12! t12v4! 7! p11i4! e12! o0! z1x9i9! 12! l7! h0! x10j10t12f6! p1! p11f2! 5r0! p11p1! f6l7w4p1! d0! y7o11! n5j10r10w3! c11! v9! 7! k2! q10k3! 4! 5v4! 4! b2u6n0! w3! w9r13j10u6i6p2! t11k6! f6! c9a8 11! p11h7! 11! s10! j9! l3p2! 0! v8! k11! j1! s7! c1! f6k2n5! r13j10u6 7! p11g1! h3! o11! x9d4! w9! p1! t7! a8a13j10t12p1! z1i4! e12! o0! z1k2! d4! w9! p1! t7! a8a13j10t12p1! z1n5a1! m1! y6! r6! i9h3! j9! r13j10u6 7! p11n0! a8a13g7! p10! k2! j4! b11j9! l3t6! t12v8p11n5! d4! w4k2b13s1g6! w3! d11j10! v8! k6! s0! g6! o11! 7! p11t8tk6! a1g13d4! t6! t0! b11! j10t12a1! a13d1r2g7! p10! k2! j4! b11j9! l3t6! n5a1! m1! y6! r6! j10t11! o11! d11j10t12g7q2! l3t6! x9h8c12o13e3! x12v2! i9! u6 7! p11g1! r0! w3! 7! k2! q10 4! g7eo11! n5j10u6r10 5j10t12a1j9s10! j10! v8! k6! s0! g6! f2! x10g7! p10! q1v7a0! j1! n5! z1 7! p11o13 8! k8g7! p10! q1v7a0! j1! n5! z1m8! i4! f13 0! v8! h8c12o13e3! x12v2! h8c12o13e3! x12v2! d4! w4a13p11x9j10t12a1j9s10! 0! v8! f6! q2w3! k2b13s1g6! w3! n0! a8a13w4k2! i9! f3i4! d1 0! v8! j10n6! w4h7! q1u6! e2! l7j10u6n5! w4j4! b11f5! b2e0! k2! n5! l3t6! i8! p11y3 9r3w4p4! g4j1! z1j4! v4! 7! p11r2! c12! q2l7g7! p10! 7! n5j10g13p1! r9! s7! 4! b2a13w3! w9r13j10u6k2! e12! w8y4! p11j4! b11f5! b2e0! k2! n5! l3p2! g7! p10! q1v7a0! j1! n5! z1n5! d4! w4t1fs10! t11! q3e5f12! w3! d11j10! v8! k6! s0! g6! o11! b6j9! d4! w3! n5 0! h6! g13j10t12b2! c6s0! i4! d1g7! p10! r3v4! e9! a10! o11! v9! g6! w4 7! j9! k11i8! l3f12! b2! 5q3w4i5! g7! cf12g7u6t7! tk6! k11! r10w3! n5j10t12b2! 5q3b2! 5q3w4f5! y2! h10! g7! cf12d4! t6! d1s1e4t6! i4! b7c12 13i4! f13 0! t6! i4! b7c12 13i4! w3! n5b2! si4! a8f5! r6! e5f12! w3! n13y6! w3! x9a1! k6! x4k2! a1! a13f2! a8a13w4k2! 0! c3! o6! i4! b2! r4m4! a13v9! q3 0! t6! m8! i4! w3! t7! z1p1! f6! m12g7q2! g7! cf12g7! v4! f2! fk6! i4! w3! m8! i4! f13f6! m12g7! cf12g7! w9g7! p10! 7! p11q11e10 9z6b2! z2n1! v10w4k2! g7! w9w4n9! 11! g6! j9! l3t6! x9h8c12o13e3! x12v2! d4! w4 7! c12b6! x10b2s10! p11v10f13 0! o3! u6x4u10! j10r10w3! j4! v4! j1! p4! s0! w5a8a13d4! t6! 7! n5b2! z2n1! v10j10t12b2! r4m4! a13v9! q3k11! r3u9! a1p11c1! 8! d0! j10t12k11k6! z1q1v7! r13f12p11 7! j9! j10t12b2t11! d4! w4d11a1p11c12lr13j10t12i9! l7! g7! p10! f3x6c9! s0! g6! d1w4k2! k11! h0! e2! j10t12j10m5s4k2! q10x4u10! w4 11! b1p11g1! r0! w3! 7! k6! p11n5 0! b2t11! k11! n5! q9! 8! y1! q11d1w4p11! e2! d4k10! k2! q10a13p11x9j10t12g7c9! c1! r0! c3c12t7a13p11d11i9! p7! 0! sv7! r9! z6o11! b6! 7! d4! w3! w9h11! j10t12y7d4! w4 7! p11 11! b1p11b6k6! p11 7! p11f2! p11n5j10m5s4t6! x8z8! h7! w4n13x4g5p11k2! 0! i9ot7t8s0! j10t12y7d4! w4 7! p11x8r0! a13k11! k3! j10m5s4w4 7! p11f2! p11 7! p11m8! i4! f13j10r10w3! 4! a1! a13 8! d0! v4! k2! j10h0! w4n9! g6 11! p11i8! p11 7! p11q3! u8k6! w4k4w3! b6i4! j10t12b2t11! d4! w4c12lr13d4! w3! n5d4! n1! p1! x4q3j10u6f5! y2! h10! i9i4i4c1! v4! 7! j9! j10t12b2! r4m4! a13v9! q3f6! m12d4! w9! p4! i4! d1j10t12g7! g13i6v9! c3p1! d4! p2! x4v8p11u12j10t12b2t11! d4! w4t7! q3! q11k8! f12r13d4! t6! c11! v9! g7a13v3ml7! 9o12! s7! y2! z1 7! n5g7! p10! i6f12y6! k11! r3u9! d4! w9! p4! i4! d1j10t12g7! g13i6v9! f6j1n9n5p11 7! p11r3y5 12! p11t7! z1d0! x8v8! g7! t6! g6! z1d12q12! u12s10! a1! i8! e0! c9k11! m12d4! t6! b6s10! q9! 7! p11x8r0! a13j10m5s4w4 7! p11f2! p11g6! o11! c11! g7! t6! j4! v4! 7! n5g7! f12b2! z2n1! v10k11! p11b6! l 0! sc9! p11 7! j9! j10t12b2! r4m4! a13v9! q3b2w8! w4n0! w3! t7! u10! 0! s5! g7! p10! 7! p11b13ss10v10g7! g13i6v9! w4n1y7f6j1h3! z1d11g7! cf12j10r10t6! 7! p11q2! t12r0! 0! p4k3! 6! q9! k2! f6! q2w3! q2! t12v4! x5! i4! i8! l3w4 7! n5d4! w4n0! w3! 7! p11o12! y6! d4s4m8! k6! d4! w3! n5b2! b7d0! q3j10r10w3! 7! p11i4! b7c12 13i4! w3! f5! y2! h10! h8t8d1w4i6p2! e12! e4q3w4a13k11! r10w3! m8! i4! f13 0! y7h8q2 8! g5p11g6! o11! b6j9! 0! y7g7q2! b2! a4n1! x12c3! w5a8n13k2! j10t12b2l7j7! s10r13b2! r4m4! a13v9! q3d4! w3! n5d4! n1! q2 8! g5p11d11c3! l7! j10t12g7! g13i6v9! g7! p10! y6! 5t6! 8! n1! r6! a1q12! k10y6! j10t12y7d4! w4n0! i9! y4! q9! q3! u8k6! g7! t6! 7! p11f2! p11u12j10t12y7k11! k6! p11t1e2! g7c9! c1! v4! k2! g7! 7! k6! k11! p1! k7! d4! p10! 7! p11f2! p11y2! w4f2! fk6! i4! w3! 7! n5g7! g13i6v9! k11! r3u9! d4k10! p11c1! i4! h8q2 8! g5p11k9! w3! 7! j9! f6! p11! p2! 0! n0! 7! k6! b2! r4m4! a13v9! q3g7s4d4! t6! n13h1 5t6! i4! b7c12 13i4! w3! 7! y7p11v2! x8w4x9b2! 5q3w4t7! z1c1! a1! a13 8! d0! y7o11! g1! r0! w3! a13p11o12! t6! k2d3k2! p11m8! i4! f13j10r10w3! w9 5v4! v7! l9v4! t1e2! b2! 5q3w4i8! p11o12! lr13d4! n1! q2 8! g5p11j9 0! o3p11! n10! i9! 6! s12! s0! r13j10t12g7! g13i6v9! g7! p10! y6! 5t6! n5 0! t6! n9! g6 11! p11x9i9! y4! q9! m8! i4! w3! y6! i6v8! j10t12y7f6! m12a1p11m8! i4! f13k11! h0! e2! 0! q5! m12w4v7! l9v4! t1e2! b2! 5q3w4i8! p11o12! lr13a1! k6! x4k2! 0! t6! n9! g6 11! p11x9i9! y4! q9! e9! c9o11! c1! j10t12b2! 5q3b2l7j7! s10r13 0! w4r3y5 12! p11x9 0! s5! g7c9! c1! v4! k2! j10t12i9g6! s12! g7! p10! a13p11d11i9! p7! j10m5s4k2! q10 7! p11f2! p11f8! x3f5! y2! h10! c3! l7! 0! g7c9! c1! v4! l6 12! g7! w9j10u6 8! y1! c11! j10r10w3! m8! i4! w3! n5e5f12! w3! d1s1e4t6! d11g7! cf12j10h0! x6! j10t12y7f6! m12a1p11m8! i4! f13k11! h0! e2! f6! m12g7! v4! t7! z1n0! w3! v7! l9v4! t1e2! b2! 5q3w4i8! p11o12! lr13i9! d11e2! j9a8o12b6s10! q9! l7y6! q3r9! lz1v9! g6! w4 7! j9! j10t12z1i8! p11k4 5b11! r13a1z1e2! g6! d1 0! t6! n9! g6 11! p11n5j10t12g7! g13i6v9! g7! p10! n13v6s1s10! c9i8! g7c9! c1! v4! n9! i4 11! r9l5! j9! r9! g7c9! c1! v4! 7! y7p11n5! d4! w4e9! s1q9! n0! a8x10j10r10w3! 7! p11q1v7v10t7x9k11! r10w3! n5a1! r10v8p11 5r0! f12! g7q12! l7 8! v2! 0! v8! d4s4d4! w3! n5i9h3! j9! r13j10u6o12! f12! p11g6! o11! e0! s13k8! d4! w4g6! d4! n1! p1! x4q3d4! sz5! d4! t6! y9! j1! q2o0! p8! c3p1! g7! f12h8f12b13y6! w4 4! a1! g6! a1! a13 8! d0! v4! e2! g6! d1 0! w4c11! v9! 7! k2! q10n0! w3! n9! b1b9! z1q1r13s10! x4g5p11l6 12! j10t12b2! r4m4! a13v9! q3 0! w4 4! e5! n0! y2n5! d4! t6! r3v4! n9! g6 11! p11a13p11o12! nq2! b2! z2k8! r6! h8y7f2! k8! j10t12g7! g13i6v9! g7! p10! p11! e2! j10m5s4g7! w9a1! g6! g7! z12! z1n7l7w4p1! f6! f5p11q1x9! n0! y6! w4b6! 7! f6! p1! p11p1! f6l7w4i6w9! t11i4! d1d4! t6! 7! p11o12! y6! k11! n5! q9! 7! p11f2! p11 7! k6! p11n5j10! p4k2j9! h3! z1n0! j10! v8k6! x4k2! v2! d4! f13f6! q2! l3t6! n5c3! r3! k6! w9o11! m11! f6l11w4x9f6! q12! r10y11k8g7! z12! z1n5! w4o5! s0! i9p11! lz1r10x6! j10u6e12! b1p11g6! o11! 7! p11k2! n5! r9! s7! i6v8! n5! l3p2! g7! p10! 7! p11 7! c12y2f2! fk6! b2! b1q9! a8v9! j10t12y7f6! m12a1p11o12! y6! w4 7! r3g0! k11! t12y7j10t12y7d4! w4k2! t12y7q3j10! v8k6! x4k2! v2! a1g13d4! t6! g2 7! k6! a1! j9p11j4! v4! g6! b2! h11! k6! u7s7! v7y6! t2k6! b2! b1q9! w9y2r3y5 12! p11n5g7q2! b2l7! k6! x11! y11s0! s10! g7s4h8c12o13e3! x12v2! d4! t6! n5! w4y7f5! k11! p1! u9! i9! i4! y6! g7! p10! 7! p11e9! x6! d4! w4l7y6! q3r9! lz1 7! p11v9! j9f12p11x9a1! k6! x4k2! v2! g7! p10! g6! b2! 5q3j10u6v7! l9v4! 7! p11q1v7a0! j1! n5! z1x9 0! a1! k6! x4k2! b2! 5q3d4! w4d10n5! z1 7! p11q1v7a0! j1! n5! z1x9 0! t6! k2b13ss10v10b2! 5q3d4! w4b3c12j10t12h8c12o13e3! x12v2! g7! p10! n13s0! v7! r10s0! s10! b2! 5q3d4! w4c11! v9! b6k6! p11c1! u1! et6! b3c12 0! v8! j10! y11v2! j10t12c3! y7j9! k6! j10t12h8c12o13e3! x12v2! j10t12f6! p1! p11x12u10! r6! d4! w4 7! p11m8! i4! w3! d11g7! cf12Interestingly, this lowers h1 to 4.31 and h2 further down to 2.51, with h1-h2 becoming terrific 1.8. Beside that, the text now reveals some regularity in its morphology, like the exclamation mark is seen only in the end of the "words", or the "words" usually don't begin with a digit. Letters and digits typically intertwine, representing what might be taken for vowels and consonants.
RE: Character entropy of Voynichese - Anton - 13-12-2017
The cipher works as follows. The alphabet is comprised of 26 letters and a space (considered as the 27th letter).
Instead of enciphering single characters, it enciphers bigrams. For this purpose, it performs kinda coordinate transform, where a bigram is encoded by the combination of the coordinate of the centre letter and the radius. The centre letter is the letter situated (in the alphabet) in between the two letters of the bigram. The radius is the distance, in the alphabet, from the centre to the letters of the bigram.
For example, consider the bigram AE. E is 4 letters far from A, so the centre letter would be C. C is two letters far from A or E, so the radius is 2. Thus AE is enciphered into C2.
Consider another example, like AF. F is 5 letters far from A, so the centre would be in between C and D. But we must point it to a letter. By convention, in the case when the distance is odd, the centre is pointed to the "upper" letter, C in this case, and the radius (2,5) is rounded downwards, which would be 2 in this case. You see, AF yields C2, like in the first example. To distinguish between AE and AF when deciphering C2, we need to use a marker which would tell us whether the distance was even or odd when enciphering the bigram. I use the exclamation mark in the cases when the distance is odd. Hence AE would be enciphered as C2, and AF as C2!.
For cases like YE, to calculate the distance just count towards space (the 27th letter), and then continue from A. The distance between Y and E is thus 7.
Bigrams cnsisting of two similar letters are enciphered with this single letter, like SS -> S.
You see, the cipher maps bigrams to bi-, tri- or, in rare cases, 4-grams.
Now let's try to cope with high h1. Since absolute values of entropy are alphabet dependent, we could suppose that the rise of h1 was, at least partly, due to the increased alphabet of the ciphertext. So let's reduce it by just encoding digits into letters, like this:
0 = A, 1 = B, 2 = C, etc.
I guess this will make the text undecipherable, because the sync will be lost at the "single letter bigram" positions in the plain text, but anyway for a quick check we'll get
Code:
h!pligvijf!wd!xjbc!qdcmte!de!tg!xf!tl!vipl h!plna!aiangh!pk!ie!em!oa!zbnfjk!yg!ol!je!ve!yd jrdwehh! l!sk!jj!ldpc!de!tg!fc!fkg!ie!wd!gb!hd!kh!gh!pk!xf!tl!viplnbsa!ghqc!aa!rg!de!tg!yj!hk!sk!wegg!ol!kc!de!wj!pb!th!aianjktmpb!zbff!sh! h!qcplie!em!oa!tl!weii!pldlxelzbfc!fkg!ie!wd! h!kc!mgdekk!kc!qkfc!fkg!ie!wd!da!gb!ylaianwegg!ol!hh! l!tl!ol!dlbcumsnyhqdhitmna!vj!uhjktmzbii!pldknf!rndesemi!kg!ghqc!gh!zm!zbml!jktmab! m!nbtg!uhvj!abgn a!od!ugml!jktmcd!ie!kg!ff!deum!qj!vk fsh!yj!jb!qcoa!ha!om!sh!dkda!ra!xevi!kc!qkxjbc!qdcmte! a!wevj!jjfmplxjbcog!kg! i!vc!gh!ve!fc!cl!xg!kg!jktmdeum!tmve! h!plie!em!oa!zbxjij! m!lh!ha!xkjktmfg!pb!plfc! fra!plpb!fglhwepb!da!yhol!nfjkrkwd!cl!vj! h!kc!qkkd! e! fve! e!bcugna!wd!wjrnjkugigpc!tlkg!fg!cjai l!plhh! l!sk!jj!ldpc! a!vi!kl!jb!sh!cb!fgkcnf!rnjkug h!plgb!hd!ol!xjde!wj!pb!th!aianjktmpb!zbie!em!oa!zbkc!de!wj!pb!th!aianjktmpb!zbnfab!mb!yg!rg!ijhd!jj!rnjkug h!plna!aiangh!pk!kc!je!bljj!ldtg!tmviplnf!de!wekcbnsbgg!wd!dljk!vi!kg!sa!gg!ol! h!pltitkg!abgnde!tg!ta!bl!jktmab!andbrcgh!pk!kc!je!bljj!ldtg!nfab!mb!yg!rg!jktl!ol!dljktmghqc!ldtg!xjhicmoned!xmvc!ij!ug h!plgb!ra!wd! h!kc!qk e!gheol!nfjkugrk fjktmabjjsk!jk!vi!kg!sa!gg!fc!xkgh!pk!qbvhaa!jb!nf!zb h!plon i!kigh!pk!qbvhaa!jb!nf!zbmi!ie!fn a!vi!hicmoned!xmvc!hicmoned!xmvc!de!weanplxjjktmabjjsk! a!vi!fg!qcwd!kcbnsbgg!wd!na!aianwekc!ij!fdie!db a!vi!jkng!wehh!qbug!ec!lhjkugnf!weje!blff!bcea!kc!nf!ldtg!ii!plyd jrdwepe!gejb!zbje!ve! h!plrc!cm!qclhgh!pk! h!nfjkgnpb!rj!sh! e!bcanwd!wjrnjkugkc!em!wiye!plje!blff!bcea!kc!nf!ldpc!gh!pk!qbvhaa!jb!nf!zbnf!de!wetbfsk!tl!qdeffm!wd!dljk!vi!kg!sa!gg!ol!bgjj!de!wd!nf a!hg!gnjktmbc!cgsa!ie!dbgh!pk!rdve!ej!ak!ol!vj!gg!we h!jj!klii!ldfm!bc! fqdweif!gh!cfmghugth!tkg!kl!rkwd!nfjktmbc! fqdbc! fqdweff!yc!hk!gh!cfmde!tg!dbsbeetg!ie!bhcm nie!fn a!tg!ie!bhcm nie!wd!nfbc!sie!aiff!rg!effm!wd!nnyg!wd!xjab!kg!xekc!ab!anfc!aianwekc! a!cd!og!ie!bc!reme!anvj!qd a!tg!mi!ie!wd!th!zbpb!fg!mmghqc!gh!cfmgh!ve!fc!fkg!ie!wd!mi!ie!fnfg!mmgh!cfmgh!wjgh!pk! h!plqlek jzgbc!zcnb!vkwekc!gh!wjwenj! l!gg!jj!ldtg!xjhicmoned!xmvc!de!we h!cmbg!xkbcsk!plvkfn a!od!ugxeuk!jkrkwd!je!ve!jb!pe!sa!wfaiande!tg! h!nfbc!zcnb!vkjktmbc!reme!anvj!qdkl!rduj!abplcb! i!da!jktmklkg!zbqbvh!rnfmpl h!jj!jktmbctl!de!wedlabplcmlrnjktmij!lh!gh!pk!fdxgcj!sa!gg!dbwekc!kl!ha!ec!jktmjkmfsekc!qkxeuk!we l!bbplgb!ra!wd! h!kg!plnf a!bctl!kl!nf!qj! i!yb!qldbwepl!ec!dekk!kc!qkanplxjjktmghcj!cb!ra!cdcmthanpldlij!ph! a!svh!rj!zgol!bg! h!de!wd!wjhl!jktmyhde!we h!pl l!bbplbgkg!pl h!plfc!plnfjkmfsetg!xizi!hh!wennxegfplkc! a!ijothtisa!jktmyhde!we h!plxira!ankl!kd!jkmfsewe h!plfc!pl h!plmi!ie!fnjkrkwd! e!ab!an i!da!ve!kc!jkha!wenj!gg l!plii!pl h!plqd!uikg!wekewd!bgie!jktmbctl!de!wecmlrnde!wd!nfde!nb!pb!xeqdjkugff!yc!hk!ijieiecb!ve! h!jj!jktmbc!reme!anvj!qdfg!mmde!wj!pe!ie!dbjktmgh!gnigvj!cdpb!de!pc!xeviplumjktmbctl!de!weth!qd!qlki!fmrnde!tg!cl!vj!ghanvdmlh! jom!sh!yc!zb h!nfgh!pk!igfmyg!kl!rduj!de!wj!pe!ie!dbjktmgh!gnigvj!fgjbnjnfpl h!plrdyf m!plth!zbda!xivi!gh!tg!gg!zbdmqm!umsk!ab!ii!ea!cjkl!mmde!tg!bgsk!qj! h!plxira!anjkmfsewe h!plfc!plgg!ol!cl!gh!tg!je!ve! h!nfgh!fmbc!zcnb!vkkl!plbg!l a!scj!pl h!jj!jktmbc!reme!anvj!qdbcwi!wena!wd!th!uk! a!sf!gh!pk! h!plbnsskvkgh!gnigvj!wenbyhfgjbhd!zbdlgh!cfmjkrktg! h!plqc!tmra! a!pekd! g!qj!kc!fg!qcwd!qc!tmve!xf!ie!ii!ldwe h!nfde!wena!wd! h!plom!yg!desemi!kg!de!wd!nfbc!bhda!qdjkrkwd! h!plie!bhcm nie!wd!ff!yc!hk!hitidbweigpc!em!eeqdweankl!rkwd!mi!ie!fn a!yhhiqc i!gfplgg!ol!bgjj! a!yhghqc!bc!aenb!xmcd!wfainnkc!jktmbclhjh!skrnbc!reme!anvj!qdde!wd!nfde!nb!qc i!gfpldlcd!lh!jktmgh!gnigvj!gh!pk!yg! ftg! i!nb!rg!abqm!kkyg!jktmyhde!wena!ij!ye!qj!qd!uikg!gh!tg! h!plfc!plumjktmyhkl!kg!pltbec!ghcj!cb!ve!kc!gh! h!kg!kl!pb!kh!de!pk! h!plfc!plyc!wefc!fkg!ie!wd! h!nfgh!gnigvj!kl!rduj!dekk!plcb!ie!hiqc i!gfplkj!wd! h!jj!fg!pl!pc! a!na! h!kg!bc!reme!anvj!qdghsede!tg!nnhb ftg!ie!bhcm nie!wd! h!yhplvc!xiwexjbc! fqdweth!zbcb!ab!an i!da!yhol!gb!ra!wd!anplom!tg!kcddkc!plmi!ie!fnjkrkwd!wj fve!vh!ljve!tbec!bc! fqdweii!plom!lrnde!nb!qc i!gfpljj a!odpl!nk!ij! g!sm!sa!rnjktmgh!gnigvj!gh!pk!yg! ftg!nf a!tg!nj!gg l!plxjij!ye!qj!mi!ie!wd!yg!igvi!jktmyhfg!mmabplmi!ie!fnkl!ha!ec! a!qf!mmwevh!ljve!tbec!bc! fqdweii!plom!lrnab!kg!xekc! a!tg!nj!gg l!plxjij!ye!qj!ej!cjol!cb!jktmbc! fqdbclhjh!skrn a!werdyf m!plxj a!sf!ghcj!cb!ve!kc!jktmijgg!sm!gh!pk!anpldlij!ph!jkmfsekc!qk h!plfc!plfi!xdff!yc!hk!cd!lh! a!ghcj!cb!ve!lg m!gh!wjjkug i!yb!cl!jkrkwd!mi!ie!wd!nfeffm!wd!dbsbeetg!dlgh!cfmjkha!xg!jktmyhfg!mmabplmi!ie!fnkl!ha!ec!fg!mmgh!ve!th!zbna!wd!vh!ljve!tbec!bc! fqdweii!plom!lrnij!dlec!jjaiombgsk!qj!lhyg!qdrj!lzbvj!gg!we h!jj!jktmzbii!plke fbl!rnabzbec!gg!db a!tg!nj!gg l!plnfjktmgh!gnigvj!gh!pk!nnvgsbsk!cjii!ghcj!cb!ve!nj!ie l!rjlf!jj!rj!ghcj!cb!ve! h!yhplnf!de!weej!sbqj!na!aixkjkrkwd! h!plqbvhvkthxjkl!rkwd!nfab!rkvipl fra!fm!ghqm!lh i!vc! a!vi!desede!wd!nfijhd!jj!rnjkugom!fm!plgg!ol!ea!snki!de!wegg!de!nb!pb!xeqdde!szf!de!tg!yj!jb!qcoa!pi!cdpb!gh!fmhifmbnyg!we e!ab!gg!ab!an i!da!ve!ec!gg!db a!wecl!vj! h!kc!qkna!wd!nj!bbbj!zbqbrnsk!xegfpllg m!jktmbc!reme!anvj!qd a!we e!ef!na!ycnf!de!tg!rdve!nj!gg l!planplom!nqc!bc!zcki!rg!hiyhfc!ki!jktmgh!gnigvj!gh!pk!pl!ec!jkmfsegh!wjab!gg!gh!zm!zbnhlhwepb!fg!ffplqbxj!na!yg!webg! h!fg!pb!plpb!fglhweigwj!tlie!dbde!tg! h!plom!yg!kl!nf!qj! h!plfc!pl h!kg!plnfjk!pekcjj!hd!zbna!jk!vikg!xekc!vc!de!fnfg!qc!ldtg!nfcd!rd!kg!wjol!ml!fgllwexjfg!qm!rkylkigh!zm!zbnf!weof!sa!ijpl!lzbrkxg!jkugem!bbplgg!ol! h!plkc!nf!rj!sh!igvi!nf!ldpc!gh!pk! h!pl h!cmycfc!fkg!bc!bbqj!aivj!jktmyhfg!mmabplom!yg!we h!rdga!kl!tmyhjktmyhde!wekc!tmyhqdjk!vikg!xekc!vc!abgnde!tg!gc h!kg!ab!jjplje!ve!gg!bc!hl!kg!uhsh!vhyg!tckg!bc!bbqj!wjycrdyf m!plnfghqc!bclh!kg!xl!ylsa!sk!ghsehicmoned!xmvc!de!tg!nf!weyhff!kl!pb!uj!ij!ie!yg!gh!pk! h!plej!xg!de!welhyg!qdrj!lzb h!plvj!jjfmplxjab!kg!xekc!vc!gh!pk!gg!bc! fqdjkugvh!ljve! h!plqbvhaa!jb!nf!zbxj a!ab!kg!xekc!bc! fqdde!wedknf!zb h!plqbvhaa!jb!nf!zbxj a!tg!kcbnsskvkbc! fqdde!webdcmjktmhicmoned!xmvc!gh!pk!nnsa!vh!rksa!sk!bc! fqdde!wecl!vj!bgkg!plcb!ub!etg!bdcm a!vi!jk!ylvc!jktmcd!yhjj!kg!jktmhicmoned!xmvc!jktmfg!pb!plxmuk!rg!de!we h!plmi!ie!wd!dlgh!cfmThis also allows us to reduce the length, which is now 6020, or just 26% over the original plain text length.
The value of h1 decreased indeed, but not dramatically - to 4.33. But the value of h2 is now larger than even the plaintext: 3.30. And h1-h2 is now miserable 1.
So the trick with the alphabet did not pass. Why? I suppose that it is because in the original version of the cipher, the resulting n-grams are mainly comprised from two different sets of characters, like this: letter-digit. This essentially lowers h2, because a letter would almost never follow a letter, and a digit follows a digit not that often (only when radius is 10 or more). The same idea holds for my previous cipher, where constructs involve using designations of rows and columns of a table - basically, the same paradigm, two sets not mingled with each other.
So I guess that's what might be happening in the VMS. What is encoded is not single letters but bigrams, and they are encoded in some table manner, thus making h2 "anomalously" low, and h1-h2 high.
RE: Character entropy of Voynichese - Koen G - 13-12-2017
I like any experiment with bigrams, but there's something that's not clear to me. How do you differentiate between AE and EA? Do you always count distance towards the right?
RE: Character entropy of Voynichese - Anton - 13-12-2017
Yes, always towards the end of the alphabet.
AE -> C2
EA -> P10
RE: Character entropy of Voynichese - ReneZ - 14-12-2017
Hello Anton,
as I am sure you are fully aware, all entropy values capture in a single value an entire distribution of probabilities. For single character entropy, it is a 1-dimensional distribution of probabilities, while for character pair entropy it is a 2-dimensional distribution.
Finding a way to reduce the entropy values of a plain text is a very interesting exercise, but the real 'proof of the pudding' will be to compare the actual distributions.
I have started a new page where I have redone some plots that I did many years ago.
This illustrates in yet another way just how different the Voynich MS text is from, say, Latin.
The You are not allowed to view links. Register or Login to view. is still very incomplete.
RE: Character entropy of Voynichese - farmerjohn - 14-12-2017
It's a pity people make firm conclusions from "stats and entropy question". Especially when they are wrong. I wonder how many talanted researchers have gone wrong way after reading them.
There are persons on this forum possessed by "who said what" question. They will definitely love some statements there. After VMS is decoded.