The Voynich Ninja
Engineering your own voynich - Printable Version

+- The Voynich Ninja (https://www.voynich.ninja)
+-- Forum: Voynich Research (https://www.voynich.ninja/forum-27.html)
+--- Forum: Voynich Talk (https://www.voynich.ninja/forum-6.html)
+--- Thread: Engineering your own voynich (/thread-3592.html)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13

RE: Engineering your own voynich - Koen G - 21-09-2021

Yeah I know, it's not entirely optimal. But at that point I gave up Smile

RE: Engineering your own voynich - nablator - 04-10-2021

(21-09-2021, 04:53 PM)Koen G Wrote: You are not allowed to view links. Register or Login to view.Yeah I know, it's not entirely optimal. But at that point I gave up Smile

I made my own similar cipher to see if I could solve it knowing the method and the parameters for sure (space + 26 letters, two cuts per ciphertext word):

Code:
tiiaoean aoooan yiielioa fiao qaaaenuy tuiib taaoool tiiioa teal qeioa nuegf temouy bumean hib qiaiean yiinn buelben yepfitioa tiimean qeioa qeal qeoan tiiaoean taparioa teaii tuellen qeemiunh qeunh taaooq lieliea aiiyun temean qaouy fvameean tafitean hiq qial tet qiyonuy liaeiinn fviiean aminn fvt fvegn tuei qiel yepaoioa fiao lim liii yepfitioa tiimean qeinn buii daiiean qeb bufitar qiel hiaaeq fvai tiiemi tiiaoean aaiial taegyun aameioa gamean fvelq kib fvelq tufityun aaaenuy hipen tiiyon liiiq liaei buelal nuyopen liii lielben liaeial tiiaiipen aaooq aeial yiiinn buooean aaeeean fvel fiemino qoemipen fvai liel limal nuyo hioouy lieliea aminn fvioa tiiaaen liaaenuy afitean qeaaeioa teiial qeaeiean aminn fvoan taaooiea lielpen aaaef tiiaooan taiiinn liema buelben aemaf taemaoan teaii qiame tiiaoean aeial qeopen fifitioa qom bufitean aaiean daaeq fvai uaaeouy liemif teinn yiiemaoan tiiaoean teaiipen aaoal uaii daiiean qepen tuamiiea fviiq fvai lielb nuoben lieliea aminn bum lim liii buel qiminn fvfitunh liiiean aonuy taaiib qifitben fvai teaeiouy liemaq aameoan taal aelal tiioouy bueg nuaaenuy koal kaii qifit fieminuy temouy tuaeiq fvai nuaaenuy koal kaii buel tuelouy fvaaeben aaeioan hiaii fiyool kiaiiouy aemiouy aemiq kiaiiouy aameoan taei qiame fifityun qomoan kfital qoaoyun aemiouy aiiioa qaeman yiial aeiean buelioa qeamin fviipen aaooan bux

It's harder than I thought. A simple hill climb algorithm leads to sub-optimal outcomes (ambiguities or too many letters). Simulated annealing is either extremely slow or doesn't reach the optimum.

Does anyone have an idea for a better algorithm?

EDIT 1: One possible idea for improvement is to prioritize moving cuts in words that produce the highest number of ambiguities.

EDIT 2: Reading the comments for the first time here: You are not allowed to view links. Register or Login to view.

Quote:Mark Knowles on You are not allowed to view links. Register or Login to view. said:
I should probably say that if one has a triglyph
$&#
Then one shouldn’t also have the biglyph
&#
And similarly not
#

Actually this is not a necessary condition for a deterministic ciphering and deciphering. Having $&# and $& and $ at the same time is also possible.

* * *

Now I realize one of the possibly reasons for having "too much structure" in a ciphertext: spaces can be optionally added/removed in many places to make it even more difficult to crack. Wink

RE: Engineering your own voynich - Mark Knowles - 04-10-2021

(04-10-2021, 08:10 AM)nablator Wrote: You are not allowed to view links. Register or Login to view.EDIT 2: Reading the comments for the first time here: You are not allowed to view links. Register or Login to view.

Quote:Mark Knowles on You are not allowed to view links. Register or Login to view. said:
I should probably say that if one has a triglyph
$&#
Then one shouldn’t also have the biglyph
&#
And similarly not
#

Actually this is not a necessary condition for a deterministic ciphering and deciphering. Having $&# and $& and $ at the same time is also possible.

I appreciate you finding this old quote. You are technically correct that this is not a necessary condition for a deterministic ciphering and deciphering, which is probably why I didn't say it was. However the situation I have described risks leading to confusion. If ($&#) = ($&)(#) = ($)(&#) in 3 different readings then of course it may well be that only one of these fits the context.

RE: Engineering your own voynich - nablator - 04-10-2021

(04-10-2021, 08:13 PM)Mark Knowles Wrote: You are not allowed to view links. Register or Login to view.If ($&#) = ($&)(#) = ($)(&#) in 3 different readings then of course it may well be that only one of these fits the context.

Here we have three mappings. I understand this doesn't fit the conversation on the blog, sorry. I was thinking of possible limitations, ways to rule out mappings easily with such a criterion as "no substrings allowed". A string and some of its substrings (at the start or end) coding for different cleartext letters at the same place (one of the three places of ciphertext words; first, middle, last in this case as every ciphertext word is made up of three parts) are not necessarily ambiguous. It actually happens in my sample cipher, without any danger of confusion because only one choice of part1, part2, part3 is possible for each word of the ciphertext.

RE: Engineering your own voynich - mja0075 - 28-02-2023

(04-10-2021, 08:10 AM)nablator Wrote: You are not allowed to view links. Register or Login to view.
(21-09-2021, 04:53 PM)Koen G Wrote: You are not allowed to view links. Register or Login to view.Yeah I know, it's not entirely optimal. But at that point I gave up Smile

I made my own similar cipher to see if I could solve it knowing the method and the parameters for sure (space + 26 letters, two cuts per ciphertext word):

Code:
tiiaoean aoooan yiielioa fiao qaaaenuy tuiib taaoool tiiioa teal qeioa nuegf temouy bumean hib qiaiean yiinn buelben yepfitioa tiimean qeioa qeal qeoan tiiaoean taparioa teaii tuellen qeemiunh qeunh taaooq lieliea aiiyun temean qaouy fvameean tafitean hiq qial tet qiyonuy liaeiinn fviiean aminn fvt fvegn tuei qiel yepaoioa fiao lim liii yepfitioa tiimean qeinn buii daiiean qeb bufitar qiel hiaaeq fvai tiiemi tiiaoean aaiial taegyun aameioa gamean fvelq kib fvelq tufityun aaaenuy hipen tiiyon liiiq liaei buelal nuyopen liii lielben liaeial tiiaiipen aaooq aeial yiiinn buooean aaeeean fvel fiemino qoemipen fvai liel limal nuyo hioouy lieliea aminn fvioa tiiaaen liaaenuy afitean qeaaeioa teiial qeaeiean aminn fvoan taaooiea lielpen aaaef tiiaooan taiiinn liema buelben aemaf taemaoan teaii qiame tiiaoean aeial qeopen fifitioa qom bufitean aaiean daaeq fvai uaaeouy liemif teinn yiiemaoan tiiaoean teaiipen aaoal uaii daiiean qepen tuamiiea fviiq fvai lielb nuoben lieliea aminn bum lim liii buel qiminn fvfitunh liiiean aonuy taaiib qifitben fvai teaeiouy liemaq aameoan taal aelal tiioouy bueg nuaaenuy koal kaii qifit fieminuy temouy tuaeiq fvai nuaaenuy koal kaii buel tuelouy fvaaeben aaeioan hiaii fiyool kiaiiouy aemiouy aemiq kiaiiouy aameoan taei qiame fifityun qomoan kfital qoaoyun aemiouy aiiioa qaeman yiial aeiean buelioa qeamin fviipen aaooan bux

It's harder than I thought. A simple hill climb algorithm leads to sub-optimal outcomes (ambiguities or too many letters). Simulated annealing is either extremely slow or doesn't reach the optimum.

Does anyone have an idea for a better algorithm?

EDIT 1: One possible idea for improvement is to prioritize moving cuts in words that produce the highest number of ambiguities.

Was hoping to revive this thread as I haven't been able to find any good existing tools/algorithms for verbose ciphers.
I agree that this was harder than anticipated, did you every do any further work on this?

I had luck decoding your cipher with a hill-climbing algorithm to try and find an X length(52 seemed to work for yours) key that produce the shortest text and most successful coverage of the text first and then moving on to decrypting.

You are not allowed to view links. Register or Login to view.
Plugging this into AZdecrypt's Substituiton + polyphone tool using 8grams(with "?" allowing 26 polyphones and every other character limited to 1 gives):

Code:
THE VOYNICH MANUSCRIPT IS AN ILLUSTRATED TO SE YES
AND WRITTEN IN ANOTHERS BISE UN BE KNOWN WRITINGEY
STEM REFERRED TO AS BOY NICHESE THE BELLUM ON WHICH
HIS WRITTEN HAS BEEN CARBON DATS TO THER EARLY IF IT
FEENTH CENTURY LAND STYLIST I CAN ANDS IS INDICATE
SKIM MAY HALLEAVE ENCORE POSES IN HAND DURING THE ITALIAND
RENAISSANCE THE ORIGINS CARTIER SHIP AND SPURPOSE OF
THE CANTS CRIPT ARE USED AT S VARIOUS HYPOTHESES I
HAVE BEEN SUTS GUESTS INCLUDING THAT HIS ANOTHERWISE
HN RECORDESS CRIPM FOR AL NATURALL AND AGE OR CONSTRUCT
SLAND ARE AN UNREAD AIR DECYPHERS ORM OTHERE FORM OF
CRYPTOGRAPHY MORE IMPLY ACE AN IN TIMES SEVE AR

Which sure looks like the first paragraph on the voynich wikipedia page at the time Smile

This obviously isn't quite good enough yet for fully decoding without some elbow grease (which I admittedly haven't done to reproduce your full algorithm), but thought I would share what worked for me.

As for byatan's cipher I haven't had the same luck, so possibly more involved?

RE: Engineering your own voynich - nablator - 28-02-2023

(28-02-2023, 03:41 AM)mja0075 Wrote: You are not allowed to view links. Register or Login to view.I had luck decoding your cipher with a hill-climbing algorithm to try and find an X length(52 seemed to work for yours) key that produce the shortest text and most successful coverage of the text first and then moving on to decrypting.

You are not allowed to view links. Register or Login to view.
Congratulations! I'm surprised that it worked at all: all codes are between 0 and 3 characters long, a few of them being shared between different positions, standing for different plaintext letters. I could have used the zero-length codes for a frequent letter like 'e' instead of space and shared a lot more codes between different positions to make it more difficult to solve with a general-purpose cipher/code solver program.

Solution:
You are not allowed to view links. Register or Login to view.
Key:
You are not allowed to view links. Register or Login to view.


RE: Engineering your own voynich - nablator - 10-03-2023

(28-02-2023, 03:41 AM)mja0075 Wrote: You are not allowed to view links. Register or Login to view.As for byatan's cipher I haven't had the same luck, so possibly more involved?
A possible breakdown of byatan's cipher in 3 columns (27+23+22 codes, not ambiguous):
F              ai             T
K              z              z
n              m              nuy
m              ai             g
q              a              r
Ti             oo             iea
b              a              r
n              m              aei
T              m              z
q              oo             iea
Ti             ae             ean
b              a              r
n              m              aei
T              ei             b
Tiii           i              z
Ti             ei             r
Ki             m              b
m              v              g
K              aii            n
F              v              g
Tii            i              g
Ki             ei             iea
ki             i              n
F              m              q
Fiii           i              z
m              a              yun
T              ai             yun
qiii           a              T
m              m              q
TI             ei             g
f              aai            g
Fi             a              ean
K              aae            aei
F              ai             yun
F              oo             nuy
Ti             a              vinn
Ti             ae             aei
Tiii           i              g
F              ai             T
K              z              z
K              aE             g
qii            m              iea
m              oo             l
m              ai             l
m              ai             iea
Tiii           i              g
Kii            a              g
Ki             ei             iea
fi             ee             ean
Ti             i              g
K              ai             iea
n              aai            ben
n              e              r
n              ae             k
q              e              K
ni             i              nuy
m              m              iea
TI             a              yun
n              a              t
F              ooi            x
f              am             ean
f              v              T
Kii            i              yun
K              aaa            ouy
mi             i              g
m              m              n
t              a              uYnh
Fi             oo             g
K              aei            T
ki             a              q
m              e              aei
Ti             a              vinn
Ti             e              nuy
F              ai             yun
F              oo             g
fi             i              n
F              v              T
Ti             aa             ouy
q              a              yun
Ti             a              ean
K              am             ean
Fi             i              n
F              a              aei
m              ae             l
F              i              r
Ti             a              n
F              oo             yun
Fii            i              yun
F              ai             yun
F              oo             T
fii            i              r
Ti             a              n
F              oo             yun
Fii            i              yun
Ti             oo             z
fi             ei             k
f              aoo            b
K              ae             yun
m              m              r
t              e              aei
Ti             ae             ean
F              m              l
m              m              q
Fi             a              q
m              e              aei
Ti             a              vinn
Tii            i              n
F              oo             iea
Ti             ae             ean
b              a              r
n              m              aei
T              ei             b
mii            i              z
fii            i              g
K              aii            n
F              a              q
F              m              yun
Ki             e              r
m              v              r
f              aai            aei
F              m              yun
T              ae             aei
K              am             g
Ti             ee             g
F              ai             yun
n              a              t
F              ooi            nuy
m              ai             l
Tiii           i              g
q              e              K
ni             i              nuy
n              ae             yun
n              a              z
mii            i              t
F              ooi            ouy
Ki             a              nuy
K              aii            n
F              ei             g
m              aa             ean
Ti             i              uYnh
f              ao             yun
n              a              nuy
qii            ei             yun
F              z              l
Fi             a              g
fi             i              nuy
f              aai            z
K              aa             yun
f              ao             b
m              m              n
T              ai             g
K              aa             yun
Ti             e              aei
K              ei             yun
n              m              yun
n              aa             ean
Fi             i              g
fi             a              ean
m              v              q
qii            m              n
m              ai             K
F              i              l
m              e              q
qii            i              pen
Ki             e              aei
Ki             i              n
F              ee             z
Ki             m              z
K              aa             l
f              aai            nuy
F              m              ean
Fi             i              t
F              ooi            q
T              ei             yun
K              ai             T
K              aii            ean
T              m              pen
Ti             a              l
T              ai             z
f              aa             z
fi             m              g
m              ai             l
f              aai            iea
Tiii           i              g
Ki             ei             g
Tii            i              T
fii            i              nuy
Tiii           i              z
Ti             ee             g
Ki             a              r
n              m              aei
K              a              l
k              ae             ean
F              m              r
n              z              g
K              aei            z
K              a              yun
n              a              ean
F              m              g
T              aa             T
fii            i              nuy
Ti             m              yun
T              ae             aei
n              m              nuy
Ti             ae             t
fi             ae             ben
m              ai             l
TI             ei             g
Tiii           i              g
Fii            i              z
mi             i              nuy
Fi             i              yun
T              ai             K
K              aa             b
Fi             v              iea
qii            a              l
m              ooi            yun
n              a              yun
Ki             m              aei
K              az             T
Tii            i              T
fii            i              nuy
Ti             m              yun
T              ae             aei
Ti             ae             nuy
TI             m              r
F              ei             nuy
k              aa             q
q              ooi            l
F              m              ean
qiii           a              yun
m              z              g
K              aei            z
K              a

Converted to a single character per column:
!Aa"Bb#Cc$Ad%De&Ef'De#Cg(Cb%Ef&Fh'De#Cg(Gi)Hb&Ge*Ci$Id"Jj!Id+Hd*Gf,Hj!Ck-Hb$Dl(Al.Da$Ck/Gd0Kd1Dh"Lg!Al!Ec&Dm&Fg)Hd!Aa"Bb"Md2Cf$En$An$Af)Hd3Dd*Gf4Nh&Hd"Af#Ko#Oe#Fp%Oq5Hc$Cf/Dl#Dr!Ps0Qh0Ia3Hl"Rt6Hd$Cj7Du1Ed"Sa,Dk$Og&Dm&Oc!Al!Ed4Hj!Ia&Tt%Dl&Dh"Qh1Hj!Dg$Fn!He&Dj!El8Hl!Al!Ea9He&Dj!El8Hl&Eb4Gp0Ui"Fl$Ce7Og&Fh!Cn$Ck1Dk$Og&Dm+Hj!Ef&Fh'De#Cg(Gi:Hb9Hd"Jj!Dk!Cl*Oe$Ie0Kg!Cl(Fg"Qd&Nd!Al#Dr!Pc$An)Hd%Oq5Hc#Fl#Db:Hr!Pt*Dc"Jj!Gd$Th&Hu0Vl#Dc2Gl!Bn1Dd4Hc0Kb"Tl0Vi$Cj(Ad"Tl&Og"Gl#Cl#Th1Hd4Dh$Ik2Cj$Aq!Hn$Ok2Hv*Og*Hj!Nb*Cb"Tn0Kc!Ch1Hr!Pk(Gl"Aa"Jh(Cv&Dn(Ab0Tb4Cd$An0Kf)Hd*Gd+Ha9Hc)Hb&Nd*De#Cg"Dn;Fh!Ce#Bd"Sb"Dl#Dh!Cd(Ta9Hc&Cl(Fg#Cc&Fr4Fo$An/Gd)Hd8Hb6Hc1Hl(Aq"Ti1If2Dn$Pl#Dl*Cg"Wa+Ha9Hc&Cl(Fg&Fc/Ce!Gc;Tk%Pn!Ch.Dl$Bd"Sb"D

Is there a homophonic substitution solver that can use this conversion as input, with the specification that the ASCII ranges [33-64], [65-96] and [97-128] are monoalphabetic?

These ranges are not the type of parameters that can be found in any program that I know... maybe a modified version of an existing program could solve it?

RE: Engineering your own voynich - RobGea - 10-03-2023

There is the homophonic cipher solver developed to solve the Zodiac cipher, but works on any homophonic/substitution cipher: ZKDecrypto.
Windows app but theres source code on the git.
You are not allowed to view links. Register or Login to view.

There is also Cryptool2 but thats windows only, though theres a java version but idk if that has got a homophone solver.
You are not allowed to view links. Register or Login to view.

Also, "AZdecrypt is a fast and powerful hillclimbing classical cipher solver written in You are not allowed to view links. Register or Login to view.. "
but idk anything about it or FreeBASIC.
You are not allowed to view links. Register or Login to view.

There is an online lite version ,that might be worth a try.
You are not allowed to view links. Register or Login to view.

RE: Engineering your own voynich - nablator - 10-03-2023

(10-03-2023, 12:11 PM)nablator Wrote: You are not allowed to view links. Register or Login to view.Is there a homophonic substitution solver that can use this conversion as input, with the specification that the ASCII ranges [33-64], [65-96] and [97-128] are monoalphabetic?

Is this what AZdecrypt calls "Substitution + monoalphabetic groups" ?
When I click the "Solve" button it displays:
Error: solver only works with letter 8-grams or higher

Or I could convert all three parts to the same range and call it a generic polyalphabetic substitution cipher with a key of length 3:

AAABBBCCCDADEDEFEFGDECCGHCBEEFFFHGDECCGHGIIHBFGEJCIDIDBJJAIDKHDJGFLHJACKMHBDDLHALNDADCKOGDPKDQDHBLGAALAECFDMFFGIHDAAABBBBMDRCFDENDANDAFIHDSDDJGFTNHFHDBAFCKOCOECFPEOQUHCDCFODLCDRAPSPQHPIASHLBRTVHDDCJWDUQEDBSALDKDOGFDMFOCAALAEDTHJAIAFTTEDLFDHBQHQHJADGDFNAHEFDJAELXHLAALAEAYHEFDJAELXHLFEBTGPPUIBFLDCEWOGFFHACNDCKQDKDOGFDMKHJAEFFFHGDECCGHGIZHBYHDBJJADKACLJOEDIEPKGACLHFGBQDFNDAALCDRAPCDANIHDEOQUHCCFLCDBZHRAPTJDCBJJAGDDTHFHUPVLCDCRGLABNQDDTHCPKBBTLPVIDCJHADBTLFOGBGLCCLCTHQHDTDHDIKRCJDAQAHNDOKRHVJOGJHJANBJCBBTNPKCACHQHRAPKHGLBAABJHHCVFDNHABPTBTCDDANPKFIHDJGDKHAYHCIHBFNDJDECCGBDN[FHACECBDBSBBDLCDHACDHTAYHCFCLHFGCCCFFRTFODANOGDIHDXHBVHCQHLHAQBTIQIFRDNDPLCDLJCGBWAKHAYHCFCLHFGFFCOCEAGC[TKEPNACHNDLDBDBSBBD

I guess AZdecrypt could maybe solve it with substitution + polyphones: when I click the "Solve" button it opens a dialog where I can enter:
Set letters # for all symbols to: 3
The button seems to detect correctly that some symbols (Z and [) appear only twice in the cipher text, all right.
Then what? There is no "Start Solver" or "OK" button.

You are not allowed to view links. Register or Login to view. could you please explain how you started the solver in substitution + polyphones mode and where did you find an English 8-grams file?

RE: Engineering your own voynich - Mark Knowles - 10-03-2023

I don't quite understand how you can solve, all but the most simple ciphers of the time with a program like these without knowing context. I have been working on the decipherment of letters recently where understanding context is crucial to revealing the cipher.