A One-Page Ledger Method for Generating Voynich-Like Text

[Dunsel]

The manuscript doesn't fall apart because every word was seen and accepted by a human before it became part of the source pool for the next word. The constraints are the pattern recognition abilities and the habits of the scribe himself.

I couldn't agree with you more. And again, that brings human psychology into the equation. Which patterns?  And then, how do you model those patterns?  At what point do the weights and rules become more than a 15th century scribe can do?  That's where I am now.

For examaple:

In the other post on <ed> I mentioned I did some machine learning, trying to find syllables.  I refined that down to where only 3 words in Takahashi and 14 in Zandbergen/Landini can't be mapped has having good, learnable n-grams in Scribe 1 content.  It came up with a count of 150 main n-grams and 50 more that that act as connectors.  I added that learned chunk data into my generator and it produced the code blocks I pasted below.  

The ledger model still attempts to do a mutation and validates it. The new word is then, it's passed off to the chunk model to see if it contains valid n-gram chunks with valid neighbors.  It looks more Voynich like that previous runs and still matches statistics pretty well. 

But, is this still human doable.  I think so but, it may require more than just the scribe's eyes . There may have been an audible component, where the scribes were familiar enough with the output tokens that they pronounced them.  I know I pronounce Voynich words to see if they sound right. I pronounce words as I write them.  Might a Voynich scribe do the same?

Code:

<f1v> <f1v.P1.1;K>        fchol.shaiin.odad.fchol.odd.ose.cfhealy.chol <f1v.P1.2;K>        chotey.yoo.or.syaiir.cther.ckhea.chtain <f1v.P1.3;K>        pchol.ycheeo.chol.shodd.pose.odad <f1v.P1.4;K>        pydaey.okain.yor.pchol.oydar.pchol.kshiy.yorr <f1v.P1.5;K>        fchol.os.fchol.s.daiin <f1v.P1.6;K>        osdar.okod.qos.daiim.pshol.ycheeo <f1v.P1.7;K>        oar.okeol.oateos.opeos.daiin.fchol.chool.mor <f1v.P1.8;K>        os.chol.oos.pchol.kaiin <f1v.P1.9;K>        cfhoain.fcho.chal.rokain.cchol.fchol.cthar.s <f1v.P1.10;K>        kor.chol.chols.ydain.s.lchol.sshey <f1v.P1.11;K>        chool.diiin.fchol.kor.s.fchol.yocho <f25r> <f25r.P1.1;K>        ros.dloctha.sory.dain.eey.dloctha.koshey.shol.daraii <f25r.P1.2;K>        kshy.dlocta.ycho.kshy.dain.ikan <f25r.P1.3;K>        cthooldar.chodaiils.cthy.okaiirg.chodaiil.ctholdar <f25r.P1.4;K>        chotey.odar.cfhol.csol.shory.dloctha.ecthoary <f25r.P1.5;K>        kdail.dlocta.dlocfha.cthey.dair.dain.daid <f25r.P1.6;K>        cphoy.dlocta.dloctha.tchol.dloctha.cthoary <f25r.P2.7;K>        okar.chal.dair.cphoy.cfal.dlocta.okaiir.eos <f25r.P2.8;K>        oycheey.cphodaiil.daire.chhealy.chealy <f25r.P2.9;K>        oar.tchol.dlocta.tchol.dlocfha.dlocpha.dlocka.dlocpha.cphealy <f25r.P2.10;K>      kaiirg.eckhoary.fchol.tchol.dlocphe.dair <f25r.P2.11;K>      kail.dlocka.os.pchil.otchol.dlocka.ckdey.pchol.dsory <f25r.P2.12;K>      dlocpha.pchol.cthol.oaky.chodaiila.ctoil <f50v> <f50v.P1.1;K>        dan.cphoy.chor.dshkoldy.dan <f50v.P1.2;K>        kyey.teody.ydain.cphar.dkchar.y <f50v.P1.3;K>        cfhol.cphesaiin.chor.dshpoldy.sckhey <f50v.P1.4;K>        cshyds.kodshey.cphoy.cthar.chor.kon.choydr.dain <f50v.P1.5;K>        oas.chol.cphar.shok.shon.otol <f50v.P1.6;K>        shos.rchol.oiin.scphey.ooiin.sho.cphar.oydar <f50v.P2.7;K>        y.ro.oeykoal.rchol.kchol.fchol.chodor <f50v.P2.8;K>        rol.olys.ytais.eolys.dshpoldy.cthey.y.sho <f50v.P2.9;K>        okol.fchol.dshkoldy.y.dshpoldy <f50v.P2.10;K>      cfhyl.chor.ytain.stchey.chsy <f50v.P2.11;K>      eoe.ckhar.rchol.dshpolday.ddchar.oteos <f50v.P2.12;K>      cphey.ckhey.oekoal.oteol.chey.stchey.cfhol <f50v.P2.13;K>      y.oas.chey.chol.alg <f50v.P2.14;K>      pydeeyl.rol Page-count rule: output pages = Voynich pages = 100 Voynich pages available: 225 Voynich pages compared: 100 Generated pages: 100 Output tokens: 8234 Output vocabulary size: 2463 Output hapax count: 1557 Voynich comparison tokens: 8519 Voynich vocabulary size: 2436 Voynich hapax count: 1642 Ledger invalid: 0 Operation counts:   external_copy: 1065   external_copy_fallback: 14   external_ed1: 1961   external_short_copy: 13   initial_gallows_construct: 88   initial_gallows_seed: 10   local_copy: 951   local_copy_fallback: 164   local_ed1: 1954   novel_ed1: 1830   seed: 184 Gallows action counts:   attested_ed1: 2903   gallows_delete: 54   gallows_insert: 63   gallows_substitute_in: 35   gallows_substitute_out: 121   gallows_swap: 162   initial_construct: 88   initial_seed: 10 Top 20 output bigrams (100 pages):   ho: 2214   ch: 1934   he: 1295   sh: 1274   ol: 1254   ai: 1116   ey: 997   in: 887   da: 815   th: 778   ct: 749   or: 739   ii: 720   ar: 685   ha: 510   od: 499   hy: 468   cp: 382   eo: 373   ee: 368 Top 20 Voynich bigrams (100 pages):   ch: 3082   ho: 1920   ai: 1433   in: 1374   ii: 1281   ol: 1239   he: 1116   da: 1116   sh: 1033   dy: 1024   hy: 948   ok: 930   or: 922   qo: 764   ot: 749   ar: 562   ey: 554   od: 548   ee: 541   eo: 467 Top 20 output trigrams (100 pages):   cho: 969   hol: 836   sho: 702   cth: 682   hey: 677   aii: 571   dai: 520   iin: 505   che: 497   cph: 352   ain: 333   she: 332   tho: 275   hod: 262   oda: 226   kch: 221   ckh: 218   eey: 204   cfh: 186   dar: 181 Top 20 Voynich trigrams (100 pages):   cho: 1236   iin: 1181   aii: 1139   dai: 728   che: 718   hol: 596   chy: 492   sho: 467   hor: 467   cth: 435   tch: 420   kch: 392   edy: 366   qok: 359   she: 295   hey: 291   ody: 253   qot: 251   otc: 244   heo: 241

After generating 1000 pages, it starts to look less Voynich-like but still has familiar patterns.

Code:

<f500v> <f500v.P1.1;K>      shokcheey.ods.lochy.ol.dar.cfhaiin.shkaiir.moaokchy <f500v.P1.2;K>      opsho.cfhol.kooishey.ol.cfhaiin.ol <f500v.P1.3;K>      oyr.maokchy.kachys.ool.chol.oal.chol.shoasho <f500v.P2.4;K>      fool.oal.keishey.oalg.cthol.lochy.y.ol <f500v.P2.5;K>      maokhy.cfhol.dain.ool.oal.cpod <f500v.P2.6;K>      oaokchy.ol.ool.ol.oaikdon <f500v.P2.7;K>      cphey.y.shear.ord.ckhaiin.shtasho.cthey.ckhhaeiin.oaidoon <f500v.P2.8;K>      ool.maokhg.okchey.okeey.sor.cpod.lol <f500v.P2.9;K>      old.deydd.cthol.she.dain <f500v.P2.10;K>      ol.shey.dair.kol.ols.ostairin.tol.roe.ol <f500v.P2.11;K>      sheo.old.cfhol.shory.ol.cod.shyoolso.ror.cfhoaiin <f500v.P2.12;K>      maokhy.ois.shoolsho.lshoolsho.oil.toche.skaiir

And here's page 10,000.  I assumed if the model was going to collapse, it would have done so by this point.  I can't provide any stats to compare to Voynich other than to say it's a bit heavy on longer words which could be adjusted.

Code:

<f5000v> <f5000v.P1.1;K>      kore.dain.shey.chy.otaiin.qosoier.kore <f5000v.P1.2;K>      raiisoey.cthal.cthodaiils.pdeiimy.airg.qosoilr.ylsoier <f5000v.P1.3;K>      dain.chal.y.chal.oor.chodain.cthodaiils.odaiin <f5000v.P1.4;K>      dpdaeim.cthy.ore.qosoier.daicthy.y.kshy <f5000v.P1.5;K>      qosoier.odar.dain.kshy.olsdeoey.eefy <f5000v.P1.6;K>      chy.dain.oor.qoosoieor.dair.lshytoo.cthodaiils.qore <f5000v.P1.7;K>      dain.airg.cphey.kodaiin.dair.alshyoe <f5000v.P2.8;K>      fairg.s.sshyood.qosoilr.chok <f5000v.P2.9;K>      sory.olsoior.olsooer.dair.alshyey.alshiey <f5000v.P2.10;K>    fair.cthodaiils.fairg.olsoier.oor.sshysod.dair.pachys.s <f5000v.P2.11;K>    oir.alshyoe.alshyee.qosoier.shey.chok.olsoar.ckhealy