• Options
• Login
• Desktop Style

Least effort generator

• Index

  • The Voynich Ninja
  • Voynich Research
  • Analysis of the text
  • Least effort generator
• Least effort generator

• Least effort generator

  tikonen > 13-07-2026, 09:21 PM

  Disclaimer. This is a hobby project and I'm not trying to solve the manuscript, I think it's a medieval scam. Just hoping to hear ideas, comments and critique.
  
  My working assumption is that the manuscript is a forgery made to part rich people of their money.  I've been thinking how it was generated in seemingly consistent way. Maybe it was done by sweat shopping few cleric students for pocket money and minimal time and and money was invested to train them.
  
  So I guess there was possibly a template or a collection of tokens the scribes used to learn to generate VM's fantasy words with some personal twist, mistakes and tweaks. The binomial distribution of a word length gives some support to this idea, indicating that the word lengths may be random. After some practice the tokens were not needed anymore and scribe would just make it on the fly. VM's high word "kindness" to the previous rows could also support this idea.
  
  To test the feasibility of this idea I wrote a simple program that when given a word from VM it finds a minimal number of predefined tokens in order to construct the word. I did not consider words that are used only twice or less and the ones with rare letters. 
  
  Here is and example of one token list set I conjured.  
  Word is constructed by picking a token from each list (or skipping it) and concatenating them to a word. 
  As you can guess list s0 is considered first when building a word, then list s1 and so on until s5 as last. A token from each list can be used only once for a word. 
  This limited example already covers (with above mentioned filtering) 34% of all the unique words and 75% of all words (many words are used multiple times). With this set the average number of needed tokens for common words with >=50 instances is 2.1. (For >=10 it's 2.5). So basically one can generate most VM words by picking just few tokens.
  
  s0 = ['qok', 'qoke', 'qot', 'ot', 'd', 'q', 'l', 'yk']
  s1 = ['ch', 'sh', 's', 'ok']
  s2 = ['e', 'o', 'r', 'k']
  s3 = ['r', 'k', 'o', 'e']
  s4 = ['cth', 'ckh']
  s5 = ['daiin', 'aiin', 'ain', 'ey', 'edy', 'dy', 'y', 'ar', 'or', 'al', 'am', 'air', 'ody', 'ol']
  
  Example: Word 'chedy' would be token 'ch' from list s1 and 'edy' from s5.
  
  Position on each token in each list reflects how often it's used in VM words. Each word may have multiple "solutions", algorithm picks shortest.  
  It's relatively easy to make a larger set for >95% word coverage but then of course the average number of token picks increases (slowly). 
  
  Some points:
  - One can generate non VM words if tokens are chosen totally randomly. Some external rules are required. 
  - Real VM words do not use much s2 and s3. Omitting them drops coverage from 75% to 66%.
  - Impossible to draw line how much is template and how much is just random on the fly variation of scribes
  - There are some words that have high occurrence but complicate the set, they need a token that is used mostly on that word. 
   
  I don't know if this has any merit but at least it's an interesting exercise.

• RE: Least effort generator

  oshfdk > 13-07-2026, 09:28 PM

  (13-07-2026, 09:21 PM)tikonen Wrote: You are not allowed to view links. Register or Login to view.Some points:
  - One can generate non VM words if tokens are chosen totally randomly. Some external rules are required. 
  - Real VM words do not use much s2 and s3. Omitting them drops coverage from 75% to 66%.
  - Impossible to draw line how much is template and how much is just random on the fly variation of scribes
  - There are some words that have high occurrence but complicate the set, they need a token that is used mostly on that word. 

  
  Hi!
  
  This is certainly a possibility, the idea itself is not particularly new. Did you look into a way to generate statistically probable distributions or sequences of these tokens?
  
  Also, there can be easy counterexamples. If you take Chinese written phonetically (pinyin), then overwhelmingly all words will be created using a combination of 1-3 simple tokens. It's likely the same is true for Vietnamese, which actually uses a script based on Latin characters. This doesn't mean that all Vietnamese books are just a few scribes randomly combining tokens.

• RE: Least effort generator

  tikonen > 18-07-2026, 01:22 PM

  (13-07-2026, 09:28 PM)oshfdk Wrote: You are not allowed to view links. Register or Login to view.This is certainly a possibility, the idea itself is not particularly new. Did you look into a way to generate statistically probable distributions or sequences of these tokens?
  
  Also, there can be easy counterexamples. If you take Chinese written phonetically (pinyin), then overwhelmingly all words will be created using a combination of 1-3 simple tokens. It's likely the same is true for Vietnamese, which actually uses a script based on Latin characters. This doesn't mean that all Vietnamese books are just a few scribes randomly combining tokens.

  I'm planning to map the sequences to numeric form for easier analysis. In any case if this kind of system was used as a training tool it's unclear how it would show in the final text. Maybe nothing conclusive can be said but as said earlier I would like to come up with a simple system that could have been used with the knowledge available at the time.
  
  Thank you for pointing out some counter examples, I've no knowledge of Pinyin and will have to look into it as a case example. Based on several text examples I glanced I don't immediately see obvious comparison to VM style text. Of course Vietnamese nor South-East Asian books cannot be generated with a random method but my assumption stands that VM is just gibberish with just enough structure to make it look like a real language. Even today many comedians are really skilled in talking in gibberish that sounds exactly like a real language when they or listeners don't speak the language.
• Next Oldest Next Newest