10-04-2017, 08:10 PM
(This post was last modified: 10-04-2017, 09:36 PM by DonaldFisk.)
(10-04-2017, 06:33 PM)ReneZ wrote: You are not allowed to view links. Register or Login to view.There are many interesting aspects to what you have done, and this can be used for some additional interesting experiments. It will be worth to come back to that later.
On the other hand...
While it may not seem obvious at first sight, I see that Nick caught on to it as well, and your approach is conceptually the same as Gordon Rugg's.
Your result is better than Gordon's in one respect, and worse in another.
You generate text that really looks very much like the Voynich text, much more so than that of Gordon.
On the other hand, Gordon presents a simple means how it could be generated (in theory), whereas you don't.
Both are methods that try to reverse engineer the Voynich text "as we know it", but the result is not an exact match.
The method (both yours and his) would require further targeted tweaking in order to match the missing bits.
These include:
- to make sure that m appears predominantly at line ends (which is not yet the case)
- to make sure that f and p appear predominantly at first lines of paragraphs (which they don't yet).
- the special properties of the line-initial words ("line as a functional unit")
Having said that, I have a couple of questions.
1) Do I see correctly that you have different state transition probabilities for the different sections of the MS
2) Is each word started 'from scratch' or do the probabilities continue over word spaces.
The most interesting thing I find is that the Zipf law is followed so well just from the word generation based on state transition probabilities.
There's much more, but it will have to be later.....
Although I knew of Gordon Rugg's theory, I began this work assuming that there was meaningful content in the text, and this was reinforced by the PCA of the pages, which matches the illustrations quite well. I suspect Gordon Rugg's theory, with words as prefix+root+suffix instead of states based on individual glyphs, is equivalent to a special case of my theory.
I am concerned that my method for generating text would be significantly slower, but there might be a mathematically equivalent way of doing it faster. I haven't thought of one, though.
I paid very little attention to the level above words, but I did add checks for word-initial p and f, and when they occur my text generator starts a new line. I should implement this properly, as well as the other known the line and paragraph properties.
1) Yes, after running a cluster analysis, I divided the manuscript into 12 separate page clusters (4 herbal, 3 text, 3 bio, 1 pharma, and 1 astro) and each has its own state transition diagram.
2) Yes, each word starts from scratch. If this disagrees with a known property of the text, it would be worth seeing whether several space states work better than one.

