+- The Voynich Ninja (https://www.voynich.ninja)
+-- Forum: Voynich Research (https://www.voynich.ninja/forum-27.html)
+--- Forum: Analysis of the text (https://www.voynich.ninja/forum-41.html)
+--- Thread: Character entropy of Voynichese (/thread-148.html)
RE: Character entropy of Voynichese - ReneZ - 14-12-2017
Only very few people are fully aware of the implications of the statistics, of which the entropy is only one, but an important one.
It is exactly because of this that, for the last decades, people have been trying to translate the text by simple substitution (or something very similar), and come up with meaningless text.
This will probably keep going on.
RE: Character entropy of Voynichese - Anton - 14-12-2017
Hello Rene,
My latest posts here are not part of any systematic work, but rather ad-hoc attempts to grab a trace of idea of what might be going on there in the VMS (supposing a ciphering technique is in place). Moreover, as Koen correctly pointed above, there are text characteristics of the Voynichese that are not addressed at all by my examples. However, they are not intended to mimic the Voynichese completely . What these examples show is that one does not need a verbose cipher (in the strict sense of the word) to lower entropies (which is something that was disputed earlier in this forum). That's actually their main focus. When an idea of some transform visits me, I write a script in Matlab to encipher a text chunk and see what it yields. More of a fun.
Thx for the link, that's not a quick read, will need some time to provide comments, if any.
RE: Character entropy of Voynichese - Anton - 14-12-2017
Rene, I have read the text, and I have a question - what's the procedure for calculating "first character"/"last character" entropies? Do you omit all characters in the text, except first/last characters of the vords, and then calculate entropy over the remainder, or what?
Btw, I think "Curva" is not a good international term, since it sounds like something not euphonic in Polish language.
RE: Character entropy of Voynichese - Torsten - 14-12-2017
(14-12-2017, 09:50 AM)ReneZ Wrote: You are not allowed to view links. Register or Login to view.Only very few people are fully aware of the implications of the statistics, of which the entropy is only one, but an important one.
It is exactly because of this that, for the last decades, people have been trying to translate the text by simple substitution (or something very similar), and come up with meaningless text.
This will probably keep going on.
Hello Rene,
How can you decide if someone is fully aware of the implications of the statistics without knowing the solution?
I would say that people try to solve the VMS because it is there and not because of weird statistics. In the past the most common mistake in deciphering undeciphered scripts was to guess the system behind the script. Many people think that it is only necessary to find the meaning for each glyph to read an unknown script. But to understand a text in an unknown script in an unknown language you have to reconstruct both the meaning for each glyph and the language used. In my eyes the same is true for the VMS.
What did we know about the system behind the text of the VMS? What did we know about the language of the VMS? One fact is that sequences of glyphs repeat in vertical and horizontal direction (gallow glyphs, line initial and line final letters, ...). What else did we know?
RE: Character entropy of Voynichese - ReneZ - 14-12-2017
Hello Anton,
I computed the single character entropy of all word-initial characters by simply counting the word-initial characters, and computing their frequency distribution. Similarly for word-final characters. As an example, Italian words typically end with a vowel, a quite reduced set, so a reduced character entropy for word-final characters.
Torsten, I don't know, of course, from everybody whether they understand the implications of some statistics or not.
In addition, having the statistics is one thing, but drawing the right conclusions from them quite another.
What I do know, however, and this is from communicating with a large number of people, that many people do not understand. All the people who are still playing with simple substitution do not understand.
Now it is not a crime not to understand, but it is not right to say that the statistics are wrong, or that they don't count for 'Voynichese'.
Specifically for your approach, there is no conflict at all with the entropy values that are observed for the MS text (i.e. string of symbols).
RE: Character entropy of Voynichese - Anton - 14-12-2017
Quote:I computed the single character entropy of all word-initial characters by simply counting the word-initial characters, and computing their frequency distribution.
But you compute frequencies across all the text, or across subsets of first/last letters only?
In the following example:
Code:
this is a test messagethe frequency of 't', for the word-initial, would be 2 or 3?
EDIT: Ah, OK, surely the former. Otherwise it would not make sense. I wonder what would be the respective result for second letters.
RE: Character entropy of Voynichese - Torsten - 16-12-2017
(14-12-2017, 07:02 PM)ReneZ Wrote: You are not allowed to view links. Register or Login to view.Torsten, I don't know, of course, from everybody whether they understand the implications of some statistics or not.
In addition, having the statistics is one thing, but drawing the right conclusions from them quite another.
What I do know, however, and this is from communicating with a large number of people, that many people do not understand. All the people who are still playing with simple substitution do not understand.
Now it is not a crime not to understand, but it is not right to say that the statistics are wrong, or that they don't count for 'Voynichese'.
Hello Rene,
the idea behind my question was that if someone didn't know the solution he should be aware that each of his conclusions is maybe wrong.
Quote:Specifically for your approach, there is no conflict at all with the entropy values that are observed for the MS text (i.e. string of symbols).
If someone wants to solve an undeciphered script he should try to extract the rules for this script. This is the way undeciphered scripts where deciphered in the past and this is the way I was trying for the VMS. All I have done was to extract rules for the system behind the VMS. Therefore it is no surprise if there is no conflict between the text of the VMS and the rules described by me for this text. The question is only if the rules described by me are enough to describe the text of the VMS?
RE: Character entropy of Voynichese - Anton - 16-12-2017
As a sidenote, at present no rules extracted or proposed do concern the gallows coverage. Neither it is accounted for in any statistical computation over the text.
RE: Character entropy of Voynichese - Torsten - 16-12-2017
(16-12-2017, 01:35 AM)Anton Wrote: You are not allowed to view links. Register or Login to view.As a sidenote, at present no rules extracted or proposed do concern the gallows coverage. Neither it is accounted for in any statistical computation over the text.
It possible to name plenty of rules concerning the use of gallows. For instance paragraph-initial lines start in 86 % of the cases with a gallow glyph. Paragraph-initial lines contain more words using gallow glyphs. Words containing a gallow glyph are on average longer. The glyphs p and f are frequent for the first line of a paragraph but rare anywhere else. The glyphs t and p are more commonly used at the beginning of words then expected. Words rarely end with a gallow glyph. If there is a word ending with a gallow glyph there is an increased chance for a second word ending with the same gallow glyph nearby. etc.
Concerning this gallow glyphs see also the following articles You are not allowed to view links. Register or Login to view., You are not allowed to view links. Register or Login to view. and You are not allowed to view links. Register or Login to view..
RE: Character entropy of Voynichese - Anton - 16-12-2017
Hi Torsten, what I'm speaking of is not gallows characters as such, but the "gallows coverage"- the phenomenon whereby the gallows' loop covers the subsequent characters. It's discussed in You are not allowed to view links. Register or Login to view. and subsequent posts. I'm not aware of anybody's having systematically investigated that and looking if any rules are exhibited.