(03-10-2017, 08:14 AM)ReneZ wrote: You are not allowed to view links. Register or Login to view.While dong some experimentation in this area, I ran into the following complication.
When working with character pair statistics, which are relevant both for entropy calculations and HMM analyses, there are three different ways of treating word spaces.
1) One counts word spaces among the characters to be analysed.
2) One deletes word spaces completely
3) One treats word spaces as breaks in the string.
Hello Rene,
most of the tests I have run followed method (1).
I believe that method (2) could be meaningful for languages (like Italian) that change word endings on the basis of the start of the following word. For instance, the Italian conjunction "e" becomes "ed" before a word starting with "e-": Adamo ed Eva; articles have different forms on the basis of the first sounds of the following words etc. Similar phenomena seem to happen in Voynichese.
I believe that method (3) was the one used by Reddy and Knight, but I haven't found a way to simulate it with the Python script I have been playing with.