HMM automatic vowel detection

[MarcoP]

Marco, when you mention a limit of 1000 characters, is this the length of the text?
From what I understand of the algorithm, the text should not have to be stored in memory, and should
only have to be read once, so such a low limit would not make much sense.

The overall VMS has about 160,000 characters (not counting spaces).

Hello Rene,
I am really using the software as a black-box, but I have the impression that the size has been limited to reduce processing time. I see empirically that the optimization algorithm takes longer when I increase the number of characters to consider. I guess the optimization must somehow match the output of the model with the actual data at each step: the bigger the set of data the slower the optimization process.

___________________________


I have made some new experiments on the basis of You are not allowed to view links. Register or Login to view. (thank you, David!).
Rene pointed me to an explanatory You are not allowed to view links. Register or Login to view. (thank you, Rene!).

I have downloaded the You are not allowed to view links. Register or Login to view. mentioned by Reddy and Knight.

Unluckily, the downloaded file is not identical to what is published in the paper. For instance, this is the first line of You are not allowed to view links. Register or Login to view. as published in the paper and in the online file:

Code:

        BAR ZC9 FCC89 ZCFAE 8AE 8AR OE BSC89 ZCF 8AN OVAE ZCF9 <f81v.1> BAR.ZC9.PCC89.ZCFAE.8AE.8AR.OE.BSC89.ZCF.8AN.OVAEZCF9-

As you can see, the first letter of the third word differs and the last word is split into two words in the paper.

Another problem is that Currier character 1 (corresponding to EVA:iiil iiil) doesn't occur in the voynich.now file (the sequence is encoded as IIIE). But these doubtful points are really marginal and are unlikely to have an impact on the results (iiil occurs 2 times in the whole manuscript, according to voynichese.com).
You are not allowed to view links. Register or Login to view.
Here I list the occurrences of each Currier character in the whole Voynich.now file:

You are not allowed to view links. Register or Login to view.

O 13689
9 10355
C 10315
8 7635
A 7490
S 6553
E 5654
F 5438
R 3711
P 3256
4 3232
Z 2804
M 2360
2 1358
N 1055
B 745
Q 705
X 559
J 495
* 246
I 210
V 202
T 165
W 146
D 110
3 86
U 71
6 60
Y 50
7 44
G 24
0 19
K 16
L 7
H 7
5 2
1 0

I have considerably increased the number of characters to be processed by the Python script (30000) and the results I obtain are once again different. The transition matrix results again in the two states alternating. Characters 9, O, A, S, Z (EVA: y o a ch sh y o a ch sh) are generated by one of the two states. The other state has a higher probability loop (43%) which I still think depends on the presence of Currier:C (EVA:e) which tends to occur multiple times consecutively. The different results could depend both on the new information in the much increased data set and on differences between the voynich.now file and the ivtt transcription I previously used.

While 9, O and A are consistently grouped together, it is not clear which other characters belong with them. C (EVA:e) or S/Z (EVA:ch/sh)?

You are not allowed to view links. Register or Login to view. (Sukhotin) applied on a different transcription pointed out EVA:o y a e (o y a e) as the characters most likely to correspond to vowels. But the issue is certainly difficult and, while I find the results of these algorithms interesting and instructive, I am sure they cannot provide any convincing result without further linguistic analysis.