[ey] and [dy] Feature Patterns

[Emma May Smith]

The measures I'm using are z--scores, not token counts. The z--scores show how unusual the occurrence of a feature is in one position of a distribution. If the distribution of a feature is very flat, then a small increase or decrease in occurrence will show up as quite significant. If the distribution is very uneven, then event a relatively large increase or decrease in a position won't be significant. Thus token counts might look the same, but have different meanings.

I think z--scores do have flaws, but:
1) Wordbreak combinations which can be seen with frequency counts also show up well with z--scores.
2) Elements of Latin morphology (as mentioned above) also show up with z--scores.

The benefit is that the z--score is a single measure which---while not perfectly comparable---lets us identify things of interest which we might otherwise miss.

I would like to upload all the data I have (I'm more than willing to share) but the upload limit here is lower than the size of the file. So I'll share a few results to demonstrate the different counts/z--scores. (I also run through an example in my Voynich Day talk, and it is available in the slides.)

For [ey.], [ey.], as an End--End pattern (the distributions are broadly, but not totally, symmetrical):

Counts: ['ey.', 'ey.', [(1936, 289), (1981, 298), (2035, 301), (2086, 314), (2147, 325), (2202, 356), (2259, 385), (2320, 434), (2314, 434), (2266, 385), (2199, 356), (2132, 325), (2067, 314), (2013, 301), (1947, 298), (1889, 289)]]

z--scores: (('ey.', 'ey.'), [-0.8, -0.7, -0.9, -0.7, -0.6, 0.2, 0.9, 2.1, 2.2, 0.8, 0.2, -0.5, -0.6, -0.7, -0.5, -0.5])

The distribution is quite flat, ranging from about 0.15% likelihood to 0.17% in positions +/- 2-8, but then peaking at 0.187% immediately adjacent. That is, the difference in likelihood between adjacent position and the next highest is almost the same size as the total variation in the distribution.

For [ey.], [edy.], the End--End pattern has the following distribution:

Counts: ['ey.', 'edy.', [(1936, 376), (1981, 384), (2035, 376), (2086, 386), (2147, 368), (2202, 348), (2259, 364), (2320, 403), (2314, 342), (2266, 397), (2199, 359), (2132, 365), (2067, 372), (2013, 368), (1947, 362), (1889, 355)]]

z--scores: (('ey.', 'edy.'), [1.4, 1.3, 0.7, 0.7, -0.3, -1.3, -1.1, -0.2, -2.1, -0.1, -1.0, -0.4, 0.3, 0.5, 0.7, 0.9])

We can clearly see the numerical dip immediately following [ey] (the ninth entry), being 50 tokens lower than either side. That's a 1/8th decrease, and the lowest count in the distribution, which gives us a z--score of -2.1.

Neither of these would be very easy to spot otherwise, especially in a mass of data from hundreds of pairs. Moreover, it shows relationships which are not an absolute yes/no, but a nuanced preference/dispreference: words ending [ey] are more likely to be followed by another word ending [ey], and less likely to be followed by one ending [edy]. Individually I don't think these patterns tell us much, but if we can link several together into a bigger theory, then we might gain some insight.

(Koen---or anybody with the authority---can I add a bigger file to this post? About 305 kb, but all text so no risk of viruses.)