04-03-2016, 07:17 PM
(04-03-2016, 02:05 AM)Torsten wrote: You are not allowed to view links. Register or Login to view.Quote:The key question is, what verifiable features does it predict that we don't yet know about?
For me this key feature was that rare words do co-occur with similar ones. You can check this yourself. Choose a rule for selecting some low frequent types and check if this words do co-occur with similar ones.
For instance glyphs beside 'i' and 'e' occur rarely duplicated. The bigram 'll' occurs 28 times and the bigram 'dd' occurs 23 times (see You are not allowed to view links. Register or Login to view.). If you check this words you will probably find patterns like the three 'dd' words on page You are not allowed to view links. Register or Login to view. (see You are not allowed to view links. Register or Login to view.).
I half-agree. The occurrence of 'dd' and 'll are suspicious in You are not allowed to view links. Register or Login to view. and You are not allowed to view links. Register or Login to view. respectively, but in other cases it's subject to interpretation.
The occurrence of non-terminal 'n' could be taken as another example:
You are not allowed to view links. Register or Login to view.
In the text produced by your generator, what's the ratio between vocabulary size and text size?
The VM has a ratio of 0.21. The A folios have a ratio of 0.3 and the B folios have a ratio of 0.21, so there is some internal variation.
The following are the ratios for sample texts in known languages:
Pliny's Natural History (Latin): 0.27
Bible (Latin): 0.22
Bible (Hebrew): 0.22
Dante (Italian): 0.22
Moby Dick (English): 0.11
Short Stories (Pinyin): 0.21
I expect that you should be able to match a ratio of 0.21 by constraining the copy operation. The question is, how much?

