29-06-2020, 01:27 PM
(29-06-2020, 06:40 AM)ReneZ wrote: You are not allowed to view links. Register or Login to view.In this thread, the terms 'normal' and 'binomial' are arbitrarily mixed,
No, definitely they are not mixed, and the more so "arbitrarily". From the very first post it's stated that the binomial distribution becomes close to normal with the increase in N. Which is the case with large vocabularies.
The fact is that the VMS word type frequency distribution is very close to normal. The question is then what's the root of that. Is that "natural" - which would be a point of interest for those who develop a natural language theory, or is that a result of some "manipulation" - which may, of course, include the case when the underlying mechanism produces something binomially distributed.
Yes, I've thought of the Pascal's triangle (which is no more Pascal's than the Voynich manuscript is Voynich's), but what you describe is a very interesting idea... Let's say decompose vords into shorter sustained subvords and look at their frequency distribution...
Yet all techniques, beginning with Stolfi's binary-decimal encoding and ending with suggestions in this thread imply a purely nomenclator solution. One generates a vocabulary of "invented" words, maps it to the vocabulary of his source plain text (thus creating the dictionary), and then for each word he needs to encode he refers to this dictionary. Mind he needs to include all word forms, like genders, tenses etc. This is possible, but extremely cumbersome.