1 hour ago
(This post was last modified: 1 hour ago by Jorge_Stolfi.)
(Yesterday, 04:06 PM)npcompl33t wrote: You are not allowed to view links. Register or Login to view.If many dimensions move together, there is a real underlying factor.The use of the word "move" is already biased towards the interpretation of that "underlying factor" being time.
In the book example, height, width, volume, weight, number of words, number of pages, number of letters per page etc are all correlated variables. But the ordering of books by any of those attributes is not significant. It has nothing to do with the order in which they were written, printed, bought, or read -- which seems to be implied if one says that those variables "move together".
You may have seen You are not allowed to view links. Register or Login to view.. It shows that height and width of old manuscripts (at least, those in Rene's dataset) are indeed strongly correlated, so that the points (h,w) fall very close to a straight line. I see several clusters of points (marked), but Rene apparently sees a continuum of sizes instead. Either way, while the correlation is significant (and suggests some theories about the Medieval making and marketing of vellum), the ordering of the manuscripts along that line obviously is not significant.
Quote:[Rule reversal] doesn't suggest the ordering [by statistics] is not meaningful. Reversals are exactly what you would expect with changes that are random drifts.
But one would expect the changes to be "random" only if one expects the text to be gibberish. I guess that this is your preferred theory, yes? Well, it is not mine. Even before my recent discoveries about the Stars section, my probability for "gibberish" was already close to zero. So that may be a point where we are bound to permanently disagree.
On the other hand, if the text is meaningful, and one is considering the possibility that the statistical changes may be due to "spelling reforms" or to the Author's word preferences gradually changing due to mere passage of time, one should not expect those changes to be reversed. They would not be "random".
In fact, even if one subscribes to the "gibebrish" theory, and thus expects the shifts to be random, one should not see several idependent rules being reversed at the same time.
Quote:Herbal A and B are the same topic, format, and style, and yet are significantly different. This was the same difference you and I had discussed and was the motivation behind this analysis.The illustrations strongly indicate that Herbal A and B are about the same general topic (herbalism), and we can see that the text format is basically the same (mostly 1-3 middle-size parags per page).
But we don't know whether they are in the same style. Maybe one is in the conditional mood while the other is imperative or descriptive. Maybe one has harvesting or dosage information that is lacking in the other. Maybe one notes the effects of each plant on life extension while the other seems more preoccupied with digestive problems, or with health problems specific to women.
Anyway, the massive difference in word usage between those two sets of pages, compared to the relative similarity between pages within each set, makes it necessary to treat Herbal-A and Herbal-B as separate sections. Whereas the additional proposed splits (like Rene's) do not seem to be sufficiently justified.
Even if you see differences between (say) "early Herbal-A" and "late Herbal-A", they may have explanations that do not justify treating those sub-sets separately.
For example, a certain old book (whose name shall not be uttered here) has a large number of short parags, each describing a remedy and its applications or benefits. In the oldest copies of that book, the remedies are grouped into three major sections by toxicity. First come the "High grade" remedies that are essentially tonics that can be taken for long periods without harmful effects. Then there are "Medium grade" remedies that should be taken only to cure specific problems and only until the problems are resolved. Then there are the "Low grade" ones which are quite toxic and should be taken only as a last resort, in the smallest possible doses and under close surveillance by the physician.
The entries have all similar format, language, and semantic structure, and style. But most "High grade" ones have the words "long term use", and they list mostly common benefits like long life, strength, intelligence, good complexion, virility, milk production, fertility, etc. The Medium grade entries hardly have that "long term use", except to warn about side effects like hallucinations or change of mood. On the other hand they list many more diseases that cannot be treated with the tonics of the first section. Then the Low grade entries list more serious conditions and may have extra warnings about toxicity and side effects.
However, other copies of that same book have the entries sorted instead by nature -- Mineral, Vegetable, and Animal. And each of these sub-sections will have its own set of specific words, like "root", "milk", etc.
And maybe there are copies out there in still another order -- like alphabetic, by governing planet, by geographic origin, whatever.
Anyway, the statistical differences between those sub-sections seem to be small, involving maybe 50-100 words out of the 1000 lexemes used in the book. They are dwarfed by the word usage differences between that book and any other book, even on the same topic.
The point of this example is that splitting that book into sub-sections based on word frequencies (or the character and slot pair frequencies that are determined by them) would be more confusing than illuminating. If the book was in code or an unknown language, such splitting would not help deciphering it; it would instead distract from other more effective analyses.
Which I think is the case for VMS sections like Herbal-A or Stars: at the present stage, I do not see the justification for splitting them into sub-sections.
Incidentally, the example of that book also brings up another important point: when comparing the VMS with other "control" books, one should try to use books that are likely to have similar style and text structure (visible and semantic). Thus one should compare Herbal-A to other herbals like the You are not allowed to view links. Register or Login to view., rather than to the Vulgate or to Oresme's treatise.
All the best, --stolfi

