(23-12-2025, 06:55 AM)kckluge wrote: You are not allowed to view links. Register or Login to view.So I'm very comfortable defending the utility of analyses in relative bigram frequency space.
(* If you don't get the reference, you should watch Monty Python and the Holy Grail.)
I did watch it a few times but it was long ago. I didn't get the reference, but I hope it's not essential for the discussion.
I can't really reply to your points one by one, because I don't think I fully understand many of them, I'm not so wise in the ways of science and I'm not really involved with the whole languages/scribes/hands topics. Maybe that's the reason I keep "taking past" other people in my posts, I don't know.
I can try to explain my dissatisfaction with the exploratory use of PCA and similar advanced statistical techniques without some guiding hypothesis, using a simple example.
Suppose you run PCA on some data from the manuscript and you get a curiously good separation between something and something.
1) Does this reflect a specific underlying feature of the manuscript or is this a spurious random pattern? After all, absolutely any pattern can arise purely through chance and a random pattern that clearly separates two classes can still be meaningless.
2) Does this reflect some feature of the internal design of the manuscript (intentional and will help getting useful information about the manuscript) or a by product of some external factors (say, temperature and sunlight in the scriptorium, the shoes the scribe is wearing, whether the scribe is anxious or calm, any of this affecting the writing patterns, which won't tell us anything about the content)?
In most STEM studies it would be easy to handle the question of "spurious or real": you just take another sample of data, another preparation and try reproducing. So, if we were working with a natural phenomenon, we would just take 10 more of those Voynich Manuscripts, we would repeat the study on them, we would find out that, say, 8 of 10 show the same pattern in this experiment and we'll call this result significant. Maybe even later someone would investigate the two Voynich Manuscripts that failed the test, and would find out that one of them is a XX century forgery and another was soaked in water and poorly retraced, turning our original study into the pinnacle of reproducibility.
However, there is only one Voynich Manuscript and with the higher and higher number of tests performed it is expected that by change there will be more and more curiously interesting results which don't reflect any real features of the manuscript. And this makes each of these results less and less convincing.
I'm not sure how to address the question of intentional or a by-product, but I think it requires some model of what the manuscript is and how and why it was created. And I think this model should be created before running the tests and the tests should be designed to serve as the evidence for or against the model. Retrofitting the model after the test results are known is pointless, since the space of all possible models is huge. You can invent many conflicting scenarios that would explain any distribution of the data.
Now why I singled out PCA specifically: it provides visual patterns. Human brains are hardwired to extract and boost and extrapolate visual patterns. So, on the top of the possibility that the original result is spurious or a by product we add the possibility of visually misinterpreting the significance of the result. And now if I'm following the thread correctly, there is the idea that 2D PCA is not enough and 3D is the way to go, which would further blur the significance of the result.
I think the reason why there are so few interesting and (more or less) universally accepted statistical results for the manuscript is because these results better be
1) simply presented (mono dimensional or scalar);
2) unimodal (do not compare different domains open to interpretations - like texts and images);
3) using relatively well defined feature of the manuscript and a large sample size.
An example ticking all of the above would be Stolfi's binomial distribution of the word and token lengths, which seems useful even without some model explaining it (I don't remember if the Chinese hypothesis came to be before this result or after).
I'm not sure whether the above is coherent, but this is my view of the matter. Overall, I don't mind reading exploratory statistical studies if there is some clear understanding of their essential limitations when dealing with a single immutable object of study.