07-07-2026, 01:02 AM
@asteckley
first of all, please accept my apologies for mixing the reply to you and to @Mauro in my previous post. I want to thank you for your critical analysis, which prompted me to reopen the data and subject the analysis to much stricter stress tests. I discovered that you were absolutely right on several methodological critiques regarding the way I presented the data, even though the underlying phenomenon seems to robustly survive classical statistical testing.
To answer your first question about accuracy…I had not clearly defined the target. The algorithm was set up as a binary classification at the page level—that is, predicting whether the page will contain at least one doublet or not, based on the rest of the vocabulary on that page. The accuracy in cross-validation is 60.7%. I completely agree with you on the weakness of the Y-Scrambling test because in a text structured by topics like the VMS, scrambling the labels makes it obvious that the real data will result in an extreme outlier. This demonstrates a non-random correlation (p-value 0.049), but it is certainly not enough to define such a correlation as a true "syntactic rule".
I also agree with your observation regarding the Bar Chart and t-SNE. The bar chart shows a continuous probability gradient, and describing them as "two separate subsets" was an oversimplification in my interpretation. Similarly, the t-SNE map, as you noted, groups pages based on overall vocabulary similarity, but it does not prove any direct causal link with doublets…so…in future revisions of the paper, the use of these tools will be scaled back to a purely exploratory role.
However, I welcomed with great interest your methodological suggestion: to verify whether the triggers exist using exclusively classical statistics.
I therefore analyzed the entire vocabulary using Fisher's Exact Test. To avoid the multiple comparisons problem and the appearance of false positives in the vast VMS vocabulary, I applied a rigorous Bonferroni Correction.
By testing the 450 main words of the manuscript (those present in at least 10 pages), the significance threshold (alpha) to consider a word a potential "trigger" dropped from 0.05 to 0.00011. In the absence of a real correlation, the probability of finding false positives below this extreme threshold is almost zero. Instead, the analysis isolated 28 different words that overcome this statistical barrier.
The raw data based on Takahashi's transcription (filtering out punctuation) show a rather clear trend:
qoteedy: Appears in 38 pages.
Of these, 38 contain at least one doublet (100%).
In pages without qoteedy, the probability of finding doublets drops to 52.7%. (P-Value < 0.000001).
olkeey: Appears in 26 pages. 26 contain doublets (100%) against a baseline of 55.6%. (P-Value 0.000001).
(Incidentally, the main triggers present p-values so infinitesimal — e.g. < 0.000001 — that they would survive the Bonferroni Correction even if it were calculated on the entire unfiltered vocabulary of 8,000 words).
To rule out that this correlation is a statistical artifact, I subjected these results to five specific "stress tests".
Syntactic Proximity: This could have been a random co-occurrence on the same page. By measuring the physical distance, the median distance between the main triggers and the generated doublet is only 24-25 words. The trigger and the doublet are systematically found within the same paragraph, suggesting a strongly local dependency.
Dialect Control (Simpson's Paradox): I hypothesized that the triggers were simply vocabulary words of "Dialect B", and that scribe B naturally had a tendency to write more doublets. I therefore isolated the analysis exclusively to Dialect B pages. The result does not change: within Dialect B alone, the absence of qoteedy generates doublets in 51% of cases, while its presence brings doublets to 100%. The correlation occurs independently of the scribe's overall style.
Geographical Distribution: You rightly noted that there are anomalous folios with extreme densities of doublets (e.g., f108). I verified the spatial distribution of the triggers to ensure they were not confined there. The word qoteedy is found 84.2% of the time outside the anomalous folios. The rule is not a geographical "glitch" localized to a few pages.
Typology of Triggered Doublets: I verified what is being duplicated. The absolute most frequent doublet in the entire manuscript is daiin daiin. However, when qoteedy is present, the daiin daiin doublet is almost totally suppressed. In its place, the trigger forces the duplication of very different words or, frequently, of isolated bigrams like ol ol and ar ar. (Considering the hypothesis that 'ol' and 'ar' may represent counting sequences, this could imply an "operative" function for these words, though I do not want to go too far into interpretations).
Independent Validation (Cross-Transcription): To avert the risk that the correlation depended on a human error by Takahashi in transcribing ligatures, I repeated the Fisher Test by downloading the ZL3b transcription by René Zandbergen. Even in this file, the main triggers survive clearly: olkeey and olchedy remain predictive at 100%, while qoteedy settles at 97.5%.
Your critique was invaluable in making me recalibrate my terms. Defining an overall accuracy of 61% (from the ML model) as an "iron rule" was methodologically reckless. However, by isolating the data with univariate statistics, I believe it clearly emerges that there is a subset of vocabulary that radically—and syntactically locally—alters the probability of observing specific duplicated structures.
I thank you again for pushing me to use more rigorous verification tools. I would be glad to know your opinion on these proximity and distribution analyses, and what other tests I could possibly run next.
Best regards,
Alfredo
first of all, please accept my apologies for mixing the reply to you and to @Mauro in my previous post. I want to thank you for your critical analysis, which prompted me to reopen the data and subject the analysis to much stricter stress tests. I discovered that you were absolutely right on several methodological critiques regarding the way I presented the data, even though the underlying phenomenon seems to robustly survive classical statistical testing.
To answer your first question about accuracy…I had not clearly defined the target. The algorithm was set up as a binary classification at the page level—that is, predicting whether the page will contain at least one doublet or not, based on the rest of the vocabulary on that page. The accuracy in cross-validation is 60.7%. I completely agree with you on the weakness of the Y-Scrambling test because in a text structured by topics like the VMS, scrambling the labels makes it obvious that the real data will result in an extreme outlier. This demonstrates a non-random correlation (p-value 0.049), but it is certainly not enough to define such a correlation as a true "syntactic rule".
I also agree with your observation regarding the Bar Chart and t-SNE. The bar chart shows a continuous probability gradient, and describing them as "two separate subsets" was an oversimplification in my interpretation. Similarly, the t-SNE map, as you noted, groups pages based on overall vocabulary similarity, but it does not prove any direct causal link with doublets…so…in future revisions of the paper, the use of these tools will be scaled back to a purely exploratory role.
However, I welcomed with great interest your methodological suggestion: to verify whether the triggers exist using exclusively classical statistics.
I therefore analyzed the entire vocabulary using Fisher's Exact Test. To avoid the multiple comparisons problem and the appearance of false positives in the vast VMS vocabulary, I applied a rigorous Bonferroni Correction.
By testing the 450 main words of the manuscript (those present in at least 10 pages), the significance threshold (alpha) to consider a word a potential "trigger" dropped from 0.05 to 0.00011. In the absence of a real correlation, the probability of finding false positives below this extreme threshold is almost zero. Instead, the analysis isolated 28 different words that overcome this statistical barrier.
The raw data based on Takahashi's transcription (filtering out punctuation) show a rather clear trend:
qoteedy: Appears in 38 pages.
Of these, 38 contain at least one doublet (100%).
In pages without qoteedy, the probability of finding doublets drops to 52.7%. (P-Value < 0.000001).
olkeey: Appears in 26 pages. 26 contain doublets (100%) against a baseline of 55.6%. (P-Value 0.000001).
(Incidentally, the main triggers present p-values so infinitesimal — e.g. < 0.000001 — that they would survive the Bonferroni Correction even if it were calculated on the entire unfiltered vocabulary of 8,000 words).
To rule out that this correlation is a statistical artifact, I subjected these results to five specific "stress tests".
Syntactic Proximity: This could have been a random co-occurrence on the same page. By measuring the physical distance, the median distance between the main triggers and the generated doublet is only 24-25 words. The trigger and the doublet are systematically found within the same paragraph, suggesting a strongly local dependency.
Dialect Control (Simpson's Paradox): I hypothesized that the triggers were simply vocabulary words of "Dialect B", and that scribe B naturally had a tendency to write more doublets. I therefore isolated the analysis exclusively to Dialect B pages. The result does not change: within Dialect B alone, the absence of qoteedy generates doublets in 51% of cases, while its presence brings doublets to 100%. The correlation occurs independently of the scribe's overall style.
Geographical Distribution: You rightly noted that there are anomalous folios with extreme densities of doublets (e.g., f108). I verified the spatial distribution of the triggers to ensure they were not confined there. The word qoteedy is found 84.2% of the time outside the anomalous folios. The rule is not a geographical "glitch" localized to a few pages.
Typology of Triggered Doublets: I verified what is being duplicated. The absolute most frequent doublet in the entire manuscript is daiin daiin. However, when qoteedy is present, the daiin daiin doublet is almost totally suppressed. In its place, the trigger forces the duplication of very different words or, frequently, of isolated bigrams like ol ol and ar ar. (Considering the hypothesis that 'ol' and 'ar' may represent counting sequences, this could imply an "operative" function for these words, though I do not want to go too far into interpretations).
Independent Validation (Cross-Transcription): To avert the risk that the correlation depended on a human error by Takahashi in transcribing ligatures, I repeated the Fisher Test by downloading the ZL3b transcription by René Zandbergen. Even in this file, the main triggers survive clearly: olkeey and olchedy remain predictive at 100%, while qoteedy settles at 97.5%.
Your critique was invaluable in making me recalibrate my terms. Defining an overall accuracy of 61% (from the ML model) as an "iron rule" was methodologically reckless. However, by isolating the data with univariate statistics, I believe it clearly emerges that there is a subset of vocabulary that radically—and syntactically locally—alters the probability of observing specific duplicated structures.
I thank you again for pushing me to use more rigorous verification tools. I would be glad to know your opinion on these proximity and distribution analyses, and what other tests I could possibly run next.
Best regards,
Alfredo