28-04-2025, 12:00 PM
Approaching the question from a new perspective has been highly enriching. It is always valuable to start from a dialectical process as the engine of scientific inquiry.
Leaving behind the hypothesis of lunar phases, which might have involved a certain degree of arbitrariness, as Radio said, we have completely reformulated the analysis toward a model of thematic structure based on clustering pages, aiming to determine whether macroblocks exist. Also, we fixed the pre-processing code to clean EVA as Nab suggested.
By applying a systematic pipeline that combines thematic modeling (LDA, NMF), hierarchical clustering, and strict statistical validations (Random Forest, Permutation Test, Bootstrap, Hopkins Statistic), along with comparative controls using authentic and random texts, the results are pretty interesting.
Without forcing any parameters, the pages of the Voynich Manuscript tend to cluster spontaneously into three large macroblocks, which notably coincide with the traditional divisions that have been proposed for a long time (Botany, Zodiac/Astronomy, Pharmacology, and Recipes). Each section has different repeated words.
The macroblocks are derived automatically from the page topic distributions (LDA) combined with hierarchical clustering, and are not manually imposed.
These groupings do not appear in the control texts (a point I have taken into account very much following earlier discussions) or in synthetic random controls, and they disappear when the text is shuffled. This may support the hypothesis that there is a real and coherent internal structure within the Voynich Manuscript, likely of a thematic nature.
I maintain that the existence of blocks that vanish when permutation or ablation tests are applied indicates a structured linguistic intention.
As with any analytical process, errors are always possible, but that is the beauty of dialectical progress: advancing together as a community to explore new directions.
I will share the complete pipeline shortly.
Leaving behind the hypothesis of lunar phases, which might have involved a certain degree of arbitrariness, as Radio said, we have completely reformulated the analysis toward a model of thematic structure based on clustering pages, aiming to determine whether macroblocks exist. Also, we fixed the pre-processing code to clean EVA as Nab suggested.
By applying a systematic pipeline that combines thematic modeling (LDA, NMF), hierarchical clustering, and strict statistical validations (Random Forest, Permutation Test, Bootstrap, Hopkins Statistic), along with comparative controls using authentic and random texts, the results are pretty interesting.
Without forcing any parameters, the pages of the Voynich Manuscript tend to cluster spontaneously into three large macroblocks, which notably coincide with the traditional divisions that have been proposed for a long time (Botany, Zodiac/Astronomy, Pharmacology, and Recipes). Each section has different repeated words.
The macroblocks are derived automatically from the page topic distributions (LDA) combined with hierarchical clustering, and are not manually imposed.
These groupings do not appear in the control texts (a point I have taken into account very much following earlier discussions) or in synthetic random controls, and they disappear when the text is shuffled. This may support the hypothesis that there is a real and coherent internal structure within the Voynich Manuscript, likely of a thematic nature.
I maintain that the existence of blocks that vanish when permutation or ablation tests are applied indicates a structured linguistic intention.
As with any analytical process, errors are always possible, but that is the beauty of dialectical progress: advancing together as a community to explore new directions.
I will share the complete pipeline shortly.