28-08-2024, 09:24 AM
Presumably there are spatial clusters of related words in Anna Karenina, and they evolve in correlated ways, since it is a coherent narrative generated by a stateful author. But is the text intrinsically directional? With a transcription alone, would Venusians be able to determine whether it was written from front to back or back to front?
A more constrained copy-and-mutate process will definitely yield text with a kind of abstract statistical directionality. With reference to a single specific line of text, the universe of all possible lines contains a greater number of lines at greater edit distance than at lesser edit distance. On these grounds alone, sequential random modification of the reference line will usually produce lines at greater and greater edit distance from the original (until equilibrated). Obviously this asymmetry (Patrick's characteristic #3?) might be obscured by a human scribe's distractions, an inconvenient mutation rate, or the macro-state metric chosen to detect it. Given the very small number of word tokens per vms line, the present attempt to spot intra-page evolution of the vocabulary (Patrick's characteristic #1) uses a metric (MMED) informed by the proposed mutation process. Maybe statistical directionality could even be detected in poetry or musical notation, which are composed by humans using incomplete and not-strictly-random self-citation. A minimal positive result is shown at the end of the post.
Torstens's generated_text.txt is a transparent case of Voynichesque self-citation. Dividing it this time into 37 consecutive pages of 32 lines each, we can calculate the MMED measure with respect to the first line of the current page, or the first line of the following page:
Each point represents the average, over all pages, of a line's MMED score. On the left, this measure of edit distance evolves away from the first line as we progress down the page. On the right the trend is inverted, as expected: edit distance decreases as we approach the reference line on the following page. By construction, there is page-to-page continuity across the 1200 lines of generated text, so a reference line from the 'future' of each page still affects its MMED scores. In this test with a sample size and mutation texture that is broadly similar to the vms, directionality is observable in the co-averaged pages. (For You are not allowed to view links. Register or Login to view. I overlapped the page contents, so the absolute MMED values were not correct. The plots above are... less erroneous.)
Apparently the scatter in results for the vms can be reduced by analyzing a subset of paragraph text that is more uniform, from page to page, than the text as a whole. Quire 20 has rough unity of layout, Currier language, and hand, with many long lines per page. Now considering the first 32 lines of 23 separate Q20 pages, and analyzing exactly as above,
A trend does seem to emerge in the current-page plot on the left, with later lines again evolving away from the first. On the right, with edit distance keyed to the following page, no trend is visible through the noise. Unlike generated_text.txt, Q20 shows no evidence here of page-to-page continuity (in the canonical page order).
Curiously, the vms trend may be driven by the first line in particular. When we use the second line on the page as a reference, subsequent Q20 lines do not evolve away from it as convincingly as they did from the first:
Meanwhile generated_text.txt on the right (with its consistent line-by-line generation algorithm) behaves as it did with the first-line reference.
Nablator posted a transcription of Speculum Humanae Salvationis over in the You are not allowed to view links. Register or Login to view.. Using 85 columns from the long central chapters (with headings, capital letters, and punctuation removed), the MMED trend keyed to the first line of each column is understandable:
As with chapters of the King James Bible, no large-scale down-column trend is evident. But the value for the second line specifically is a clear outlier, with a significantly lower MMED value. Just skimming the transcription with my Venusian grasp of Latin, I see that the first couplet of each column rhymes, and often contains a word repeated or inflected between the lines. This degree of line-to-line similarity is apparently enough to show up in the plot, and indeed to account for the effect size seen in Q20.
Of course the trends are maddeningly weak. Since EVA is a graphical notation, 'edit' similarity is mostly capturing visual resemblance. And the Word As A Functional Unit is baked in by default, until a better-preforming analysis is discovered.
Thematically appropriate soundscape: Alvin Lucier, You are not allowed to view links. Register or Login to view..
A more constrained copy-and-mutate process will definitely yield text with a kind of abstract statistical directionality. With reference to a single specific line of text, the universe of all possible lines contains a greater number of lines at greater edit distance than at lesser edit distance. On these grounds alone, sequential random modification of the reference line will usually produce lines at greater and greater edit distance from the original (until equilibrated). Obviously this asymmetry (Patrick's characteristic #3?) might be obscured by a human scribe's distractions, an inconvenient mutation rate, or the macro-state metric chosen to detect it. Given the very small number of word tokens per vms line, the present attempt to spot intra-page evolution of the vocabulary (Patrick's characteristic #1) uses a metric (MMED) informed by the proposed mutation process. Maybe statistical directionality could even be detected in poetry or musical notation, which are composed by humans using incomplete and not-strictly-random self-citation. A minimal positive result is shown at the end of the post.
Torstens's generated_text.txt is a transparent case of Voynichesque self-citation. Dividing it this time into 37 consecutive pages of 32 lines each, we can calculate the MMED measure with respect to the first line of the current page, or the first line of the following page:
Each point represents the average, over all pages, of a line's MMED score. On the left, this measure of edit distance evolves away from the first line as we progress down the page. On the right the trend is inverted, as expected: edit distance decreases as we approach the reference line on the following page. By construction, there is page-to-page continuity across the 1200 lines of generated text, so a reference line from the 'future' of each page still affects its MMED scores. In this test with a sample size and mutation texture that is broadly similar to the vms, directionality is observable in the co-averaged pages. (For You are not allowed to view links. Register or Login to view. I overlapped the page contents, so the absolute MMED values were not correct. The plots above are... less erroneous.)
Apparently the scatter in results for the vms can be reduced by analyzing a subset of paragraph text that is more uniform, from page to page, than the text as a whole. Quire 20 has rough unity of layout, Currier language, and hand, with many long lines per page. Now considering the first 32 lines of 23 separate Q20 pages, and analyzing exactly as above,
A trend does seem to emerge in the current-page plot on the left, with later lines again evolving away from the first. On the right, with edit distance keyed to the following page, no trend is visible through the noise. Unlike generated_text.txt, Q20 shows no evidence here of page-to-page continuity (in the canonical page order).
Curiously, the vms trend may be driven by the first line in particular. When we use the second line on the page as a reference, subsequent Q20 lines do not evolve away from it as convincingly as they did from the first:
Meanwhile generated_text.txt on the right (with its consistent line-by-line generation algorithm) behaves as it did with the first-line reference.
Nablator posted a transcription of Speculum Humanae Salvationis over in the You are not allowed to view links. Register or Login to view.. Using 85 columns from the long central chapters (with headings, capital letters, and punctuation removed), the MMED trend keyed to the first line of each column is understandable:
As with chapters of the King James Bible, no large-scale down-column trend is evident. But the value for the second line specifically is a clear outlier, with a significantly lower MMED value. Just skimming the transcription with my Venusian grasp of Latin, I see that the first couplet of each column rhymes, and often contains a word repeated or inflected between the lines. This degree of line-to-line similarity is apparently enough to show up in the plot, and indeed to account for the effect size seen in Q20.
Of course the trends are maddeningly weak. Since EVA is a graphical notation, 'edit' similarity is mostly capturing visual resemblance. And the Word As A Functional Unit is baked in by default, until a better-preforming analysis is discovered.
Thematically appropriate soundscape: Alvin Lucier, You are not allowed to view links. Register or Login to view..
![[+]](https://www.voynich.ninja/images/collapse_collapsed.png)