03-12-2024, 07:10 AM
It looks to me like the biggest difference is in the treatment of space characters (ie. word boundaries). I was interested in creating diagrams similar to here:
You are not allowed to view links. Register or Login to view.
My identification of possible vowels is based on the figures there, as I don't have a lengthy enough text yet, normalized in Voynichese to do my own analysis with (I've done shorter analyses, but I can only reproduce the most obvious patterns, as you say, some substantial text is needed), though, I'm working on it; Going through some EVA text others have created and trying to figure out how to reasonably handle the discrepancies and uncertainties. I'm hoping to test the plausibility of some hypotheses about potential abbreviations.
When I update my handling of spaces to match what you are describing, I get the same results you are describing. (I strip everything but spaces and alphabetic characters, and I normalize text to lower case.)
If this was the procedure followed in the post at the top of this thread, then this is more similar to the non-EVA Voynich results. (I tend to think that that's more plausible as the intended character system.)
For Toki Pona I am using a concatenation of intermediate level Toki Pona texts available here:
You are not allowed to view links. Register or Login to view.
Considering how far these results are from the norm, and how much more similar they are to Voynichese, I was thinking it might be interesting to try a pidgin to see how that changes things. After all, while the broad statistics of Toki Pona look promising, the resulting diagram I get has a very different feel to it than what I get looking at English or Latin. I haven't really sat down to think through why the letter-combination frequencies, when visualized, might look so boxy and compartmentalized in Toki Pona, but Voynichese doesn't. It looks much more like a natural language from this perspective. (If I've been unclear, the diagrams I am talking about are the grids showing leading letters, and following letters, along with a color on the grid indicating how frequently they appear together.)
You are not allowed to view links. Register or Login to view.
My identification of possible vowels is based on the figures there, as I don't have a lengthy enough text yet, normalized in Voynichese to do my own analysis with (I've done shorter analyses, but I can only reproduce the most obvious patterns, as you say, some substantial text is needed), though, I'm working on it; Going through some EVA text others have created and trying to figure out how to reasonably handle the discrepancies and uncertainties. I'm hoping to test the plausibility of some hypotheses about potential abbreviations.
When I update my handling of spaces to match what you are describing, I get the same results you are describing. (I strip everything but spaces and alphabetic characters, and I normalize text to lower case.)
If this was the procedure followed in the post at the top of this thread, then this is more similar to the non-EVA Voynich results. (I tend to think that that's more plausible as the intended character system.)
For Toki Pona I am using a concatenation of intermediate level Toki Pona texts available here:
You are not allowed to view links. Register or Login to view.
Considering how far these results are from the norm, and how much more similar they are to Voynichese, I was thinking it might be interesting to try a pidgin to see how that changes things. After all, while the broad statistics of Toki Pona look promising, the resulting diagram I get has a very different feel to it than what I get looking at English or Latin. I haven't really sat down to think through why the letter-combination frequencies, when visualized, might look so boxy and compartmentalized in Toki Pona, but Voynichese doesn't. It looks much more like a natural language from this perspective. (If I've been unclear, the diagrams I am talking about are the grids showing leading letters, and following letters, along with a color on the grid indicating how frequently they appear together.)