• Options
• Login
• Desktop Style

More rigorously testing the hoax hypothesis

• Index

  • The Voynich Ninja
  • Voynich Research
  • Analysis of the text
  • More rigorously testing the hoax hypothesis
• More rigorously testing the hoax hypothesis

• RE: More rigorously testing the hoax hypothesis

  Aga Tentakulus > 16-01-2023, 01:30 AM

  I wonder when people talk about fantasy words or just hoax words. Could one then transform the words back into normal words of a language, and how many would be necessary? Or is a jumble of meaningful words enough here?
  
  Where is the limit?

• RE: More rigorously testing the hoax hypothesis

  MarcoP > 16-01-2023, 01:10 PM

  (15-01-2023, 04:56 AM)ReneZ Wrote: You are not allowed to view links. Register or Login to view.Hi Marco, thanks for this. I remember your earlier post, but had not thought of it in this context.
  
  Let me try to understand.
  In my two example texts (one derived from a meaningful plain text, the other from a fully scrambled text), the first step: finding likely function words, would lead to exactly the same result. The fact that the second is meaningless is not detected, and that is due to the fact that it still has some 'meaning' hidden deeply inside it.
  
  It then depends on the next step: clustering word types, whether this meaninglessness can be detected. This would require that the metod is taking the distance between words into account. If it does not, it will still consider the scambled text just as meaningfull as the original one. I don't know the answer to this.

  
  Hi Rene,
  from what I understand of the paper, things are exactly as you say. There is a general problem that (like e.g. Sukhotin's algorithm) Smith and Witten's method was not designed to output a reliability measure, but in my opinion this is something that can be easily added to it.
  
  The first step (identifying function words based on frequency) requires using multiple texts, since the intersection of the most frequent words from multiple sources is much more reliable (some content words will be frequent because of the specific context, e.g. "plant" in a herbal). If one considers the different Voynich sections as individual texts, the small intersection of the 1% most frequent word types could be sufficient from the early failure of the method.
  
  Coming to the scrambled vs non-scrambled example, as you say, problems show in the following step. Function words are further categorized according to the intersection between the sets of different words that can immediately follow (this is actually similar to  Sukhotin's algorithm).
  
  

  Smith and Witten Wrote:The relative size of the intersection of the first-order successors of two function words is a measure of how often the words are used in similar syntactic structures. Where two closed-class words share an unusually common structural usage, we assume that they are functionally similar.

  
  The significance of the intersection is computed by comparing actual counts with expected counts "under the assumption of random sampling". In a randomly scrambled text, all variation will be due to statistical fluctuations and no significant clustering will be possible.
  
  Though the method described in the paper is very simple (and certainly not robust), one could argue that this line of reasoning was expanded in continuous-bag-of-words approaches, where the N words immediately preceding and following (instead of the single following word) are used for modelling word types. In combination with neural networks, this led to Word2Vec and ultimately contributed to recent AIs like GPT.
  
  

  (15-01-2023, 04:56 AM)ReneZ Wrote: You are not allowed to view links. Register or Login to view.With respect to MATTR, the scrambling should be clearly visible in the result. Ideally the curve should be flat with some random noise on top. However, a "non-flat" MATTR does not indicate meaning, of course.
  
  I really can't remember if this was tested at the time when MATTR was discussed here.
  
  What the experiments presented at the conference show is that human-generated meaningless text does not appear random. This is not unexpected. Any test for 'meaning' should be able to distinguish human-generated meaningless text from computer-generated random text.

  
  I don't think MATTR on randomly scrambled text has been discussed much. That looks like an interesting line of investigation. A while ago, I posted You are not allowed to view links. Register or Login to view. about the effects of scrambling on full and partial reduplication.
  This showed that scrambling increases the rate of reduplication in linguistic texts, but lowers it in the Voynich manuscript.
  
  Another possibly related measure was discussed discussed in Cárdenas et. 2016, as mentioned You are not allowed to view links. Register or Login to view..

• RE: More rigorously testing the hoax hypothesis

  Koen G > 16-01-2023, 01:35 PM

  (16-01-2023, 01:10 PM)MarcoP Wrote: You are not allowed to view links. Register or Login to view.This showed that scrambling increases the rate of reduplication in linguistic texts, but lowers it in the Voynich manuscript.

  
  The fact that scrambling reduces the rate of reduplication in Voynich text seems inevitable. It has a high variety of word types, as seen in its high mid-to-large window MATTR, but an unusually low small-window MATTR. So if you scramble the words, nothing of this small-window behavior will be left. Since the overall word type variation is on the mid-high end, there won't be much coincidental reduplication when the text is scrambled.
  
  The experiment I would like to do is the following:
  
  * Texts to include: a number of different languages and different text types. EVA transliterations. A number of large nonsense texts written by people.
  * Make a scrambled version of each text.
  * Compare MATTR at various intervals: 5, 10, 20, 50, 100, 200, 500, 1000, 2000, 5000... Maybe by dividing original MATTR by scrambled MATTR. The higher this number is, the more clustered vocabulary is within that window.
  * See how Voynichese compares to regular texts and to nonsense texts. For example by plotting them on a logarithmic scale.
  
  Again, the missing ingredient here is lengthy nonsense texts. It would be interesing to see how they behave over longer stretches, and if their MATTR drops off at the same rate as in regular texts.

• RE: More rigorously testing the hoax hypothesis

  MichelleL11 > 16-01-2023, 02:29 PM

  Another text to include, if possible, would be text produced using Torsten Timm’s algorithm to see how it holds up under this kind of analysis.

• RE: More rigorously testing the hoax hypothesis

  degaskell > 16-01-2023, 05:37 PM

  (16-01-2023, 02:29 PM)MichelleL11 Wrote: You are not allowed to view links. Register or Login to view.Another text to include, if possible, would be text produced using Torsten Timm’s algorithm to see how it holds up under this kind of analysis.

  I would go further and suggest using multiple texts produced by a range of variations on Torsten's algorithm, in order to determine what kinds of structure can and can't be produced by self-citation as a class of methods rather than just what structure is present in a single document generated by self-citation.

• RE: More rigorously testing the hoax hypothesis

  MarcoP > 16-01-2023, 07:01 PM

  I just read the two Malta papers by Gaskell and Bowern: they are excellent, not only for the corpora of cipher systems and human-generated gibberish they discuss, but also for the extensive set of quantitative measures applied when comparing with Voynichese.
  
  This is a passage from "Gibberish after all?" (emphasis mine)
  
  

  Gaskell and Bowern Wrote:A more significant limitation of this work is that, because of the short length of our text samples, we are unable to test whether gibberish can replicate the larger structural features (such as “topic words”) which have been observed in the VMS [5–7]. At present, these features pose a serious challenge to proponents of the hoax hypothesis. However, while it is premature to assume that gibberish can replicate these features, it is equally premature to assume that it cannot; in theory, the properties of a scribe’s gibberish might drift considerably over the course of the weeks or months required to generate a VMSlength manuscript, introducing significant large-scale nonrandomness. If the scribe took breaks between sections, or only kept out material from the current section to reference when copying vocabulary, further spatial patterns might arise. 3 Insofar as possible, our results appear consistent with this hypothesis.

  
  In my opinion, the way the vocabulary changes is at least as much of a problem for the idea of a meaningful text. As I already said in this thread, the degree of variation in lexicon is not compatible with the simple shift in topics. If one assumes that there is a one-to-one correspondence between Voynichese and plain-text (something like a nomenclator or Rene's thought experiment discussed above), the most frequent words should not change much with different topics. If there is no such correspondence, direct word-basis topic analysis does not make sense.
  
  Also, I don't think I spotted anything in the other paper (Enciphered after all?) that could suggest that this drift is the effect of a cipher mechanism: I guess that one could resort to nulls or some ambiguous system where the scribes could make arbitrary choices according to preferences that varied during the production of the text.
  
  Sadly, my first impression is that these papers shift the balance in favour of meaninglessness. Here is another passage from the paper:
  
  

  Quote: Compared to meaningful texts, gibberish has lower mean information content (compression); lower mean conditional character entropy (entropy); higher mean occurrences of repeated characters and words (repeated_chars, repeated_words); higher mean bias in where characters appear in a line (charbias_mean) and where words appear in a 200-word section (wordbias_mean); higher mean autocorrelation of word lengths (wordlen_autocorr; see below)

  
  The overlap of the features of gibberish with Voynichese is rather striking: low conditional character entropy, repeated words, bias for where characters appear in a line (line effects).

• RE: More rigorously testing the hoax hypothesis

  Aga Tentakulus > 16-01-2023, 07:18 PM

  I have now read and understood everything. But I don't understand what you're getting at.
  I assume that Lisa Fagin Davis is right in her assumption that there are 5 writers. It doesn't matter whether there are 4 or 6. I will assume that there are 5 writers.
  
  I will now take 5 texts with 5 different languages. Example:  Norwegian, Albanian, Turkish, French and Spanish.
  As long as I don't understand any language, they are all the same. They tell me nothing and therefore have no meaning for me.
  I can't establish a connection between the languages either.
  
  But in the VM there is a connection. The same words are used again and again, regardless of the hand.
  Since everyone uses the same thing over and over again, one can speak of a system, even if one doesn't understand it.
  
  Now I ask myself again, what do you actually want to prove with the algorithms and programmes? Something that is actually clear when you think about it.
  
  Translated with You are not allowed to view links. Register or Login to view. (free version)
• Next Oldest Next Newest