In Support of Guido Pérez's Suggestion
#10
(07-01-2026, 11:52 AM)nablator wrote: You are not allowed to view links. Register or Login to view.I wanted to give guidoperez the benefit of the doubt. Now after checking a few things, I doubt there is anything salvageable in there. It's all either miscalculated or AI-hallucinated.

I think that the first paper is contaminated by LLM slop, and I agree that the way the statistics were computed is quite wrong in many ways, making the results meaningless.  But the idea does not seem to be LLM hallucination.

The first flaw was defining "segment" as  one line  for the VMS, then a paragraph or a recipe for the control texts.   The J index should increase with segment size (token count), so it is important that all files have segments of similar sizes.

Using the "C" transcription was another error.

Quote:The VMS doesn't have a short list of extremely common "stop words" that appear in almost every line, this is certainly true and it is the only fact worth discussing.

Indeed.  Rather than trying to compensate for those common words, it would be more informative to list the most common words in the A ∩ B sets for each text. 

All the best, --stolfi
[+] 1 user says Thank You to Jorge_Stolfi for this post
Reply


Messages In This Thread
RE: In Support of Guido Pérez's Suggestion - by Jorge_Stolfi - 07-01-2026, 02:06 PM

Forum Jump:


Users browsing this thread: 2 Guest(s)