The oddities of the bigram "ed"
#15
(16-02-2026, 01:07 AM)Fontanellean wrote: You are not allowed to view links. Register or Login to view.
(16-02-2026, 12:29 AM)Dunsel wrote: You are not allowed to view links. Register or Login to view.I understand what you're saying about bigram statistics being driven by common words and subject matter. That absolutely happens in natural languages. Your examples make sense in that context.  But what I'm seeing here doesn’t look like normal topic drift.  I'm not just saying that “ed is frequent.”  I'm saying that a very high-frequency bigram simply disappears on many pages, and then appears heavily on others, within and across topics. That kind of page-level on/off behavior is what caught my attention.

If there’s a natural-language example where a very high-frequency digraph shows this kind of behavior, I’d genuinely be interested to see it. It would help clarify whether what we’re seeing here is unusual or not.

You don't even have to look at bigrams. Take an instruction manual with one page written in British English and another in Pinyin for Mandarin. "Z" will hardly appear at all on the former but will be all over the place in the latter.  Smile

Well, I suppose that's one explanation.
Reply


Messages In This Thread
The oddities of the bigram "ed" - by Dunsel - 15-02-2026, 03:04 AM
RE: The oddities of the bigram "ed" - by ReneZ - 15-02-2026, 04:09 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 12:01 AM
RE: The oddities of the bigram "ed" - by eggyk - 15-02-2026, 11:02 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 12:29 AM
RE: The oddities of the bigram "ed" - by oshfdk - 16-02-2026, 01:11 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 01:15 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 03:55 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 11:27 PM
RE: The oddities of the bigram "ed" - by Dunsel - 18-02-2026, 12:46 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 05:27 AM
RE: The oddities of the bigram "ed" - by ReneZ - 15-02-2026, 09:52 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 12:10 AM
RE: The oddities of the bigram "ed" - by oshfdk - 16-02-2026, 12:37 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 01:02 AM
RE: The oddities of the bigram "ed" - by oshfdk - 16-02-2026, 01:07 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 01:27 AM
RE: The oddities of the bigram "ed" - by Rafal - 16-02-2026, 12:46 AM
RE: The oddities of the bigram "ed" - by nablator - 17-02-2026, 10:37 AM
RE: The oddities of the bigram "ed" - by oshfdk - 16-02-2026, 02:16 AM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 05:40 AM
RE: The oddities of the bigram "ed" - by davidma - 16-02-2026, 11:39 AM
RE: The oddities of the bigram "ed" - by Skoove - 16-02-2026, 11:48 AM
RE: The oddities of the bigram "ed" - by ReneZ - 16-02-2026, 12:46 PM
RE: The oddities of the bigram "ed" - by Dunsel - 16-02-2026, 11:41 PM
RE: The oddities of the bigram "ed" - by Dunsel - 17-02-2026, 02:10 AM
RE: The oddities of the bigram "ed" - by Koen G - 17-02-2026, 12:39 AM
RE: The oddities of the bigram "ed" - by Dunsel - 17-02-2026, 01:50 AM
RE: The oddities of the bigram "ed" - by ReneZ - 17-02-2026, 07:32 AM
RE: The oddities of the bigram "ed" - by Dunsel - 18-02-2026, 12:41 AM
RE: The oddities of the bigram "ed" - by ReneZ - 17-02-2026, 12:16 PM
RE: The oddities of the bigram "ed" - by Dunsel - 18-02-2026, 12:51 AM
RE: The oddities of the bigram "ed" - by nablator - 17-02-2026, 09:00 PM
RE: The oddities of the bigram "ed" - by ReneZ - 18-02-2026, 12:54 AM
RE: The oddities of the bigram "ed" - by Dunsel - 18-02-2026, 02:20 AM
RE: The oddities of the bigram "ed" - by Dunsel - 19-02-2026, 01:16 AM
RE: The oddities of the bigram "ed" - by Dunsel - 17-03-2026, 05:54 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  The oddities of the bigram "ed" pt. 4 : The Chunking of Scribes Dunsel 13 3,390 29-05-2026, 06:27 PM
Last Post: Dunsel
  The oddities of the bigram "ed" pt. 3 : It's not just "ed" Dunsel 18 5,680 20-02-2026, 08:43 PM
Last Post: Jorge_Stolfi
  The oddities of the bigram "ed" pt. 2: Same as it ever was. Dunsel 16 5,648 18-02-2026, 06:45 PM
Last Post: Jorge_Stolfi

Forum Jump:


Users browsing this thread: 2 Guest(s)