08-06-2025, 01:21 PM
The models I trained don’t explicitly incorporate or reference slot grammars, so any patterns they exploit emerge purely from statistical regularities in the character sequences. Interestingly, the relatively low perplexity scores (especially for the full and biological sections) suggest that the models are indeed capturing consistent structures — possibly beyond what slot grammars alone describe.
Slot grammars tend to focus on local, template-like patterns (e.g. word frames like “qokeedy”), but transformer models can, in theory, learn longer-range dependencies and more abstract relationships between glyphs. Whether what they learn is beyond slot grammars is hard to prove directly.
So while I can't guarantee that the models go beyond slot grammar patterns, the approach opens the door to testing exactly that — and it’s something I’m definitely interested in pursuing further.
Slot grammars tend to focus on local, template-like patterns (e.g. word frames like “qokeedy”), but transformer models can, in theory, learn longer-range dependencies and more abstract relationships between glyphs. Whether what they learn is beyond slot grammars is hard to prove directly.
So while I can't guarantee that the models go beyond slot grammar patterns, the approach opens the door to testing exactly that — and it’s something I’m definitely interested in pursuing further.