(22-05-2026, 02:45 PM)oshfdk wrote: You are not allowed to view links. Register or Login to view.I'm not sure I understand how would human production achieve this? It is trivial on a computer that can process the whole text up to the present moment, compute counts, check presence, etc. Humans can only physically look at past text and notice some words that are present there.
I think you're underestimating what a human can do and overestimating what a computer can do. As I mentioned before, I've had the sheet source theory lined out for months. It's taken that long and 3600+ lines of python code to even come close to modelling human behavior without cheating. When I missspell a word, you can spot it easily. A computer needs lookup tables and algorithms. Humans have a built-in pattern recognition systems that are not easy to put into mathematical terms.
Let's look at some of the things I've had to consider and would also have to be considered in your model.
- Current line position - Where are we on the line? Do we need a gallows token here?
- Current page position - Do we need to create a new paragraph? Does this need to be a short line to precede a paragraph on the next line?
- Selected source token - Which of those 20 words are we going to choose? Or, are we going to choose a word on the page we just wrote?
- Selected mutation type - Are we going to copy the word or mutate it? We need a probability.
- Copy vs mutate probability - What percentage of words are copied and which are modified?
- Glyph insertion/deletion weights - When a word is chosen to modify, what letter can be deleted? What letter can be inserted or swapped for another.
- Length-preservation bias - When we mutate, do we need to keep the same word length or can we shorten or lengthen it?
- Repeat-nearby-family bias - Do we need to make this word look like another word we just wrote down?
- Avoid-too-many-new-forms bias - Have we used a word and it's mutation too many times already. Do we need to find a new word?
- Proposed token - Ok, we have chosen the word and we're ready to write it.
- Rejection reason - Does the word look stupid? Does it have 4 vowels? 5 consonants? Have we repeated the word 4 times? Does it look like a possible word with vowels and consonants? Accept or reject and start over.
Now, that being said, I have looked at other models. One that often proves just as valid as my sheet model, and sometimes more so, is the previous few pages model.
Here's a table from data in an earlier attempt to locate the 'parents' of words. You can easily see that most words can be sourced from the same page. Then, just looking a few pages back, you can find sources for other words. But, I rejected this method as it would require the scribe to keep flipping sheets over to look at the back side to see what word to copy or mutate and ED2 can change 50% of the letters in a 4 letter word which seemed unrealistic to me and bordered on making up results to fit the theory. And that's when I came up with the sheet model which is much more human doable and, for the most part, uses ED1 and copy.
Total tokens on f20r: 78
| Source Window | Copy | ED1 | ED2 | Any Match |
|---|---|---|---|---|
| Same page | 11 | 25 | 25 | 61 |
| 1 page back | 3 | 4 | 5 | 12 |
| 2 pages back | 0 | 0 | 2 | 2 |
| 3 pages back | 0 | 0 | 0 | 0 |
| 4 pages back | 1 | 0 | 0 | 1 |
| 5 pages back | 0 | 0 | 1 | 1 |
| Far lookback | 0 | 1 | 0 | 1 |
So, are there other explanations and could yours be one of them? Definitely. Am I going to spend another 3 months modelling it? No offense but uh, no. I would need to see some concrete numbers before even considering it as a possibility.