Relations among pattern studies?

[pfeaster]

I have a couple of questions about spaces, and wonder if you have any thoughts?


1. You mention the difference between Strong and Weak breakpoints, but I wonder if all breakpoints tend toward 100% in the right conditions? I mean, this should be obviously so, but I think that those conditions might be quite complex in some cases. Break points aren't strictly predictable for some parts of the pattern, which is a striking contrast to the near 100% predictability of some transitions.

It might seem self-evident that for any extended sequence of glyphs, there should be one and only one correct way of breaking it into words, and that if we aren't yet able to predict the actual spacing in some situations, that's because the conditions governing those situations are more complex than we understand.

But I suspect some breakpoints may really be "weaker" -- more ambiguous or tentative -- than others.

One piece of evidence for this involves patterns among comma breaks in the ZL transcription -- that is, cases in which a well-informed and experienced transcriber wasn't sure whether a space was present or not, so that they're marked with a comma (,) instead of a period / full stop (.).

A while back, I ran some statistics and found that the glyph pairs that were often separated by commas (reflecting transcriber uncertainty) also tended to have a greater tendency to be written inconsistently -- that is, sometimes definitely with a break, but also sometimes definitely without a break, with neither option being clearly preferred.

I suppose it's possible that the transcriber may -- consciously or unconsciously -- have factored knowledge of which glyphs do and don't typically have spaces between them into decisions about which breaks to mark as ambiguous.  But whenever I've checked the transcription against a facsimile, the judgment call seems correct.  Cases marked (.) or without a break look definite; cases marked (,) look ambiguous.

Of course, we might suppose that the presence or absence of a space between two glyphs should depend on the particular word or words the glyphs belong to.  In English, after all, there's no space in another, but there is a space in an otter.  But in the VMS, the spaced and unspaced variants don't seem to cluster as we'd expect by analogy with that type of distinction.  The same pairs of words or word parts tend to occur written definitely together, definitely apart, and uncertainly.

So that's what led me to classify these as "weak" breakpoints.  My hypothesis is that these breakpoints are secondary and optional in a way that "strong" breakpoints aren't, and that they might reflect some kind of structural division that's convenient or helpful to reflect in writing, but not essential.

It's worth noting that some vernacular European writing traditions roughly contemporaneous with the VMS were likewise consistently inconsistent about certain spacing decisions -- I've seen French, German, and Italian examples.  And in at least some of those cases, I've encountered a (potentially) similar tendency towards ambiguous spacing, which is to say, a kind of tentative half-space used in situations where spacing is otherwise particularly inconsistent.  It can be frustrating when you're trying to transcribe an old legal document and can't resolve these ambiguities by falling back on how spacing is "usually" handled.  This often seems to involve definite articles and elements that can function both as prepositions or prefixes.

2. Given this, are spaces part of the same system as the loops or something which interweaves with it?


I'm curious as sometimes the differences in token counts for words with small changes is used as evidence for the underlying system, but those are dependent on the presence of spaces. That is, [chol] and [daiin] and [choldaiin] are all different word types, but parts of the same loops.

My hypothesis is that spaces are inserted into a continuous stream of glyphs to make it easier to parse, but that they don't add any information beyond that, much like the commas in the numeral 1,234,567.  But it's just a hypothesis, and it could certainly be tested more robustly than I've tried to do.  I have not, for example, experimented to see what would happen if a space were treated as another option in transitional probability matrices.  I also haven't tried to work out or model any "rules" for spacing that extend beyond the single glyphs to either side of each potential breakpoint.  There's a lot more that could be investigated here.

In addition to strong and weak breakpoints, I suspect there may be another force at play that discourages both very short and very long words in running text, but I haven't tried to confirm that or work out any details.  (Possible example: if spacing rules would ordinarily dictate that the first word in a line should be "y," the "y" will instead be joined to the following word.)

The main case where I used discrete words as evidence was, I guess, the word counts from my stochastically generated text.  In that case, I generated the glyph sequence first and broke it into words in a second, entirely independent step (using the calculated probability of a space between each glyph pair).  What I think the results show is that a glyph sequence generated based on transitional probability matrices, chopped crudely into words at typical breakpoints, yields something reasonably close to the vocabulary and relative word frequencies of Voynichese.  That is, it seems possible that the observed word structure could arise out of this dynamic, or one similar to it.  But the reason for breaking the stream of glyphs into words here was really just to make the results compatible and comparable with word-count statistics.  I have to say I was surprised when [chedy] popped out at the top of the list -- I wasn't expecting that!