What's confusing about this VBM method is that the metrics do vary significantly depending on the underlying text. This actually fits quite well with the VMS, because those metrics also vary significantly in some cases depending on the section.
I've prepared a graph here. (Note: there's a lot of AI involved in this. The VMS-side numbers should bei correct, but the generator-side numbers I haven't cross-validated - that would have been too much work. But the point isn't the exact numbers anyway; it's how much this VBM structure is influenced by the underlying text. I had tested this with several generators — the exact numbers vary, but the dependence on the underlying text remains strong in the same areas.)
So I ran my test corpus of German texts through the current, "frozen" version of the VBM generator. All rows are sampled to the same window size (that of the smallest section), which is why the section values differ from the usual full-section figures - it's the only way to make the columns directly comparable.
You can see, for example, how the hapax rate fluctuates between 17.1 and 25.6 percent based solely on the source text. Yule's K (vocabulary concentration) also fluctuates between 39.0 and 57.6, while h2 and h3 remain roughly the same.
I just wanted to illustrate that roughly. As I said, the exact numbers should be taken with a grain of salt. But the general trend is clearly visible across my others generators as well. too.
I've prepared a graph here. (Note: there's a lot of AI involved in this. The VMS-side numbers should bei correct, but the generator-side numbers I haven't cross-validated - that would have been too much work. But the point isn't the exact numbers anyway; it's how much this VBM structure is influenced by the underlying text. I had tested this with several generators — the exact numbers vary, but the dependence on the underlying text remains strong in the same areas.)
So I ran my test corpus of German texts through the current, "frozen" version of the VBM generator. All rows are sampled to the same window size (that of the smallest section), which is why the section values differ from the usual full-section figures - it's the only way to make the columns directly comparable.
You can see, for example, how the hapax rate fluctuates between 17.1 and 25.6 percent based solely on the source text. Yule's K (vocabulary concentration) also fluctuates between 39.0 and 57.6, while h2 and h3 remain roughly the same.
I just wanted to illustrate that roughly. As I said, the exact numbers should be taken with a grain of salt. But the general trend is clearly visible across my others generators as well. too.