[Article] A new Timm & Schinner publication regarding the Malta conference

[Torsten]

1) It is fatally methodologically flawed to do any sort of cluster analysis on 102 data points ...

Concerning the "failure on three fronts" of our "cosine distance" analysis:

1. The "curse of dimensionality" does exist, but in many cases it is a blessing rather than a curse. Several successful methods of statistical physics would not work in less than billion-dimensional space. Two vectors always define a plane, regardless of dimensionality, which, coarsely speaking, "focuses" the statistics; a situation different from simple data distribution. Unfortunately, we were not able to find any website dealing with advanced mathematical statistics of this kind in a more than superficial way, so your hat may remain uneaten. But perhaps you might consider this argument: If you were right, then how could topic modeling programs work at all? Basically, they use the same principle. And they indeed are marvelously successful tools – just not for analyzing the VMS (because there are no topics to analyze - see chapter 2 "Context-dependent self-similarity" in Timm & Schinner 2020, p. 3ff). 

Thus:
a) Topic modeling works (despite the "curse of dimensionality").
b) In the VMS it cannot even correctly identify the two Currier clusters.
c) The non-existence of separated sections (and topics) requires an explanation.


2. We also were using ed-statistics in our 2020 paper (see Timm & Schinner 2020, p. 6). However, when considering not only the Herbal A+B and Bio sections, but rather all sections, then a different picture (without dramatic jumps!) emerges for the frequencies of ed-tokens:
 

Code:

SECTION    <ed>-COUNT PERCENTAGE  TOKEN-COUNT Herbal (A)       12      0.15%       8,087 Pharma (A)       17      0.67%       2,529 Astro            28      1.31%       2,136 Cosmo           257      9.55%       2,691 Herbal (B)      528     16.33%       3,233 Recipes (B)    2073     19.42%      10,673 Bio (B)        1925     27.85%       6,911

See also the discussion by Zandbergen: "… the overall statistics demonstrate that there is a continuum, and the other (not herbal) pages actually 'bridge the gap'." (Zandbergen You are not allowed to view links. Register or Login to view.)


3. From our perspective, there are indeed no function words present in the VMS, "since a token dominating one page might be rare or missing on the next one. However, all pages containing at least some lines of text do have in common that pairs of frequently used words with high mutual similarity appear" (Timm & Schinner, 2020, p. 3). As a result, we conducted our cosine distance analysis based on our preliminary findings, and the results align with our initial assumption. What is unscientific about this? While the method is open to discussion, would you have raised concerns if our Figure 1 had indicated a change in slope?


Finally, regarding the "topic word" remark we choose to overlook the overt sarcasm. Believe it or not, we are aware of this concept in topic modeling, however we were asking about this term in the context of the VMS. Reddy and Knight just evaluate whether the words are randomly distributed: "That is, do certain words 'burst' with a high frequency within a page, or are words randomly distributed across the manuscript?" (Reddy & Knight p. 6). And Montemurro & Zanette write: "While uninformative words tend to have an approximately homogeneous (Poissonian) distribution, the most relevant words are scattered more irregularly, and their occurrences are typically clustered" (Montemurro & Zanette, p. 2). There are no uniformly distributed tokens in the VMS. So, according to this, all words in the VMS must be "topic words." Thus, we insist that we are not just "half-right", but at least "three-quarter-right".

Note, preprints of our papers are available at You are not allowed to view links. Register or Login to view.