A mathematical approach to double words, conditional logic and the missing pages

[asteckley]

I read your comments regarding the physical permutation test. In reality, setting up the analysis with those rules would result in a structural error... applying a physical shuffle to a Bag-of-Words model triggers a macroscopic homogenization bias, and it is obvious to expect results that would artificially favor the algorithm even more.

I'm not really sure what rules you are referring to since I didn't really present any rules (and I don't see that Mauro did either) nor did I suggest to do anything like this latest processing that you described.

In any case, none of this can be commented on when you haven't defined what you are using for "accuracy". And the whole approach for your randomized data testing (and the use of Y-Scrambling for the current purposes) remains -- to me-- questionable.

I’ll just recap my assessment of all this so far:

• You have said that you found two sets of words that strongly correlate with word doublets: “triggers,” which have a positive correlation, and “inhibitors,” which have a negative correlation. You have also suggested that these two sets effectively form two identifiable subsets or clusters within the overall lexicon. But I have not yet seen evidence that demonstrates this.
  
• Your bar chart appears to show only a simple, continuous trend of coefficients across the full set of words. It does not seem to show two distinct groups or clusters. Likewise, your cluster plot does not appear to support the claim about triggers and inhibitors, and it is not clear what data the plot is actually presenting.
  
• Your observations about the t-SNE results—and the extent to which they show some clustering—suggest only that there is some kind of structure in the data. That, by itself, is not surprising. But the nature of that structure cannot really be identified by simply applying t-SNE. At most, the result shows what a black-box dimensional reduction looks like. And the visualization you showed does not clearly indicate any relationship to word doublets, at least not one that you have explained.
  
• So far, I do not see anything that connects those dimensional-reduction results to word doublets, or really to any other specific attribute of the text—unless there is some special processing of the data that you have not described.
  
• Your randomization tests also remain suspicious to me. What you have explained about them so far has not resolved that concern. It is still unclear what exactly you are measuring as “accuracy.” So far, your explanations of the randomization tests --and the expanded analysis you last described-- seem consistent with the expectation that the original-data case would be an extreme outlier in the distribution shown in the violin plot.
  
• The only clear discovery you have presented—which, if correct, is indeed a surprising one—is that the density of word doublets is four to five times greater on five particular folio pages, and that those pages may also be unusual in their semantic position within the overall book. But there does not appear to be any role for your proposed trigger and inhibitor word clusters in that particular result.
  
• Confirming and analyzing the spikes in doublet density does not require special algorithms such as t-SNE or cluster identification, or any machine learning techniques for that matter. It only requires good old-fashioned statistical analysis to determine whether those spikes are unlikely to occur by chance. Similarly, if some words have a strong proximal affinity for, or aversion to, word doublets—that is, if there really are triggers or inhibitors—then that too can be tested using good old-fashioned statistical techniques.
  

I am not saying with certainty that your analysis does not support some of your conclusions. I am saying only that, if it does, then you have not yet described the details needed to justify drawing those conclusions.