(I apologize that I have not fully digested other people's comments and your replies. From what I read there, it seemed clearer if I responded directly to your initial post and your zenodo paper only.)
What you have done is very interesting and possibly valuable work, but I am unclear on several aspects of it.
First, if I understand correctly, your first chart shows the words with the highest correlation — that is, the unique words (aka word types) most likely to occur as doublet sequences — and the words with the lowest correlation — that is, the words least likely to occur that way. All other words found in the manuscript, if plotted, would fall somewhere between the red group of bars and the blue group of bars. Is that correct?
If so, then the complete bar graph would presumably show a continuous trend of words, with correlations decreasing gradually from 0.933 for “sy” down to 0.665 for “chees.” In other words, your chart has clipped out the long middle range of a continuous trend related to how often unique words occur in doublet form -- no indication of clustering per se. This observation would not be too surprising given the many structural attributes of the manuscript's text that have already been found and written about. (None of which truly indicate the presence of meaningful text as opposed to the product of a language simulation device, by the way.)
Your cluster chart then shows the unique words as they would be located in a low three-dimensional space. The dimensional metrics themselves are generated by the (black-box) t-SNE algorithm, so we cannot say what they actually represent. However, we can be sure that none of those dimensions corresponds directly to the logistic correlation coefficient shown in the first plot, since “sy” is not located at an extreme on any of the three axes.
The coloring of red and blue in that cluster plot is presumably assigned simply by the sign of the z-direction value. (So, if one could view that chart in true three dimensions from above, the points in the red cluster would be closer to the viewer than the blue ones, or vice versa.) The apparent overlap of the red and blue areas in the chart would therefore just be the result of the contouring approximation used to render the clusters as solid shaded areas.
So is this chart not just showing the positive vs negative sign of an unidentifiable metric in the low-dimension reduction from the t-SNE algorithm?
Or did you instead use a seperate clustering algorithm on the points in the t-SNE output space in order to define two clusters?
In any case, I am having a hard time understanding why you conclude from those plots that there is clustering that is related to word doublets in particular -- or to any other identifiable attribute. What am I missing there?
With the Monte Carlo simulations and the violin plot, again, if I understand your description correctly, the predictive accuracy concerns whether one can predict whether a given word belongs to one cluster rather than another. Is that correct?
If so how did you define the clusters within each of the separate 1,000 trials? Were they defined simply by the sign of a z-dimension of the t-SNE output, or by a separate cluster-identification or cluster-assignment method? How is it certain that the clusters found within each of the 1,000 trials represent the same clustered properties (since each trial is a fully randomized bag of words being independently analyzed using a block-box algorithm)?
Whatever the method, your violin plot does seem to show that there is some structure to the order of words in the manuscript at some scale, and that random shuffling removes that structure. But again, I am not clear how you conclude that this structure has anything to do specifically with word doublets, as opposed to other structuring effects, (such as the self-citation patterns described by Timm or the differing vocabularies found in Currier languages A and B).
The most interesting plot, to me, is your “Cross Analysis of LSA Thematic Blocks vs. Syntactic Doublet Words”! But the data and analysis used to produce that plot seem to be completely independent of any of your previous analyses and plots, is it not?
As I understand it, you have plotted the density of word doublets per folio page and compared that to the thematic assignments that have been observed through folio sequence of the manuscript. And you have found that five particular folios stand out by having four to five times as many word doublets as all the other folios. Those five pages each happen to either be the terminal page of a thematic section, or (at least in the last of the five cases) to precede a section of missing folios. Is that correct?
If so, that is a very interesting observation.
I do not mean to cast doubt on the rest of your work, but the details are unclear enough that it is hard to identify its research value. The concentration of word doublets on five particular folios, and their apparent location in the sequence of folios, is however, very intriguing.
(By the way, the work about to be published by Colin Layfield and Lisa Fagin-Davis might be worth reviewing when it becomes available soon, to see if it affects the folio sequence positioning part of your finding.)
Regards,
Andrew
What you have done is very interesting and possibly valuable work, but I am unclear on several aspects of it.
First, if I understand correctly, your first chart shows the words with the highest correlation — that is, the unique words (aka word types) most likely to occur as doublet sequences — and the words with the lowest correlation — that is, the words least likely to occur that way. All other words found in the manuscript, if plotted, would fall somewhere between the red group of bars and the blue group of bars. Is that correct?
If so, then the complete bar graph would presumably show a continuous trend of words, with correlations decreasing gradually from 0.933 for “sy” down to 0.665 for “chees.” In other words, your chart has clipped out the long middle range of a continuous trend related to how often unique words occur in doublet form -- no indication of clustering per se. This observation would not be too surprising given the many structural attributes of the manuscript's text that have already been found and written about. (None of which truly indicate the presence of meaningful text as opposed to the product of a language simulation device, by the way.)
Your cluster chart then shows the unique words as they would be located in a low three-dimensional space. The dimensional metrics themselves are generated by the (black-box) t-SNE algorithm, so we cannot say what they actually represent. However, we can be sure that none of those dimensions corresponds directly to the logistic correlation coefficient shown in the first plot, since “sy” is not located at an extreme on any of the three axes.
The coloring of red and blue in that cluster plot is presumably assigned simply by the sign of the z-direction value. (So, if one could view that chart in true three dimensions from above, the points in the red cluster would be closer to the viewer than the blue ones, or vice versa.) The apparent overlap of the red and blue areas in the chart would therefore just be the result of the contouring approximation used to render the clusters as solid shaded areas.
So is this chart not just showing the positive vs negative sign of an unidentifiable metric in the low-dimension reduction from the t-SNE algorithm?
Or did you instead use a seperate clustering algorithm on the points in the t-SNE output space in order to define two clusters?
In any case, I am having a hard time understanding why you conclude from those plots that there is clustering that is related to word doublets in particular -- or to any other identifiable attribute. What am I missing there?
With the Monte Carlo simulations and the violin plot, again, if I understand your description correctly, the predictive accuracy concerns whether one can predict whether a given word belongs to one cluster rather than another. Is that correct?
If so how did you define the clusters within each of the separate 1,000 trials? Were they defined simply by the sign of a z-dimension of the t-SNE output, or by a separate cluster-identification or cluster-assignment method? How is it certain that the clusters found within each of the 1,000 trials represent the same clustered properties (since each trial is a fully randomized bag of words being independently analyzed using a block-box algorithm)?
Whatever the method, your violin plot does seem to show that there is some structure to the order of words in the manuscript at some scale, and that random shuffling removes that structure. But again, I am not clear how you conclude that this structure has anything to do specifically with word doublets, as opposed to other structuring effects, (such as the self-citation patterns described by Timm or the differing vocabularies found in Currier languages A and B).
The most interesting plot, to me, is your “Cross Analysis of LSA Thematic Blocks vs. Syntactic Doublet Words”! But the data and analysis used to produce that plot seem to be completely independent of any of your previous analyses and plots, is it not?
As I understand it, you have plotted the density of word doublets per folio page and compared that to the thematic assignments that have been observed through folio sequence of the manuscript. And you have found that five particular folios stand out by having four to five times as many word doublets as all the other folios. Those five pages each happen to either be the terminal page of a thematic section, or (at least in the last of the five cases) to precede a section of missing folios. Is that correct?
If so, that is a very interesting observation.
I do not mean to cast doubt on the rest of your work, but the details are unclear enough that it is hard to identify its research value. The concentration of word doublets on five particular folios, and their apparent location in the sequence of folios, is however, very intriguing.
(By the way, the work about to be published by Colin Layfield and Lisa Fagin-Davis might be worth reviewing when it becomes available soon, to see if it affects the folio sequence positioning part of your finding.)
Regards,
Andrew