Entropy is the logarithm of perplexity, as far as I understand.
It is possible that the transformer can find some rules that work beyond simple probability map from an ngram to the next character. Would be interesting to compare the log2 of transformer perplexity with the statistical ngram entropy.
Edit: maybe as a simple fix you can show the perplexity as powers of 2 for the vertical axis, this should give us the bit entropy figures as the exponents.
It is possible that the transformer can find some rules that work beyond simple probability map from an ngram to the next character. Would be interesting to compare the log2 of transformer perplexity with the statistical ngram entropy.
Edit: maybe as a simple fix you can show the perplexity as powers of 2 for the vertical axis, this should give us the bit entropy figures as the exponents.