Has anyone ever "deciphered" a paragraph?
#21
(13-04-2021, 01:46 AM)obelus wrote: You are not allowed to view links. Register or Login to view.
(12-04-2021, 08:14 PM)ReneZ wrote: You are not allowed to view links. Register or Login to view.
(12-04-2021, 05:41 PM)Mark Knowles wrote: You are not allowed to view links. Register or Login to view.It is perfectly possible for a cipher to decrease entropy.



It would be of great interest to have a historical example of this.



Morse code is a fine example.  The alphabetic representation of English is highly redundant at 4 bits/character;  Shannon himself demonstrated by experiment that the information content of written English is in the range 0.6-1.3 bits/character.  By exploiting the relative frequencies of English letters, Morse code compresses text to approximately 2.5 bits/character.

There's some misunderstanding here.

First, more bits per character mean less redundancy, not more. The more information per character on average, the less characters are needed to convey your message, hence the more efficient (or less redundant) you are with your writing.

So, if something is 4 bits per character, and then some procedure makes it 2.5 bits per character, then it's expansion, and not compression.

Second, one cannot be too frequent in reiterating that two H1 values are not directly comparable for alphabets of different size. What should be compared instead are H1-H0, or H1/H0 values.

Third, I guess what is stated here with reference to Shannon and compared to English H1 of 4 bits/character, - is not H1, but rather the stuff called "entropy rate", which is actually the limit of N-th order conditional entropy when N approaches infinity. The two parameters are not directly comparable, of course.

Fourth, I don't know what's the 2.5 figure for the Morse code. Assuming the alphabet size of 3 character (dot, dash, and pause (space)), no character entropy (no matter the order) for texts so encoded can exceed 1.58, since that would be H0 (logarithm of 3 base 2), and any entropy of order N would be less than H0.
[+] 2 users say Thank You to Anton for this post
Reply


Messages In This Thread
RE: Has anyone ever "deciphered" a paragraph? - by Anton - 13-04-2021, 02:07 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  How do paragraph-initial gallows effect a paragraph? Skoove 7 4,255 29-10-2025, 09:11 AM
Last Post: Mauro

Forum Jump:


Users browsing this thread: 1 Guest(s)