Spaces and Entropy
#10
(03-05-2022, 05:49 PM)Bernd wrote: You are not allowed to view links. Register or Login to view.Generally I think removing spaces is a good idea for entropy statistics as it also removes a lot of potential parsing errors (what's a space, half-space and so on).
But can we infer from this that spaces in the VM indeed function like they do in ordinary texts? This would indicate they are not nulls and glyph combinations across spaces are different from ghyph combinations confined by spaces. I would expect that, has this been investigated?
Spaces are needed if same glyph can be a letter or the starting of a bigram/3-gram that forms a letter. It would be a way to distinguish between both cases. Spaces would be letter/letters separators and by extension glyph combinations would be only within spaces.
Reply


Messages In This Thread
Spaces and Entropy - by Koen G - 03-05-2022, 02:01 PM
RE: Spaces and Entropy - by Ruby Novacna - 03-05-2022, 02:17 PM
RE: Spaces and Entropy - by Koen G - 03-05-2022, 02:21 PM
RE: Spaces and Entropy - by Ruby Novacna - 03-05-2022, 02:42 PM
RE: Spaces and Entropy - by Koen G - 03-05-2022, 02:56 PM
RE: Spaces and Entropy - by Ruby Novacna - 03-05-2022, 04:10 PM
RE: Spaces and Entropy - by Emma May Smith - 03-05-2022, 05:22 PM
RE: Spaces and Entropy - by Koen G - 03-05-2022, 06:04 PM
RE: Spaces and Entropy - by Bernd - 03-05-2022, 05:49 PM
RE: Spaces and Entropy - by Juan_Sali - 03-05-2022, 06:25 PM
RE: Spaces and Entropy - by Koen G - 03-05-2022, 06:25 PM
RE: Spaces and Entropy - by Koen G - 03-05-2022, 06:39 PM
RE: Spaces and Entropy - by davidjackson - 03-05-2022, 07:19 PM
RE: Spaces and Entropy - by Emma May Smith - 03-05-2022, 11:11 PM
RE: Spaces and Entropy - by Ruby Novacna - 03-05-2022, 11:24 PM
RE: Spaces and Entropy - by Koen G - 03-05-2022, 11:35 PM
RE: Spaces and Entropy - by Emma May Smith - 04-05-2022, 01:19 AM
RE: Spaces and Entropy - by ReneZ - 04-05-2022, 05:16 AM
RE: Spaces and Entropy - by Bernd - 04-05-2022, 12:22 PM
RE: Spaces and Entropy - by Koen G - 04-05-2022, 01:32 PM
RE: Spaces and Entropy - by MichelleL11 - 04-05-2022, 04:19 PM
RE: Spaces and Entropy - by Bernd - 05-05-2022, 11:27 AM
RE: Spaces and Entropy - by Koen G - 04-05-2022, 04:44 PM
RE: Spaces and Entropy - by ReneZ - 04-05-2022, 06:00 PM
RE: Spaces and Entropy - by Koen G - 04-05-2022, 06:26 PM
RE: Spaces and Entropy - by ReneZ - 05-05-2022, 06:11 PM
RE: Spaces and Entropy - by cvetkakocj@rogers.com - 05-05-2022, 05:17 PM
RE: Spaces and Entropy - by Searcher - 07-05-2022, 02:38 PM
RE: Spaces and Entropy - by Koen G - 20-05-2022, 10:45 AM
RE: Spaces and Entropy - by Bernd - 20-05-2022, 02:22 PM
RE: Spaces and Entropy - by Koen G - 20-05-2022, 03:59 PM
RE: Spaces and Entropy - by Searcher - 20-05-2022, 04:10 PM
RE: Spaces and Entropy - by Anton - 22-05-2022, 12:28 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Confirmation that spaces are really spaces quimqu 34 13,809 21-11-2025, 08:44 AM
Last Post: ReneZ

Forum Jump:


Users browsing this thread: 1 Guest(s)