Input data accuracy
#7
Hi Rene,
I ran a couple of simple experiments. As always, I may have made errors.

These are % histograms of word-start and word-end characters after and before Certain (.) or Uncertain (,) spaces (ZL EVA). There are some large and interesting differences, but both types of spaces appear to respect the basics of Voynichese word structure: e.g. there are very few words ending with 'e' or 'q' even among uncertain words.

   

   

I also modified the first ~38K words of Dante's Comedy by inserting a space after some specific characters.
E.g. the first lines are:

nel mezzo del cammin di nostra vita
mi ritrovai per una selva oscura 
che la diritta via era smarrita 

dante_e transforms them in the following way:

ne l me zzo de l cammin di nostra vita
mi ritrovai pe r una se lva oscura 
che  la diritta via e ra smarrita

Of course these replacements increase the number of tokens. Unless I messed up something, it seems that also in this case splitting words results in fewer word types and hapax words.

   
[+] 1 user says Thank You to MarcoP for this post
Reply


Messages In This Thread
Input data accuracy - by ReneZ - 07-05-2021, 08:26 AM
RE: Input data accuracy - by geoffreycaveney - 07-05-2021, 02:12 PM
RE: Input data accuracy - by ReneZ - 07-05-2021, 02:31 PM
RE: Input data accuracy - by geoffreycaveney - 07-05-2021, 02:37 PM
RE: Input data accuracy - by ReneZ - 12-05-2021, 12:06 PM
RE: Input data accuracy - by MarcoP - 12-05-2021, 05:58 PM
RE: Input data accuracy - by MarcoP - 13-05-2021, 06:49 PM
RE: Input data accuracy - by ReneZ - 14-05-2021, 06:03 AM
RE: Input data accuracy - by MarcoP - 15-05-2021, 07:51 AM
RE: Input data accuracy - by ReneZ - 15-05-2021, 10:50 AM
RE: Input data accuracy - by MarcoP - 15-05-2021, 11:25 AM
RE: Input data accuracy - by ReneZ - 15-05-2021, 12:36 PM
RE: Input data accuracy - by ReneZ - 16-05-2021, 08:10 AM
RE: Input data accuracy - by MarcoP - 20-05-2021, 08:26 AM
RE: Input data accuracy - by ReneZ - 25-12-2022, 09:48 AM
RE: Input data accuracy - by ReneZ - 31-12-2022, 05:26 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)