Making a test for translations unbiased by images

[oshfdk]

In theory, one could verify the translation by seeing if the passage actually makes sense or just outputs gibberish, and whether it can reasonably match the visual content. And word definitions would need to be reasonably consistent across the entire text as well.

"Translation" usually implies a lot of flexibility in interpreting the text. If your method is deterministic and can be reproduced following simple unambiguous rules with no or little possibility of multiple interpretations, then yes, producing meaningful passages would be very interesting.

I would appreciate your test with the word scramblings so I can see if my method truly holds up.

Thank you! It is very important to test new approaches that help people self-evaluate their solutions.
I'm attaching a text file with 10 pairs of lines. In each pair one line is verbatim as it appears in ZL EVA transcription, and another is the same line with most words and word pieces shuffled.

Your mission, should you choose to accept it, is to identify for each pair which of the two lines is the original (for example, it produces some meaningful translation) and which is the scrambled version (produces word salad/gibberish). You can do this in public (post your results) or in private (just write back when you are done), it's up to you. I will then post the list of the original unscrambled lines. The main goal is for you to understand whether your method can successfully identify meaningful text. Since it's trivial to just find the correct lines in the transcription file, publishing the results doesn't prove anything and not publishing them doesn't mean anything bad either.

Just in case the following is a technical description of how this file was created. The source lines were taken from ZL EVA transcription using the following command:

Code:

cat zl.txt| perl -ne 'print if(/\S{70,}/)' | grep -o -E "\S+$" | grep -v -E '[<{:?]' | shuf -n 10

Which takes all lines longer than 70 characters, removes the locus tag, discards all lines containing special formatting, uncertain readings, etc and then randomly selects 10 lines from the result.

Then these 10 lines were passed through the scrambler code, which keeps the very beginning and ending of each line and mixes word and word pieces splitting at spaces and non-pedestaled gallows characters and shuffling the pieces in a way that preserves their preceding and following separators (so, if a certain chunk was between a gallows character and a space, it will end up between a gallows character and a space).


  pairstask.txt (Size: 1.44 KB / Downloads: 10)