[Languages] Option to implement unicode normalization for answer key and student's input in quizzes/exams

[mmtscn]

Hi,

There are some known issues with Hebrew in the past years.

I think I have found the cause to all the known problems. Some are related to the OS and/or keyboard that people use to type Hebrew.

One particular issue, however, is the ways that the browsers handles the words teacher/students typed into the browser.

According to my limited tests, all major web browsers such as Chrome, Firefox do not use Unicode Normalization to process what users typed into the browser. Safari (the MacOS version), however, upon user submission on Canvas, will normalize the user inputs. On many languages, there will be no difference. Unfortunately, with Hebrew that comes with vowels and cantillation marks, the issue is huge.

When the teacher types in the answer key, using a browser other than Safari, the text is not normalized. When the students uses Safari to take the quizzes/exams, the input is normalized. The result is that the student's answer is marked as incorrect, although it looks identical to the answer key.

שָׁ (\uFB2A\u05B8) and שָׁ (\u05e9\u05b8\u05c1) and שָׁ (\u05e9\u05c1\u05b8) all look the same, but are treated as three different answers.

The one in the middle, שָׁ (\u05e9\u05b8\u05c1), is the normalized text. 

Excellent explanation of the issue of normalization with Hebrew may be found in the SBL Hebrew User Manual, pp. 8–17.

In this single example, to account for all student's input scenarios, I have to list all three answers as the correct answer, although they look exactly the same to human eyes!

My suggestion:

Please provide an option for us to normalize both the answer key and the student's input, so that we don't need to worry about these variations!

This is crucial in dealing with Hebrew on Canvas!!!

PS: I have confirmed it myself with JavaScript.

If I first normalize both answer and user input before comparing the strings, they give a much better outcome:

  input = input.normalize('NFC');

  if (typeof ans === 'string') { // one correct answer.
    ans = ans.normalize('NFC');
    if (input == ans) {
      mark.innerHTML = '';
    } else {
      mark.innerHTML = '';
    }
  } else if (typeof ans === 'object') { // multiple correct answers.
      for (i = 0; i < ans.length; i++) {
        ans[i] = ans[i].normalize('NFC');
      }
      if (ans.includes(input)) {
        mark.innerHTML = '';
      } else {
        mark.innerHTML = '';
      }
  }

[Stef_retired]

@mmtscn Thanks for sharing this feedback. As the discussion moves forward, would you confirm that you conducted this test using a New Quizzes fill-in-the-blank question? If you haven't, would you test it using the new quiz tool and let us know if the result is the same (How do I create a Fill in the Blank question in Ne...)?

[mmtscn]

Hi @Stef_retired ,

Thanks a lot for you prompt reply.

I'm glad to see that the New Quizzes are more flexible with the fill-in-the-blank questions. I had not tried it before. I just tested it. It's definitely been improved. I have some interesting findings that I'd like to share.

I created two identical questions, the first on Safari, the second on Google Chrome.

Then I had a preview test on Safari and Google Chrome respectively.

They renders the answers and the inputs differently, but also consistently. The result is kind of satisfying.

Put it simply: Safari (right side in the screenshot below) consistently decomposes the שׁ (\uFB2A) letter into שׁ (\u05e9\u05c1) and switches the Shin Dot with Qamets: {SHIN}{QAMATS}{SHIN DOT}. In other words, Safari consistently normalizes both the answer key and the input.

Google Chrome (left side in the screenshot below) consistently does not normalize the answer key and the input.

I would say that this is indeed an improvement. In this circumstances, I don't need to worry about the normalization. I only need to to worry about the different inputs of students, when they use the browsers that do not normalize the text.