Adam Engel

Emerson College's testing experience of New Quizzes

Discussion created by Adam Engel on Aug 14, 2019
Latest reply on Aug 22, 2019 by Todd Van Zandt

Hello, I’m part of the Instructional Technology Group at Emerson College. We’re happy with the overall way that New Quizzes is developing. We especially like that questions can be associated with specific passages that stay alongside those questions, that the interface for providing feedback on automatically-graded questions is clear and non-intimidating, and that the quizzes are easy for students to navigate.

 

After testing, here are aspects of New Quizzes we’d like to see improved:

 

  • Faculty experience of editing a quiz
    • There aren’t tooltips to help the user know what buttons do.
    • The wording on some of the options is confusing. “Stimulus” seems to have replaced the “Text (no question)” option, but that’s not made clear anywhere within the platform.
    • The use of “Question stem” is inconsistent.
      • In a fill-in-the-blank question, the stem field’s grey default text reads "Add question stem (optional)," while the statement field, if the stem is blank, displays the error "Question stem cannot be blank" (see attached screenshot). This seems contradictory.
      • It isn’t clear what the difference is between the Question Stem and the Statement.
      • Essay questions have the field "Enter question/prompt" instead of “Question stem,” which is inconsistent.
    • Re-titling the quiz in the editor does not change the name of the assignment given to it when first creating it. Clicking the old assignment name takes you into the same editor. To rename, the user must click the vertical ellipses button and “Edit” while in the list of assignments.
    • Without seeking out a detailed explanation, faculty will not understand why a quiz’s questions can add-up to a different point value than the total points the quiz itself is worth. This will likely cause more confusion than benefit.
    • The interface for New Quizzes doesn’t resemble anything else in Canvas, which will steepen its learning curve.
      • Everywhere else in Canvas, when you’re done editing content, you click either “Update” or “Save.” In New Quizzes, the save button is “Return,” which doesn’t let users know that their work will be saved, nor is it clear to where they are returning.
      • The term “Edit” is used for content creation elsewhere in Canvas, but in New Quizzes it’s “Build.”
      • The course navigation menu disappears while editing/viewing a quiz.
  • Student experience of usability and question order
    • Students have the option to “pin” a question, but there are no tooltips or explanations of what that does unless a student expands the question navigator.
    • It isn’t clear how to expand the question navigator. The arrow icon at top-left of an assessment seems like it would expand the course navigation menu, not the question navigator.
    • Why is the question navigator hidden by default? Either this or the course navigation menu could helpfully occupy this space.
  • Moderation
    • Why not bundle the moderation settings, or at least the settings to allow extra time/attempts, with the due date and available from/until dates? In current quizzes, our faculty rarely realize that “Moderate this quiz” is in a different place than the “Assign to” box, so we usually have to moderate quizzes for them.
  • Feedback timing
    • New Quizzes doesn’t have the “Show students the correct answers at...” date setting.
  • Connection to gradebook and Speedgrader
    • Can’t display one question at a time or otherwise jump between questions in Speedgrader, even if the questions were provided to students one-by-one.
  • Regrading questions
    • After going through the regrading process, there’s no clear indication of if the changes have applied or when they’ll be reflected.
    • Regrading seems to wipe question-level feedback if it was already entered.
  • Accessibility
    • In hot spot / image questions, New Quizzes gives a clear warning to faculty that: “This question type is not accessible to users requiring keyboard-only access or screen readers.” Can this warning be more prominent?

 

I searched for feature requests about each of these issues, but didn’t find any. Please let me know if any of these changes are already in a feature request that I missed, or on the development roadmap!

Outcomes