Multiple answer questions with partial credit

[CurtisLeibrand]

When writing test questions with "multiple answers" I am only getting two options for grading:

1. Partial credit with penalty

2. Exact

I am not getting the partial credit with no penalty.  My institution is adopting Canvas for the coming fall semester, not sure my IT department understands my question. This is under "New Quizzes" and NOT "Classic". How do I add the option for partial credit without penalty?

Curtis

[Gabriel33 · Accepted solution]

To clarify, do you want a question with multiple wrong options and multiple right answers?

For example: What is 5+5?
Possible answers: "10" (correct), "Ten" (correct), 3 (wrong)

Where if the student picks any one of the two correct answers, they should get full marks?

Indeed, that doesn't seem to be an option for Multiple Answer type questions, and I can't think of an example where that's ideal, but you can get close to that with:

• Fill in the Blank: How do I create a Fill in the Blank question in Ne... - Instructure Community - 985
  Select the Answer Type to be Open Entry, and in it you can choose multiple correct answers.
  The main difference is that here the student would have to write the answer, but you could provide them the options in the text of the question.
  
  
• Multiple Choice: How do I create a Multiple Choice question in New ... - Instructure Community - 984
  Select to "Vary points by answer", and grant the maximum points to more than one option
  The main difference here is that you can only have one set of choices per question, and in each the student can only pick one answer (even if they realize there are 2 correct).

Alternatively (though this is probably messier), after students finished the quiz, you can change the correct answer and allow for both the new and old answers be accepted: How do I regrade a quiz question in New Quizzes? - Instructure Community - 1293

[Gabriel33 · Accepted solution]

To clarify, do you want a question with multiple wrong options and multiple right answers?

For example: What is 5+5?
Possible answers: "10" (correct), "Ten" (correct), 3 (wrong)

Where if the student picks any one of the two correct answers, they should get full marks?

Indeed, that doesn't seem to be an option for Multiple Answer type questions, and I can't think of an example where that's ideal, but you can get close to that with:

• Fill in the Blank: How do I create a Fill in the Blank question in Ne... - Instructure Community - 985
  Select the Answer Type to be Open Entry, and in it you can choose multiple correct answers.
  The main difference is that here the student would have to write the answer, but you could provide them the options in the text of the question.
  
  
• Multiple Choice: How do I create a Multiple Choice question in New ... - Instructure Community - 984
  Select to "Vary points by answer", and grant the maximum points to more than one option
  The main difference here is that you can only have one set of choices per question, and in each the student can only pick one answer (even if they realize there are 2 correct).

Alternatively (though this is probably messier), after students finished the quiz, you can change the correct answer and allow for both the new and old answers be accepted: How do I regrade a quiz question in New Quizzes? - Instructure Community - 1293

[James · Accepted solution]

@CurtisLeibrand 

Allowing multiple answers without penalty has some serious flaws that mean Canvas will likely never implement it as the name describes. The biggest flaw is that a student could select all of the answers and get 100%.

Some people have suggested workarounds, such as limiting the number of answers to avoid that.

With Classic Quizzes, the only way to grade was "partial credit with penalty". Some areas have areas that need for all items to be marked correctly or incorrectly, such as nursing where one wrong answer could kill a patient, and Canvas added the "exact match" option with New Quizzes.

Back in 2016, I wrote a script that added different ways to grade multiple answer questions in classic quizzes. It did a lot more than that, but it was one of the things I looked at. I added the ability to grade multiple answer questions in different ways. The first was as all or nothing (the exact match that Canvas later added to New Quizzes) and the second was giving partial credit on multiple answer questions based on the percentage of correctly answered responses (essentially turned multiple answer questions into a series of true-false questions and Canvas handles this with stimulus questions in New Quizzes). I did not implement partial credit with no penalty.

Before implementing those two techniques, I considered other ways and wrote about them in a blog post called Understanding Multiple Answers Questions. I looked at what Angel, Blackboard, D2L, Moodle, Pearson, and Sakai offered.I used that to consider the different ways of grading and what could be done through the use of an external script. Assigning different weights to individual answers has merit, but it was too complicated to implement on my end.

The biggest issue with Multiple Answers questions is that each person has a different take on what it should mean. They see "multiple answers" and transfer their previous understanding over without understanding the way that Canvas does it. That is, in some ways, a result of good design in other places by Canvas. They make it so intuitive to use that people don't look up how to do things, they just think they understand how it works. And that's fine as long as it works the way they think it should. But when it doesn't, people get upset. It didn't help that the description in the documentation back in 2016 was ambiguous (I teach math and their description was confusing).

I'm not immune to that. The first time I used them, I didn't look up how they worked, I just assumed it was the same as a series of true/false questions. It wasn't until after I had to regrade an entire quiz by hand that dug deeper.

If you are not okay with "partial credit with penalty" or "exact match", then you shouldn't be using a multiple answers question. 

If you want partial credit with no penalty, then you can use a stimulus question with multiple true-false sub-parts and make the false answers worth 0 points and the true answers worth some positive value. Yes, it's kludgy. It doesn't allow for randomization in the way I would like (disclaimer: I'm not a fan of New Quizzes).

Although Canvas allows for that, realize that a student can still go through and select all responses and get 100% on those questions.

We may need to think of different ways to ask questions than we typically have. Before this last year, I would have said that's part of using the technology available to us. Canvas doesn't offer option A so we have to look at all of the available options, decide what features are most important to us, and pick the best alternative.  After this last semester, I realized that students are using AI to get really good scores without understanding any of the content. When you think about it, that's not so different from giving partial credit with no penalty. My initial reaction was that online, auto-graded, un-monitored assignments (like quizzes) are going to be worth a whole lot less of the course grade.