In Quiz, the way Multiple Answer question is graded is not what I want. Can you add a different grading option?

[QianYu]

Currently, Canvas grade the Multiple Answers questions this way:

When creating a "Multiple Answer" question, you can set the total points for the question and assign points for each correct answer.

• Point Calculation: Canvas divides the total points by the number of correct answers. Students earn points for each correct answer they select and lose points for each incorrect answer they select.
• Example: If a question is worth 3 points and has three correct answers, each correct answer is worth 1 point. If a student selects one correct answer and one incorrect answer, they would receive 1-1=0 points.

I think this is not reasonable -- it does not give credit for students not choosing the incorrect answer. 

This is the way I would like: Divide total points by the number of given answers/choices including both incorrect and correct answers. Then students get the point if they select the correct choice AND they also get the point if they did not select the incorrect choice. Basically, this is equivalent to choose True or False for each answer choice. 

Right now, I have to convert all my Multiple Answer questions into True/False questions to implement this grading system. I hope I don't need to go through that detour.

Could Canvas offer this type of grading? 

[james_whalley]

@QianYu That's interesting. That functionality definitely does not exist right now, but you could put in a feature idea.

So, using your grading method, if there were 4 options (3 correct, 1 incorrect):

- If I select none, I get 1 point (for not selecting the incorrect).

- If I select one incorrect and no others, I get 0 points.

- If I select one correct and no others, I get 2 points (1 for correct and 1 for not selecting the incorrect).

- If I select one correct and one incorrect, I get 1 point (1 for correct and 0 for selecting the incorrect).

- If I select two correct and no others, I get 3 points (2 for correct and 1 for not selecting incorrect).

- If I select two correct and one incorrect, I get 2 points (2 for correct and no penalty for selecting the incorrect).

- If I select all, I get 3 points (for correctly selecting the 3 correct and no penalty for selecting the incorrect).

- If I select the correct three and do not select the incorrect one, I get 4 points.

And using Canvas' current method, if there were 4 options (3 correct, 1 incorrect) and I made the question worth 4 points:

- If I select none, I get 0 points (for not selecting the correct answers).

- If I select one incorrect and no others, I get 0 points (because 0 is the lowest).

- If I select one correct and no others, I get 1.33 points (+1.33 for correct).

- If I select one correct and one incorrect, I get 0 points (+1.33 for correct and -1.33 for selecting the incorrect).

- If I select two correct and no others, I get 2.66 points (+2.66 for correct).

- If I select two correct and one incorrect, I get 1.33 points (+2.66 for correct and -1.33 for selecting the incorrect).

- If I select all, I get 2.66 points (+4 for correctly selecting the 3 correct and -1.33 for selecting the incorrect).

- If I select the correct three and do not select the incorrect one, I get 4 points.

In six of these scenarios, your grading system would lead to higher scores for the students, including giving them more point for selecting all of the answers and points for selecting no answers. (Just to be clear, I am not placing any judgement on this, just trying to think through the permutations.)

Now, using your grading method, if there were 4 options (1 correct, 3 incorrect):

- If I select none, I get 3 points (for not selecting the incorrect three).

- If I select one incorrect and no others, I get 2 point (2 for not selecting two incorrects).

- If I select two incorrects and no others, I get 1 point (1 for not selecting an incorrect).

- If I select three incorrects and no others, I get 0 points.

- If I select one correct and no others, I get 4 points (1 for correct and 3 for not selecting the incorrects).

- If I select one correct and one incorrect, I get 3 points (1 for correct and 2 for not selecting two incorrects).

- If I select one correct and two incorrects, I get 2 points (1 for correct and 1 for not selecting an incorrect).

- If I select all, I get 1 points (1 for correct).

And using Canvas' current method, if there were 4 options (1 correct, 3 incorrect) and I made the question worth 4 points:

- If I select none, I get 0 points.

- If I select one incorrect and no others, I get 0 points.

- If I select two incorrects and no others, I get 0 points.

- If I select three incorrects and no others, I get 0 points.

- If I select one correct and no others, I get 4 points (+4 for correct).

- If I select one correct and one incorrect, I get 0 points (+4 for correct and -4 for selecting an incorrect).

- If I select one correct and two incorrects, I get 0 points (+4 for correct and -4 for not selecting multiple incorrect).

- If I select all,I get 0 points (+4 for correct and -4 for not selecting multiple incorrect).

In this scenario, you give points for everything but selecting all of the incorrects and not selecting the correct one. Again, it is six scenarios where you are grading higher, but this time there are more opportunities for students to get points without selecting the correct one at all.

I could see this kind of grading being useful in certain scenarios, but would recommend reflection if thinking of applying it widely.

[QianYu]

I guess it depends on the question. I used this Multiple Answers to replace some old "Explain the concept" questions in written exam. I give a concept, and ask students to select the correct statements about this concept. It makes sense to give credits if students do not select the incorrect answers. At least for this type of questions, no penalty is a reasonable grading choice. Canvas can offer this grading options and let the instructor decide what is proper to use.

[James]

@QianYu 

Multiple-answers is not the same as multiple true-false questions. With New Quizzes, you might be able to use a stimulus questions with multiple true-false sub-parts. With Classic Quizzes, you would need multiple, separate true-false questions (possibly putting them in a question group to randomize them).

There is a lot of confusion over what multiple-answers questions are. The first time I used them, I thought it was true-false like you did. Then I had to go through and regrade every question on the quiz. It only took me one time of doing that to break out the documentation and realize what was going on. I've never made that mistake again.

Part of that discovery led me to write QuizWiz, which allowed people the opportunity to grade differently than how Canvas intended. Someone wrote a week or so ago saying that it was broken, so I don't know if it still works or not (I haven't tested it recently as I don't use those questions like that anymore).

When writing QuizWiz, I did a lot of research into what other LMSes did and why the way Canvas graded was the best way to grade under the assumptions that they are making. That document is at Understanding Multiple Answers Questions.

There is a lot of disagreement with my assessment, but it usually is because someone wants Canvas to do something other than what the question type is intended to do. Much of that has to do with people coming from other LMSes where they did it differently, but some of it -- such as in my case -- is because Canvas is usually so intuitive that we don't need to look at documentation to use something, but in this case it isn't. It's clearly documented what happens, but it's not intuitive to most people.

There is a feature idea already written for [New Quizzes] Partial Credit for Multiple Answers WITHOUT Penalty that you might check out before writing your own feature request. I think Canvas combined many requests into that one at some point, so the discussion (there's a lot of it) doesn't necessarily match the title.

[QianYu]

Thanks for pointing to the previous discussions. I understand the rationale of the current Canvas grading better. Now I realized the loophole of my proposed grading option is giving credits for students who did not even respond to the question. They will get the partial credits for not selecting the incorrect answers. My current way using True-False for each choice will avoid this, because students won't get credits for not responding (not choose T or F). One way to fix the loophole is to set a rule: if students didn't select any answers at all, they will get "0" point.

[James · Accepted solution]

@QianYu 

Your loophole fix doesn't work the way you want it to. Let's say there are five parts, four are false and one is true. If the student correctly realizes the four are false and doesn't check them, but misses the single true answer, then they have -- in their mind -- correctly answered 80%. But we still won't know if they thought all five were false or if they just didn't answer the question.

Worse, if all of the answers were false, the students could not get 100% because checking any answer would be wrong.

Canvas did it right when they decided not to give credit for non-responses. A true-false question needs to have some way where the students can indicate false through an action as opposed to lack of an action.

If you want allow partial credit, then you really need to use more than one true-false question.

Now, if you want all-or-nothing, there are some tricks you can use. Please don't try these. They are terrible ideas.

If you want all-or-nothing and are using Classic Quizzes you can fake it (New Quizzes supports all-or-nothing). I don't think I've ever seen anyone recommend this -- with good reason, it sucks!.

You could be to write a multiple choice question like this one (but please don't do this!).

Which of the following are true about the quadratic equation \(ax^2+bx+c=0,~a\ne0\)?

1. There are two unique solutions to a quadratic equation.
2. The sum of the solutions is \(-\frac{b}{a}\).
3. Our class covers four ways to solve these equations.

Then, in the answers section, you have

1. Only a
2. Only b
3. Only c
4. Both a and b but not c
5. Both a and c but not b
6. Both b and c but not a
7. All three of a, b, and c
8. None of these are correct

I would never, ever, recommend someone do that, though. It removes the ability to randomize the order of the responses, which makes it easier to cheat. It's confusing. It doesn't scale as there are \(2^n\) options when there are \(n\) choices to pick from. Besides that, you either get full points or no points.

Another confusing way would be to make it a numeric question and then ask them to sum their answers. Again, that would be full points or no points.

Which of the following are true about the quadratic equation \(ax^2+bx+c=0,~a\ne0\)?

• 1: There are two unique solutions to a quadratic equation.
• 2: The sum of the solutions is \(-\frac{b}{a}\).
• 4: Our class covers four ways to solve these equations.

Find the sum of the numbers in front of the true statements. Example: if 1 and 4 are correct but 2 is incorrect, your answer would be 1+4=5.

This is bad because it removes the randomization options. It is better than the multiple choice because it doesn't have as many choices. It is harder because I had to explain how to answer the question (I taught math, so I was able to get away with this), but any time you have to explain how to do something, you should consider easier ways.