Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

- Home
- Canvas
- New Quizzes Hub
- New Quizzes Discussion
- Re: Grading Multiple Answer Questions

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

07-14-2021
07:51 AM

Grading Multiple Answer Questions

2 Replies

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

07-15-2021
09:39 AM

@pdk4 I am also an instructor, and I understand your point, but the method of only adding a percent for each correct answer requires a maximum possible number of choices in a multiple answer question. Otherwise, students can simply select every single answer and get full credit without even reading the question.

Similarly, the method of only subtracting points for every incorrect answer requires a minimum number of selected answers, otherwise students could receive full credit for selecting only one of the multiple correct answers. Subtracting points also requires a minimum score so that negative scores can be prevented.

The normal behavior in classic quizzes was: "To calculate scores for Multiple Answers quiz questions, Canvas divides the total points possible by the amount of correct answers for that question. This amount is awarded for every correct answer selected and deducted for every incorrect answer selected. No points are awarded or deducted for correct or incorrect answers that are not selected. For example, an instructor may create a Multiple Answer quiz question with 9 points possible that includes three correct choices and two incorrect choices. If a student selects two correct answers and one incorrect answer, they would be awarded 3 total points for that question. This would be calculated by awarding 3 points (9 total points divided by 3 correct answers) for each correct answer and subtracting 3 points for the incorrect answer." https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-create-a-Multiple-Answers-quiz-question...

Multiple answer questions are one of the few question types that already have partial credit, and the method used is very much in keeping with classic quizzes: "Partial credit is calculated by dividing the total points possible by the number of correct answers for the question. This amount is awarded for each correct answer selected and deducted for each incorrect answer selected. No points are awarded or deducted for answers that are not selected. Zero is the lowest possible score. For example, if a question is worth 4 points and there are 2 correct answers, Canvas divides 4 by 2 and awards 2 points for correct answers selected and deducts 2 points for incorrect answers. If a student selects 2 correct answers and 1 incorrect answer, the student will receive 2 points total (4 points awarded for correct answers and 2 points deducted for incorrect answers)." https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-create-a-Multiple-Answer-question-in-Ne...

One way to manage the possibilities if you wish to avoid a zero point score for one correct and one incorrect response, Is to limit the number of distractions and in the prompt specify the number of correct answers, "which three of the below..." for example. So in a 3 point question, If there are 5 total answers, and 3 are correct, and students know to select three, then students could select:

- 1 correct and 2 incorrect: 0 points
- 2 correct and 1 incorrect: 1 point
- 3 correct: 3 points
- 2 correct and 1 skipped: 2 points
- 1 correct, 1 incorrect, 1 skipped: 0 points
- 2 incorrect and 1 skipped: 0 points
- 3 correct and 1 extra (incorrect): 2 points
- 3 correct and 2 extra (incorrect): 1 point

It can be helpful for students to have an explanation in the instructions that this a question in which it is better not to guess blindly, because points are both added for correct answers and also subtracted for incorrect answers.

Another option that is already available is to write a multiple choice question using lists in each possible answer, and selecting vary points by answer. Then students would only get to select one option, but you could put enough possible combinations to cover typical multiple answer selections, and set the points for each combination to match your preferences.

There will be work underway soon to add partial credit to other question types, which may be more urgent than adding more ways of calculating partial credit for multiple answer questions, but it would be interesting to see a version of vary points by answer, which would allow assigning a percent of the total points for a correct and incorrect responses to each option.

That way you could have the option to set a lower percent deduction for wrong answer than percent added for correct answers. In long multiple answer questions you could even weight the possible answers by difficulty. A difficult correct answer could have a higher percent added, and an easy to recognize incorrect answer could have a higher percent deducted.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

07-15-2021
11:06 AM

Perhaps there is some misunderstanding here. The penalty deduction option we’re advocating does not eliminate giving (+) credit for correct answers. It merely adjusts the credit additionally deducted for incorrect answers in a more flexible way when the number of incorrect answer choices exceeds the number of correct answer choices.

For example, in a Multiple Answer question worth 12 points with 2 correct answers and 4 incorrect answers, here’s how the scenarios play out with the current score calculation algorithms—

(1) Only give credit for correct answers with no penalty for incorrect answers:

Each correct answer = 12 / 2 correct answers = 6 points. No deduction for incorrect answers.

Full credit can be obtained by selecting ALL answers = 6 + 6 = 12. This clearly games the system and should be avoided.

(2) Give credit for correct answers and deduct credit for incorrect answers:

Each correct answer = 12 / 2 correct answers = 6 points. Each incorrect answer = -12 / 2 correct answers = -6 points.

Selecting all answers gives 6 + 6 - 6 - 6 - 6 - 6 = -12, which would be rounded up to 0. Very harsh penalties.

Selecting 2 correct answers and only 2 incorrect answers gives 6 + 6 - 6 - 6 = 0. We would prefer this not to be zero, but rather something > 0, although not full credit. This combination entailed some thought by the quiz taker instead of just blindly choosing every answer with no thought. These two combinations should be scored differently. This is the problem with the current scoring algorithm.

So we propose this additional 3rd option—

(3) Give credit for correct answers and deduct credit for incorrect answers a different way:

Each correct answer = 12 / 2 correct answers = 6 points. Each incorrect answer = -12 / 4 INCORRECT answers = -3 points.

Selecting all answers gives 6 + 6 - 3 - 3 - 3 - 3 = 0, as desired. The system still can’t be gamed by blindly choosing every answer.

Selecting 2 correct answers and 2 incorrect answers gives 6 + 6 - 3 - 3 = 6. There is still a penalty for choosing some incorrect answers, but some credit is now given for correct answers. This combination is now distinguished from blindly choosing all answers.

This is the system we desire for questions with more incorrect answers than correct ones.

Please note that for questions with more correct answers than incorrect ones, option (2) would still allow the system to be gamed by choosing all the answers. Option (3) would give 0 score in that case and also prevent gaming. For anyone who feels that option (3)’s penalties are too harsh here, option (2) is still available.

The reason we advocate option (3) is that it can be programmed using basically the same algorithm calculations and interface as option (2). Just one calculation is altered and added, and one more checkbox is added.

Enabling the ability to assign individual credit to each answer choice would certainly be a wonderful addition, too, but it would require more new computation algorithms and user interface tools than option (3)’s and would entail much more programming. Option (3) is a quick addition that shouldn't require lots of programming overhead and would resolve lots of issues that have been raised concerning Multiple Answer scoring.

I hope this helps clarify our request and reasoning. Thank-you for considering it.

Related Topic

- Performance by Quintile Conundrum in New Quizzes Discussion
- Is there a way to have essay questions in New Quizzes display an answer key? in New Quizzes Discussion
- Hyperlink to items needing grading gone in New Quizzes Discussion
- NQ Reflection: Taking the Mystery out of Assessment Results in New Quizzes Discussion
- Timeline to move Grade by question out of beta in New Quizzes Discussion

- Reference
- Community Guidelines
- Security
- Developers
- Open Source