New Quizzes: auto-grade open responses

(6)

I'd like an option to automatically grade an open answer/essay prompt as 'correct.' Maybe something like: so long as the student enters X words then the question is marked for Y points.

My scenario is a short, free course with hundreds of initial students and open enrollment going forward. I'd like to offer a short answer prompt and an essay prompt. But I can't afford to go thru and grade this many quizzes at once, nor promptly on a continual basis.

I could (and am currently) offering these are a separate assignment they could view and respond to. But I'm not lying to anybody, they know I'm not present in that course. I feel like a short page with something amounting to 'open a document and type a response that no one will read' isn't going to be very motivational. But if I could weave these little reflections into the quiz, which is otherwise graded, I feel like the chances of participation will go up.

That's all, thanks.

15 Comments
Steven_S
Community Champion

You can already use a graded survey to assign a grade for simply responding to the questions, regardless of the answer, and that might provide adequate reinforcement to participation in your use case.

It would be helpful to be able to set a minimum word count on essay questions to quiz (and survey) responses, however.

RobDitto
Community Champion

We've been advised that after Classic Quizzes is deprecated, there will be no more Graded Surveys natively in Canvas. Thus, this idea gets my kudos & 5-star rating.  

Kelvin_Dean
Community Contributor

Rob, in my opinion, if a graded survey is a must, we would have to regrade all questions and give everyone full credit after all submissions have been received.

bishopta
Community Member

If new quizzes could auto grade open response questions, that would save me a lot of time. I like using Canvas Quizzes for spelling tests. It would be great if I could input the correct response, with which students would earn points. If it was spelled incorrectly, then they would not earn the points. this would also be helpful when grading plural nouns quizzes, as students need to be spelling those words correctly as well.

I think I saw that there was already the option to turn off spell check, so students don't have spell check to rely on when taking a spelling test. Thank you!

Steven_S
Community Champion

@bishopta  the type of auto-grading you describe should already be possible with fill-in-the-blank question types.  Auto-grading for essay questions is a different challenge, because students are expected to compose unique responses in their own words. 

As the original poster suggests length is one feature that can be auto-graded, but a length can be met by, "vguvhb vuhhvihb rszsr cyghg...."  Spelling and grammar can be reasonably accurately measured, and could be combined with length to establish a minimum length of recognizable (even if mis-spelled) words. 

That length requirement could be met by copying and pasting (or retyping with secured quizzes) a paragraph from another source, so integrating a plagiarism checker would help with auto-grading also. 

Then the requirement still has not addressed whether the words typed have anything at all to do with the question asked.  For that the instructor would have to enter a collection of key words and phrases that they expect students to use in a correct response. 

Even then, the auto check for key words and phrases is measuring that the student responses is about the correct topic, not whether the key words are used correctly.  I have had students cheat by copying random short phrases from multiple websites.  A plagiarism checker with strong enough settings can catch that (if integrated into new quizzes), but the same concept can also work in the students own words.  The students can just type random words that they expect to be related to the topic in an effort to trigger the computer check.  So, there needs to be a minimum standard for a grammar score in addition to a plagiarism check to make keyword checking work. 

Even with all of that students might write grammatically correct sentences with all of the correct keywords without plagiarizing, and still be asserting something that is an incorrect statement.  While the pre-check of an auto-grader would be useful, even if only for instant feedback that the response meets minimum standards before students click submit, instructors with concerns beyond grammar need the option to have this NOT post a grade until after we have confirmed the accuracy of what students wrote.

bishopta
Community Member

@Steven_S  I see the difference you are explaining. Thanks. However, I still do not see an option for auto grading fill in the blank, either.

jlgrignon
Community Member

I like using the New Quizzes tool to have students reflect on their ideas and journal their thoughts about various videos, readings, and images.  However, I do not have the ability to grade all of the entries.  I would like the option for Canvas to auto grade the essay type questions based on the number of words included.  So, if a student wrote the minimum number of words, they receive the credit automatically.  I would then be able to selectively review some of the submissions, but not have to grade each one every time.  

jlgrignon
Community Member

This type of feature is actually available on many of the interactive textbooks that I have tried.  Students are expected to reflect on a section of the reading in essay format.  They are given credit if they put something in the box.  I agree that some students may write nonsense, but if I randomly review a few entries each time, I can hopefully prevent most issues.  Reflection is a big part of learning and especially important in higher education.  

I think this could either be an option offered in the essay type question or there could be a new type of question called "reflection" that does not required manual grading.

Please add this feature to New Quizzes.  Thank you!

Steven_S
Community Champion

@bishopta  The fill-in-the-blank guide might help with that question type: https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-create-a-Fill-in-the-Blank-question-in-... You will see the options for creating a correct answer for the blank and deciding how close a match is required for it to be marked correct by autograding after you have created a blank.

YatingTeng
Community Explorer

The alternative solution, fill-in-the-blank, is really not ideal. Although it supports auto grading when you put a period in the grading, the box for the fill-in-the-black questions is so small (not even one line).  It's very hard for learners to respond. 

audvadia
Community Member

I completely agree with @jlgrignon. I use these types of "reflection" questions to incentivize students to come to class prepared for discussion. I don't need to individually grade their content because we go over everything together in small group and classroom discussion. However, I do want to incentivize them to do the reading and thinking before coming to class. I give three such assignments/week with 50 students in the class. Being able to auto-grade these low stakes assessments based on a minimum number of words and the use of a few key words, with spot checking to keep people honest, would be very helpful.

stclair1
Community Novice

I teach large courses and need this function desperately. I have an active learning classroom with a LOT of reflection questions and Think/pair/share notes. It would be so nice to have a "length" requirement on the reflection/essay questions that self grade for participation.

ecass
Community Explorer

I sometimes have essay type questions that I assign zero points to.  They are "think about this" type of questions and I don't want to grade them.  Because they are essay questions I have to hand grade even though everyone gets a zero.  This is VERY time consuming.   Can Canvas auto grade these as zero?

chenshall
Community Participant
Comments from Instructure

As part of the new Ideas & Themes process, all ideas in Idea Conversations were reviewed by the Product Team. Any Idea that was associated with an identified theme was moved to the new Idea & Themes space. Any Idea that was not part of the move is being marked as Archived. This will preserve the history of the conversations while also letting Community members know that Instructure will not explore the request at this time.

ProductPanda
Instructure
Instructure
Status changed to: Archived