Custom grading formula for quizzes

[JeffreyBush]

Is there a way to have a custom grading formula for quizzes? The most basic one that I am looking for right now is that they either get 100% if they get all questions right or 0% if any question is wrong (with unlimited attempts).

An obnoxious workaround would be to create 2 quizzes, put each in its own module, and require them to score 100% on the first quiz before they can do the second quiz, and then make the second quiz just a single silly question giving them full credit.

I would like to, however, use a custom formula in other cases where the above workaround would not work. For example, on a 5-question multiple choice quiz, grade it as 10, 9, 7, 4, 1, 0 for 0, 1, 2, 3, 4, 5 wrong, respectively (basically not penalizing as much for a single mistake).

I have also had issues with "extra credit" problems on classic quizzes that this may be able to address.

I can use either classic or new quizzes, or even an external plugin if that would work.

Thanks!

[James · Accepted solution]

@JeffreyBush 

As you suspect, this functionality does not exist within the native Canvas product.

For the very simplest of quizzes, you could use multiple answer questions with New Quizzes and tell it to grade using an "exact match". This gives you a one question quiz that they have to get all the parts correct for.

Your obnoxious workaround is one approach and has merit. Rather than putting each quiz it in it's own module, you could put them both in the same module. Then you would tell Canvas that the module items need completed in sequential order and put a requirement of 100% for the first quiz before they can take the second quiz. For the second quiz, you could use classic quizzes with a survey so that they get full points as long as they take the quiz. This has the benefit of automating the grading.

Another approach is to manually grade, which it looks like you're trying to avoid (I don't blame you).

Depending on how things are laid out in your course, you may not even need a second assignment. If you use modules, you could prevent them from working on the next module until they get 100%. That may be what you meant with putting them into separate modules. The much simpler, which means it may not work, approach would be to have a quiz for unit 1 that they have to score 100% on and then they cannot move to unit 2 until that 100% is obtained on unit 1's quiz.

The third approach, which is the closest to the notion of a custom formula, is to use code that performs the calculations for you and updates Canvas with the results. This is what I did with some of my assignments. I had statistics project where students had to pick variables that met five criteria. They took a quiz and then my program would run and look at their answers to see if they were satisfactory and would work for the project. It completed a rubric for another assignment and gave them a score of 0 or 10 (it was a 10 point assignment) on that assignment. The quiz itself was a survey so they got the 10 points for completing that, but they weren't able to proceed until they got 10/10 on that second assignment.

I ran that program manually over the period of about a week while students were working on it, but it could have been something I automated to run every 20 minutes (I did that with several other assignments).

I got around using module requirements and prerequisites with this project because I had the students in class and was able to tell them "Jon and Tracy, you still need to do this assignment." If it something that is automated, you'll likely want to enforce it through module requirements and/or prerequisites.

What I'm talking about is an external script, not an external plugin. There is no external plugin that uses Canvas quizzes that will do this automatically. External plugins for Canvas are called LTI tools and they would take over the whole assignment. You could write an external tool that would replace your second quiz or assignment. I did this back in 2013 with our orientation course. Students had to score at least 80% on the final quiz and then they were removed from the course and added to their original courses. I had an external tool that students clicked on in Canvas to complete their course. What we found was that many students wouldn't make the required click and so I had to write code that would go through and check for students who had completed the quiz with at least 80%. We ran that script every 20 minutes and eventually removed the LTI button and just told students that their regular classes should show up within 20 minutes (we told them an hour to be safe).

There might be an external LTI tool that handles the whole quizzing and only gives a score of 100% or 0%. If something existed, you could get by with a single assignment and have it automatically handled. But then you have to do the quizzing and grading in the external tool. All you pass back to Canvas is the final grade for that assignment. There are lots of LTI tools available, but I am not familiar enough with them to know if any allow this. Google suggests that some software, they throw CompTIA out there, may have this functionality.

As for scripts that work with native Canvas quizzes, I wrote a somewhat extensible (code it yourself) script called QuizWiz that added functionality to Classic Quizzes and SpeedGrader. I didn't program a way to give them 0 if they didn't get 100%, but it's something that could be added there. I had some things that automatically run when the quiz is loaded -- such as assigning 0 points to any essay that was left blank or giving full points for any file upload. In that case, you would open SpeedGrader, let it run, then click the next button to move to the next student and it was basically one click per student and you could go through and do that for a whole class.

I don't recommend that in this case. You would still spend too much time in SpeedGrader and you would need to do it repeatedly for each quiz until the student got 100%. It also wipes out the original score using the fudge points. Writing a script that did that in the background would be simpler and a better use of your time.

[JeffreyBush]

This is fantastic information! I did eventually realize how modules worked (I haven't used them too much) and made one assignment require the quiz and that one just gave them one critical piece of information to actually start the assignment (the quiz they had to get perfect was mainly a gatekeeper to making sure they actually fully read and comprehended the assignment - in the past I have gotten so many students telling me they didn't know X...).

In the future I would like to look into your QuizWiz for some of my grading, but at the moment I am most interested in your "offline" script that you run every X time. Do you have any place I can find these examples scripts?

[James]

@JeffreyBush 

Most of the scripts I've published are user scripts that run in the browser to enhance the experience. The stuff I run in the background on a scheduled basis rely on some libraries that I haven't published and so they won't be of much use to people since they are not complete solutions. I did give a presentation at the 2024 InstructureCon about some of the things I've done with my classes, but it was a high-level talk about how to get inspired based on your personal needs, not something that people could walk away with code (that was my 2016 presentation).

Although I don't use it myself, a popular and publicly available Python library is CanvasApi. You can probably find some examples by searching for some combination of Canvas API and Python.

You don't want to focus on the automation itself until you figure out what your needs are. Once you figure out what you want to accomplish, then you work on code to accomplish that. After it works when you manually run it, the automation is the easy part. If you have access to a server (Linux is popular), then you can set up a cron task. If you're running it from Windows, then you Windows Scheduler.

There are lots of resources (both written and people) here in the Community, especially in the Developers group.

[JeffreyBush]

I will look into the CanvasApi, looks straightforward enough. I teach CS and make many of my own tools :-), I have access to tons of servers. One of my next summer goals is to integrate moravianuniversity.github.io/intro-tools/snakebytes/ with Canvas, recording progress in Canvas and awarding points.

[James]

@JeffreyBush 

Sounds like you're in a good position to get what you need.

A tip to get you started is that scores are tied to submissions, so look at the Submissions API. You can fetch submissions quickly using GraphQL, but I've had issues when trying to use GraphQL to write the grades back (it wrote the grades but didn't set the other properties correctly).