cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
vmiletic
Community Member

Automatically grading quizzes with an external tool

Suppose I want to have an quiz in Canvas, where each question (let's say Essay or File Upload type to start somewhere) is graded by an external software tool. I'm curious about two things for this scenario:

  1. Can I get Canvas to call the external software tool via some kind of API upon student's submission and get the response and use it for grading the question? If yes, can I do so without modifying the source code, i.e. by writing an extension of sorts?
  2. Should I instead get an external tool to use Canvas API to grade the submitted answers (this Quiz Submissions - Canvas LMS REST API Documentation  if I understand correctly)? Can I use this API on https://canvas.instructure.com/ , i.e. without self-hosting?

Thanks in advance.

Labels (1)
10 Replies
pklove
Community Champion

I don't think you can do (1) without modifying Canvas.

You should be able to do (2).

You might be interested in some of the discussion at: https://community.canvaslms.com/message/91588-lti-external-assignment-with-canvas-submission-system 

stuart_ryan
Community Coach
Community Coach

Hi  @vmiletic ,

I agree with Peter, 2 should be completely doable.


I have seen something recently in the API documentation that ?may? allow you to achieve one. There are now web-hook subscriptions available Webhooks Subscriptions - Canvas LMS REST API Documentation  where you can subscribe to know that a change has taken place. 

You could (conceivably) use the web hooks to have your external tool be notified of a new quiz submitted (using the QUIZ_SUBMITTED Event Type) which you can then have your tool do the rest of the interactions with the API to push the mark back.

I have never authored something like this before, but, conceivably quite doable. I would note that Canvas in this case won't call the external tool and expect a response, it will more call the external tool to say 'hey something has happened' and then rely on your tool to do the heavy lifting once it is told of an event (such as doing the grading, and then using the parameters in the initial notification to mark using the API).

I would be very curious to see how you go if you can get this up and running!

Hope that helps,

Stuart

pklove
Community Champion

Oh, I missed the Webhooks.

It would be interesting to know if anyone is using them.

Agreed! They are only very new, I haven't even had a chance to tinker or heard of anyone that has. Would be very interesting to see how they work and play with a realtime integration. I will keep my ear to the ground at InstructureCarn this year to see if anyone has started using them and might have some examples.

Wow, thanks for quick response(s)! I checked the other question and I'll check the Webhooks Subscriptions API.

I have no problem doing the heavy lifting myself, the idea that I could be notified upon submission and don't have to trigger grading manually is already more than I expected Canvas to do.

I plan to open source the tool and will gladly share the code here.

pimmel2
Community Participant

So, let me get this straight.  There are apps out there that will grade essay questions / quizzes?!  What are folks using?   Very Interested.

Pat

vmiletic
Community Member

I assume this question is directed my way.

We do not plan to grade free form essay answers, at least not for now. However, we do plan to grade questions that expect somewhat structured answers, i.e. a shell command, a couple of configuration directives, filling a table, a list of words/numbers, or something similar.

I'm not aware of any tool that would work for our use cases. I'll be developing my own tool (likely in Python because that's what I work with most of the time) and I will post the source code.

Is there any updates on this project or tools you've developed or found useful in the venture to automatically grade coding assignments?

Unfortunately no, in the end for that course I switch to Moodle and used Preg question type instead of developing something custom.