How to link assignments to a rubric via API

[nschutz]

We have a rubric, Rubric A, created at the account level. There are currently lost of assignments linked to Rubric A. I understand that if a rubric is aligned to more than one assignment, you cannot edit the rubric. So instead of editing Rubric A, can we create a new rubric, Rubric B, and re-align the assignments that are linked to Rubric A with Rubric B via REST API?

[James]

 @nschutz , 

Let me see if I have this right. You want to 

1. Create a new rubric B at the account level, but this can be done through the web UI, it doesn't have to be done through the API.
2. Use the API to find every assignment that rubric A is used in all courses in an account.
3. Use the API to download all of the existing rubric data, including comments, from those assignments
4. Use the API to unlink rubric A from those assignments
5. Use the API to link rubric B to those assignments
6. Translate the scoring on rubric A into an equivalent rating on rubric B
7. Use the API to fill out rubric B information for each of those assignments

Did I miss anything or put extra things in there?

Here are some questions I have

1. Why?
2. Are outcomes or learning mastery tied to these rubrics? That might add extra considerations.
3. Are you sure that the rubrics all tie back to the same rubric ID and that you can match it off that rather than matching off of text of the questions? This doesn't change the overall answer, but I have found several places that the same rubric occurs with different IDs, but these were generally course rubrics that were copied from term to term. Consider it a warning to be careful.

My initial reaction, based on experience but without checking each step is that this is a bad idea and the wrong approach to take.

Generally speaking, it is a bad idea to go back and change the data from the way it was at the time it was collected. That might be a part of the reason why Canvas doesn't allow people to edit rubrics.

You didn't go into the cause, but maybe there was a typo in a rubric that you want to fix. You don't go back and change it in old rubrics. Instead, you do the translation at the analysis time. Or you download the data, clean it up, then make it available to other people for analysis, but the original data is kept intact.

If you have to fix a rubric once, you may have to fix it again. If it's a global evaluation rubric, that might change over time. At some point, you might add questions or take questions away. When this happens, you don't mess with the original data, you find some way to combine the data together for analysis. It may be that you have a question that was asked in 2016 and 2017 and then no more while another one didn't appear until 2018. You don't want your rubric to have rows that hold historical data but are no longer used and you don't want your rubric to delete criteria that are no longer used and then throw away that data.

I don't know what this rubric uses, so what I write here may not apply in your particular case.

The Submissions API allows you to grade or comment on a submission, which would allow you to supply the new rubric information, but it is likely to update the graded_at and graded_at values for the submission. This may or may not be an issue, but if I want to go through and do some analysis of how long it takes instructors to grade their work, I've just blown that possibility for anything using this rubric. If there's any kind of question about who completed the rubric in the first place (example: a course has multiple graders) that has been replaced with the ID of the user who ran the script.

You may not think that changing existing rubrics is what you wanted to do. If you have blueprint courses or master courses and are not concerned with updating the rubrics in existing courses but only new ones, that's a different issue. I'm not sure from the original post whether that's what you're after.

However, if anyone copies a course from one term to another, then they're copying the old rubric. You would then have to go through and link to the new rubric after the course is copied. If you change the rubric link in the old course, then there is a good chance that some of the existing rubric information becomes unavailable.

For example, I went in to SpeedGrader and graded a person using a rubric. Then I changed the rubric, reloaded SpeedGrader. and graded a different person. SpeedGrader only uses the single rubric -- the most current one. Even when I went to a user that was already graded with the old rubric, SpeedGrader showed the new rubric with nothing marked and the student getting 0 points. The API for the assignment only returns the second rubric, which is probably why SpeedGrader shows just the one rubric. If I pull up the submissions for an assignment and include[]=rubric_assessment, the original rubric is still there for the first student, but there is no tie back to the rubric the IDs are associated with. I can fetch every single rubric in an account and a course and go through and get the full rubric for each one of them and look for the criterion_id, so I was able to find it, but it was more prolonged.

The short version of that is that by going through and updating the assignment to refer to a new rubric when the rubric has been used for grading is that you will likely be unable to retrieve that information through the web interface. If you go into a previous course and look at that assignment and pull it up in SpeedGrader to look at something for a grade appeal, it would appear blank.

If you change the rubric in existing rubrics and don't fill out the new rubric, you lose the ability to pull up the rubric information within Canvas. If you do fill out the new rubric, then you lose information about when it was completed and who completed it.

The only way I see to do it reliably without causing some potential problems is to only make the change for assignments that have not been used. This can be in master courses, in blueprint courses, or in courses that haven't used that rubric yet.

Even this is not simple. At our school, we don't use master courses or blueprint courses much, so our faculty are very likely, at any given time, to copy information from an old course over and that point, they're back to using the old rubric. This may happen before the start of the semester or it may be that they selected content and did a partial copy before the course began and then decided to copy that one over later, so it could happen at week 12. There's no easy way to know that, although you can go through the course audit log and look for when things were copied_from another course. That API can be limited by date, so you can get the things that have recently happened since the last time you checked, but it's still time consuming.

In a perfect world, rubrics would never change. Alas, our world is not perfect.

[nschutz]

Thanks James! I so appreciate your explanation and giving me lots to think about as usual. :smileygrin:

Some clarification to my original question:

We have master courses we use to copy to live courses for each semester. These master courses have rubric A linked to their assignments but none of these assignments have any related submissions. Since we can't edit rubric A, we created rubric B. We can relink the assignments in master courses to B from the UI but the API doesn't seem to provide that option. The Assignment object has the rubric_settings property, but the update assignment doesn't allow update to this particular property.

[James]

 @nschutz ,

Thanks for the clarification. I didn't want to get into a lengthy digging session until I had a better idea of what you were after.

The lack of API support for creating / editing rubrics is what I faced when I created the Importing Rubrics from a Spreadsheet ‌ Canvancement. I had to use an internal call, not an API call to do it. In that case, I did not look at associating that rubric with an existing one because the point was to create a new rubric.

When you associate a rubric with an assignment, it uses a different internal non-documented, non-API call. It's a POST to /courses/1234/rubric_associations. It has a payload like

rubric_association[association_type]: Assignment
rubric_association[association_id]: 8185038
rubric_association[rubric_id]: 631694
rubric_association[purpose]: grading‍‍‍‍

The values will, of course, change, but that's the kind of form data you need to send. The association_type and association_id go together -- if it's an assignment, then the id is the assignment ID. The rubric_id is the id of the rubric that you want to use. Basically, the safest way to know what to do is open the developer tools, switch to the network tab, and then actually switch one of rubric A to rubric B and look at the Headers section and the Form Data.

Since it's an internal call, it's something that will need executed within a Canvas session, JavaScript is the most likely candidate for the language. You can open the developer tools in the browser and paste the code in the console.  That is, you don't have to create a UserScript if you don't want to do so or don't need finer control.

We're not using blueprint courses and I haven't looked into them much. I don't know if there's an easy way to select all the blueprint courses, but you hopefully have some naming scheme that you could search for and get a list.

It sounds like your flow would be to pick a course, download all the assignments looking for the ones that are using account rubric A, make the internal call to replace it with rubric B. You would repeat that process for each course that needed it. If you can automate the list of courses that need updated, then you could have the entire process run automatically.

This is something I would definitely test in your beta or test instances first.

[nschutz]

James,

I am looking through your collection of Canvancement on github and wondering if I can use one of your userscripts as a starting point? import-rubric.user.js seems to be a good option as it has both API calls and non-api calls unless you know of a better one to model after. Just want to verify with you before moving forward. :smileygrin:

[James]

 @nschutz ,

All of my code on GitHub is released under the ISC license, so feel free to use it. Some people have taken what I supplied and used it to launch some amazing things, far more than I have time to develop.

As far as licenses go, I wanted some control over things (so I didn't put it in the public domain) so that people would know where the "source" was instead of someone else taking it and claiming it was theirs. But I'm not a lawyer and I'd rather not spend my life savings paying one. I really don't care much about the attribution, it's mostly the "don't sue me if it breaks something" part I'm interested in.

[nschutz]

James,

Thanks so much for providing the payload and endpoint. I was able to get it to work nicely and it will save us lots of time. However, how or where do you find the "non-documented, non-API call" information? It will be helpful to know what's available for writing other user scripts. 

[James]

I usually go into the developer tools in the browser (F12) and go to the Network tab. I then do what I need to do in Canvas and look at the information that is sent to and returned from Canvas.

Since I wrote the original import script, Canvas has added some rubrics API endpoints. I have not had time to experiment to see if I could rewrite the Rubric Importer to work with the API instead of the internal call.