cancel
Showing results for 
Search instead for 
Did you mean: 
vkgupta
Surveyor II

Multiple graders cannot use rubric in Speedgrader

Is there a change in the way rubric grading works within Speedgrader so that multiple graders can no longer grade different parts of an assignment?  

In the past (as late as October/November 2017) it was no problem for each grader to select a grade from the rubric. The assignment points would be the total of these grades.

However, now when grader one goes to grade problem 1 of the assignment where problem 2 has already been assessed by different grader two, the first grade disappears from the rubric.

We always ensure that the two graders are not grading the same assignment at the same time.  Interestingly, the comments from each grader do not disappear, only the grade chosen from the rubric.

Is there a workaround to this issue? Anyone else experienced the same thing?

Thank you for any feedback

25 Replies
kblack
Community Member

Hi vkgupta‌ - I was intrigued by your question only because I didn't realize that this could even be done before in Canvas!  So if I'm understanding you correctly, your process works something like this:

-TA #1 clicks View Rubric for an assignment and grades a few row in the rubric, does NOT submit a grade, but saves the rubric.  The end result (using an example of an extremely simple, two-row rubric ) is that when TA #1 is finished, the rubric looks something like this, with just the top row completed:

Rubric after first TA grades it

-When TA #1 is finished, TA #2 comes in to finish the job, grading the next row(s) on the rubric. 

As you state, when I just tried doing this, TA #2 might be able to SEE the work of the first TA, but once s/he clicks on View Rubric again, s/he is starting from scratch, and the earlier work essentially disappears, with TA #2 looking at a completely blank rubric.  

Assuming this is all correct, I assumed that one person's feature might very well be another person's bug.  Thanks to your time frame that you spelled out, I believe I found it.  In the December 9, 2017 Production Release notes, under Fixed Bugs>>SpeedGrader, there is this statement:

Multiple Rubric Assessments

When a rubric is used for grading and saved in SpeedGrader, SpeedGrader only creates one assessment result.

 

Explanation: When an instructor used a rubric for grading and clicked the Save button, clicking the Save button multiple times created multiple assessments listed for the same instructor. Canvas code has been updated to disable the Save button until SpeedGrader has processed the rubric assessment for the student.

So, it appears that Canvas viewed this as a bug! The one thing that IS still available for you--though it would not work exactly as your current scenario uses it--is Moderated Grading, which is covered in the Instructor's online guide starting here:  How do I set up a moderated assignment to be graded by multiple reviewers?  and then continuing on from there for a few pages.  Unfortunately, even Moderated Grading will not automatically add up partially-completed rubrics left from your TAs.  The Moderator (usually the instructor) can choose from an existing TA score or create their own, which is then posted as the final score.  So using Moderated Grading would require you to add up the rubric scores on your own.  (Plus, there's a limit of two TAs and one Moderator.)

Assuming that I am correct in my answer to your immediate question, you might want to pose this as another question in the Community to see if anyone with a smarter mind than my own might be able to think of some work-around that matches what you want to do now that Canvas no longer allows you to do what you did previously.

Off-hand, the only thing I can think of at the moment would be to have the second TA note what score(s) the first TA recorded and then incorporate those when s/he reviews the work so that the end result is the same.  It puts more work on the second TA, to be sure, especially if you have a rather long, complex rubric.

I hope this helps a bit in explaining the "why," though not the "how."

Ken,

Thank you very much for the detailed reply.  You hit the nail on the head. 

The scenario you described is precisely what I used to use where multiple TAs could grade separate problems in the same assignment so that the grading of work for a large class (>150) could be done efficiently.  If Canvas treated that feature as a bug and removed it, I am sorely disappointed in Canvas yet again.

To confirm your question - a TA would choose the score from a rubric row associated with the problem they were grading without impacting another TA selection of a different row of the rubric, the scores would add automatically, and at the end I as the instructor would be able to release the combined score.

I did not have to click through hundreds of assignments, did not have to match scores sent by TAs with each name and enter into the rubric, could divide grading tasks and keep them manageable.

 I am looking at the moderated grading you indicated to see if it can accomplish what used to be straightforward earlier.  

All I can think is that this change is another step backward when using Canvas for assignments and I may end up not using it anymore.  I have already severely limited my use of the Canvas quizzing tools because of the lack of features (multiple numerical answers, multiple calculated, time limits and attempt limits on each question, ...) and their inability to update features in spite of several years of promises.

kblack
Community Member

Hi Vinay -

I'm sorry to hear that the Canvas experience has not gone well for you, though no LMS will ever address everyone's specific needs.  But regarding quizzes, there is a new quiz engine that is due to see the light of day sometime this year:  Priority: Quizzes.Next .  While I'm not sure if all of your needs will be addressed, there is a video overview at the latest annual gathering of Instructurecon here:   .

If Moderated Grading is not the answer for you, you may want to submit a Feature Idea (How do I create a new feature idea? ) later on.  I'm sorry I could not help more other than confirm that what you did no longer works.

Ken,

This is a problem even when a TA does submit the grade.

For example: We have teachers who have TAs grade the assignment with rubrics, then submit the grade. Later, the teacher may want to edit the grade by changing one of the rubric rows. However, when they click the "View Rubric" button to edit that one row, the entire rubric gets reset to zeros.

Matthew

Hi Matthew,

I did not realize this.  But I just checked and you are right and my TA grades blank out. 

So essentially, the instructor has to note down the prior grades on the rubric before attempting to change it on the rubric.  Or probably only the TA who graded it originally can change it.

Vinay

P.S. I looked into moderated grading.  Judging by the posts, it seems to have several issues regarding posting grades and then not being to change the grades, not being able to retrieve grades of reviewers (TAs), etc.  Plus of course it does not solve the issue I had. So for the time being I am staying away from it.  

Vinay,

That's correct: the only way for teachers to edit a TA's rubric grades is by writing them down (or doing a screen shot), and re-entering the same things except for what they're changing. And yes, the person who originally edited the grade can still edit it.

And you're right: Moderated Grading doesn't help this situation - a teacher can only approve the grade, not edit it.

Matthew

Sorry...just now re-joining the conversation.  Moderated grading can work, but you definitely need to know how it works first.  Though as I said earlier, it indeed is not ideal if you just want TAs to grade from part of a rubric, since the moderator would still then have to go through each partially graded rubric and fill the final one out from scratch.

You may want to turn this in to a Feature Idea, I'm afraid (https://community.canvaslms.com/docs/DOC-1330-how-do-i-create-a-new-feature-idea ).  It may get some traction from people at larger schools where this could be an issue or in similar situations to both of yours.

With Moderated Grading, you can approve a TA's grade, or make your own grade with a rubric, or choose between the grades of multiple TAs. But what you cannot do is edit the rubric grades of a TA, and change just one row (without entering the entire thing again).

Thanks for the suggestion Ken

However, I am largely disillusioned with the Feature Ideas given the "popularity" mechanism for approval in the first place and then the long timeline for any implementation.  It is much faster and easier for me to find another solution. 

As Tasha pointed out below, Instructure has simply disabled the earlier "feature" thinking it was a "bug" without realizing how useful it was.  They could just easily reverse it.

I appreciate your time to engage in this discussion.