Anonymous peer review

[CommunityTeam]

  This idea has been developed and deployed to Canvas

  Idea will be open for vote May 5th, 2015 - August 5th, 2015  Learn more about voting...

As an instructor, I'd like to have the option to assign anonymous peer reviews on assignments. In an anonymous peer review, students would know neither the name of the person whose work they are reviewing nor the name of the person who reviewed their work. Instructors would know who reviewed whose work.

transferred from the old Community

Originally Posted By: Neal Legler
Special thanks for contributions by: Stephan Deban, Lindsey Mercer, Jared Chapman

  Comments from Instructure

October 2015 Update!

Part 4 of Anonymous Peer Review is Complete

Find more information in the Canvas Production Release Notes (2015-10-10)

[10618071]

I see this as a key feature in effective peer review.

In the spirit of "skating to where the puck is going to be", please take note that other platforms that have been doing this for years are recognizing the need for asynchronous peer review and are working in that direction.

As you build your platform please consider asynchronous work flows as well. I have written proposals that are being used by other LMSs and would love to share them with Canvas. In a world of Moocs, corporate training, competency based training, quest based training, choose your own adventure style training, and cafeteria style training, it is no longer reasonable to assume that students will be in the same place in the course as all other students. We must accommodate a variety of paths and speeds through content while at the same time allowing for social interaction and engagement in learning.

I added the proposal as a feature request but it got archived as too ambitious. Hopefully the team working on this will look the proposal over and keep it in mind as they design so they can head this direction.

[browning339l]

The anonymous peer review should include the comments box and disable sending emails that a comment was left that identifies the commenter/reviewer.  A feature that includes the option to average the peer reviews would also be nice......

[fidalgok]

Hi all,

As the anonymous peer review is being built out I was hoping to point out an odd behavior I noticed while using this for assignments. In the release notes from July: Canvas Production Release Notes (2015-07-18) there was mention of Crocodoc not allowing for annotation by instructors while anonymous peer review was enabled.

• Note: In anonymous peer reviews, users cannot create annotations in Crocodoc (including instructors).

I wanted to bring this up because I had an instructor who wanted to add annotations to his students documents which apparently doesn't work while anonymous peer reviews are turned on. It would be nice if the instructor retained the ability to add annotations to documents while peer reviews are anonymous. I could be wrong but it would seem that this feature should only restrict students from knowing who the other students are that they are both (a.) reviewing and (b.) being reviewed by rather than preventing the instructor from annotating and commenting on the submission.

Below are screenshots of anonymity being turned on and off and how that affects the Crocodoc viewer.

After turning off the anonymous peer review this is what the teacher sees, which if possible would be great to have while anonymous peer review is turned on.

I noticed that in the crocodoc api there are ways to adjust if the document is editable or not by toggling the editable parameter. Is there a way to turn this on when you are viewing the assignment with a role of teacher? As of now when anonymous peer review is turned on that document parameter is set to false.

If this needs to go to a feature request I'm happy to re-post in that area, I just wasn't sure if this was part of the current build out for anonymous submissions or a completely new angle on the request.

[browning339l]

since this is closed for voting, I wilI comment here. I am in favor of

completely anonymous peer review (ie, no email send to reviewee with

name of student reviewer when they leave a comment). I did not have

issues leaving comments on my students speed grader comment box even

with anonymous turned on.

I would like to know if the comment is left in the rubric and not the

comment box if the reviewer name is still identifiable.

[Renee_Carney]

Part 4 of Anonymous Peer Review is currently on beta!

Please take a few minutes to check it out!  How do I access the Canvas beta environment?​

[10618071]

It is not clear from the interface, are both the submitter and reviewer anonymous?

[kona]

I have the same question! It's definitely not clear.

[Renee_Carney]

It will be clear soon!  Erin is getting updated screenshots that show the anonymous peer review is double blind!

[erinhmcmillan]

Hi Kyle,

We had to change that component with anonymous peer review because our engineers said Crocodoc doesn't support anonymous functionality. That's why editing is disabled in the API.

Hope that helps,

Erin

[erinhmcmillan]

Hi Jared,

Part four with notifications was the last component to making sure anonymous grading is truly anonymous. I asked for a screenshot of what the anonymity looks like in notifications from one of our QA engineers. The metadata shows that the from name is anonymous, as does the message content itself. (S1 stands for Student 1, and the assignment name is anonymous peer review.)