[Peer Review] Assign Peer Reviews by Student Group

[isherman-youngb]

This one has been suggested before, but I'd like to suggest it again: one of the options for setting peer reviews of assignments should be "by student group"--all members of a group are automatically assigned each other's assignments for peer reviews. This would save me a lot of setup--manually setting up peer reviews currently takes about 20 minutes per class, per assignment.

[scain]

 @James  I've been looking through the APIs and I'm wondering if there isn't a way to do something like this using API calls to get the user ids for a group and then compile that data so that everyone in a group is assigned to peer review everyone else in the group. I feel like this should be possible in some manner, but I'm too new to the API scene to be able to tackle it myself. Do you have any pointers for getting started on something like this?

[James]

 @scain ,

The information is there -- at least I think so -- it's been a year since I've done this and it may have changed. A lot of what I did was manual, but I think I obtained the list of group members from Canvas API but assigned the peer reviews manually.

My situation is a little different, but it can probably be modified to work with what people want to do. I used to have a big project at the end of the statistics course. It is a group project and I have two sections of statistics in the same Canvas course. Since the students are going to present the information to the class, I didn't want the students in their section to do peer reviews, but I felt that the papers could really benefit from peer reviews. My solution was to assign a peer review on a paper from section 1 to a student in section 2 and a peer review on a paper from section 2 was assigned to a student in section 1.

Each student was given three papers to review. At the time, Canvas required that you assign a peer review by the name of the person not the name of the group, even if it was a group assignment. I don't know if that's still the same or not. But I had to take the "Healthcare" group from section 1 and assign people in section 2 to grade the papers of Tom, Dick, and Harry, not the group paper of "Healthcare".

I wrote code that would randomly assign the peer reviews, making sure that it did it first by group so that the three peer reviews were for different groups. I tried as best I could to balance the distribution so that each group had roughly the same number of reviewers and so that people in a group in section 2 weren't reviewing the same 3 papers so they would have more to pull from and depending on the number of groups and the number of people in a group, it might be possible for a group in section 2 to end up reading and reviewing every paper from section 1.

When it came time to assign the peer reviews, I would cycle through the list of people in the group and assign peer reviews randomly to Tom, Dick, and Harry in roughly equal proportions.

All that code generated a spreadsheet of students and their designated peer review that then needed transferred into Canvas. At the time, I wasn't very literate on group stuff in the API and I never tried to peer reviews using the API. There was a point pf decision making: "I can spend hours upon hours writing code to do this thing that needs done once a semester or I can spend 10 minutes just putting people into the groups."

I stepped back and chose compromise because I really hate repetitive tasks. I spent one hour writing an autoHotKey script that would transfer the data from the spreadsheet and create the peer assignments through the browser.

There is a create peer review endpoint (technically listed twice in the documentation, but the second one is really delete peer review), so in theory, you should be able to use it instead of the autoHotKey. I just stopped using the peer reviews and went to three smaller projects without them so I haven't needed to pursue that.

[erlend_thune]

Hello!

We've implemented this as a part of our Mathematic MOOC javascript power functions which we trigger by adding ?mmpf to the url.

You can find the code here: frontend/powerfunctions.js at master · matematikk-mooc/frontend · GitHub

The code is dependent on other functions and templates which you will find in the parent folder: GitHub - matematikk-mooc/frontend: Custom JS and CSS frontend for inclusion in Canvas

You might also be interested in the student progress functionality which can be found in the same file.

You can read more about the project here: Matematikk-MOOC by matematikk-mooc

[scain]

Wow! That's awesome! I've been looking into it a fair bit this morning, but I think it might be a bit too complex for me to implement. Very cool seeing that you've built this though!

[Renee_Carney]

  Idea is currently in Product Radar Learn more about this stage...

[andrew_t_levin]

The students in my macroeconomics class are working together in groups, where each group analyzes a different country and gives a sequence of presentations to the entire class. For each presentation, each group is paired with a different "partner" group that reviews the preliminary draft of their presentation handout, i.e., group A reviews the draft handout of group B, and group B reviews the draft handout of group A. Those peer reviews must be completed within 48 hours, so that each group can incorporate those comments into the final version of its presentation handout. To provide an incentive for each group to produce a high-quality draft, the peer reviewers also rate the overall quality of the draft handout and the average of their ratings has a small weight in determining final grades. Moreover, each group rates the quality of the comments received from its peer reviewers, thereby providing an incentive for each reviewer to provide constructive comments. Finally, it should be noted that the group pairings alternate for each presentation, so that every group engages with a sequence of other groups throughout the term.

This design is intended to foster active team-based learning, but it's quite cumbersome to implement in Canvas, because peer reviews can only be assigned to individuals on other individuals' assignments, and Canvas doesn't allow any deadline for completing the peer reviews. (In my course, ) Therefore, several Canvas enhancements would be ideal: (1) Allow the instructor to designate a group (not just an individual) as the peer reviewer for another group's assignment. (2) Allow the instructor to specify a deadline for the submission of peer reviews. (3) Allow peer reviewers to submit a rating on the quality of the document being reviewed, where that rating may be visible to the instructor but not to the student(s) who produced that document. (4) Allow the recipients of peer reviews to submit a rating on the quality of the comments received from each peer reviewer, where those ratings may be visible to the instructor but not to the peer reviewers.

[scain]

andrew.t.levin,

This sounds like an excellent project! Your suggestions are spot-on and would solve a lot of issues that I have with the peer review tool as well. I'm an Instructional Designer in a business college and would love to hear more about your experiences with this project, even stepping away from the hurdles of making Canvas do that type of project. My biggest concern with encouraging faculty to do that type of project is that it seems like it would be especially difficult to get the reviewers to be critical of the other group's work when part of their grade is also based on how that group rates their review. For instance, in your example above, if group B were to give group A a poor rating on their draft, wouldn't group A then be likely to give group B a poor rating on their review comments? Do you spend time beforehand working with the groups to help them understand the traits of a quality draft and peer review? What does that look like? Have you run into the problem I listed here? If so, how did you handle it?

Best,

 @scain ‌

[alexander_pho]

Andrew you've definitely nailed all the points we're looking for and added some (3) and (4) that we hadn't thought about, but could definitely find extremely useful in the long run as team-based learning continues to ramp up. 

For our professional programs we tend to work a lot with discussions, and as a workaround to true intra-group peer reviews and due dates for peer reviews, we end up creating two separate graded group discussions and manually assign peer reviews to group members only. 

Here's the feature request for the Due date for completing the peer review:

https://community.canvaslms.com/thread/3207 

Best,

Alex

[tslee]

me too! this feature would support "group projects".   

two more feature requests come to my mind 

 1.  ability to peer review within my group for discussion forums

 2.  I like that in peer review page as teacher I can see who has completed and hasn't completed their peer review. Is there way to also see this in the course gradebook (via "Grades" link)?   I like the way I can see in "Grades" link whether a student has submitted an assignment (teacher sees a icon place holder in student's row) but I am not seeing another icon for whether student submits peer review.    Having 2 places for this visual cue would be helpful.

[James]

I've just released a user script that will automatically assign peer reviews to everyone in the same group as the user. It is not as flexible as some people wanted, but it may meet the needs of some.

https://community.canvaslms.com/docs/DOC-14465-assigning-intra-group-peer-reviews