[Peer Review] Peer Evaluation for Group Projects

[sgreen10]

Every semester we need the students to create a peer evaluation for their group projects. We would like for students to be able to rate their group members contribution to the group project. This feature should allow the instructors to set up the peer evaluation for students to either rate themselves or students to only rate their peers. The instructor should be able to choose the type of evaluation students can give their peers like numerical rating, likert scale rating, text comments etc. There should also be an option to include the peer evaluation as a graded assignment.

Another nice option would be to automatically add the peer evaluation to group assignments. This would allow points to be deducted from the group assignment grade automatically. Usually, we had to divide the total points for the group project across two assignments: 1) the project and 2) the peer evaluation. With the option just described the peer evaluation would be included into the group project and there would no longer be a need for two assignments.

We have done the peer evaluation in survey builders like Qualtrics in the past but it is not the best option for several reasons. Student's get confused when they have to leave Canvas to complete assignments on other websites. Their activities don't get tracked in Canvas so they don't know if it will be graded. Qualtrics (at least the survey version we have at our University) is not designed to capture feedback in a round robin format needed for peer evaluations. We have to export the results from Qualtrics, clean the data, do some calculations then input the grades in Canvas. It becomes a really tedious process.

[iys2]

Lisa, thank you for sharing this process as I miss incorporating this for group projects. I really hope that this or something similar to this could be incorporated into Canvas.

[lmd5adm]

Thanks Louise.  I've edited my post.  I noticed that some of my information did not copy over properly.

[alh245]

Another great feature of this (and what I would like to see in Canvas) is the self-evaluation piece. 

Each member of the group evaluates their team mates as well as themselves based on their contribution to the project. 

Ideally this should go back into the gradebook directly.

Working well on teams is an important part of our curriculum, gives experience similar to real-life, and leads to enhanced learning.

Thanks,

Amy

[iys2]

Amy, this is a great point and I would like to see this as well. This was also an option in the Peer Evaluation tool that Lisa presented. I truly miss this tool and am hoping that something could be integrated into Canvas. This is a chance for members of a team to speak up and provide feedback about the work they have completed as well as others on their team. This is also another way that instructors can collect feedback about the assignment itself and could be potentially used to make improvements in the actual assignment. Great point - thanks for mentioning this.

[BKINNEY]

I'm voting for this, for about the third time.

We have a solution in place through LTI that does everything you have described,  including (optionally) pushing a new gradebook column back into Canvas, but we can not share it easily for a wide variety of reasons. Even to put the code on GitHub, we'd have to make significant changes to the code base. We actually applied for a Canvas grant to do it a few years back, but we didn't win. So, we're happy with our solution, but still hoping that Instructure will come through with something.

One thing you might want to think about adding to your request is anonymity. Most of my users would not be taking advantage of our system if individual evaluations were not anonymous, in the sense that evaluatees don't know who has said what about them. Instructors do. Since the existing Canvas Peer Review is not anonymous, you shouldn't assume anonymity would be part of any group-enabled option Instructure might add.

[RobDitto]

 @BKINNEY , thanks for all you have been doing to keep this need alive as a feature idea!

Out of curiosity, what has been UDel's experience with the "Peer Reviews Appear Anonymously" setting?  While the notifications were finally anonymized, for Wharton what remains is that reviewers can still download submissions and might be able to spot the submitter's name in document properties.

[BKINNEY]

 @RobDitto ,

I don't have any personal experience to report with regard to the built-in tool's anonymity. I have no reason to suppose it behaves any differently for us than it does for you. We recommend our own inhouse tool for faculty looking for peer evaluation (which is always anonymous), and the Canvas tool for those who want peer review. For the most part, I see the former, since I created the inhouse tool. Doesn't mean I never field questions about Peer Review, but I've never heard from anyone who was concerned about anonymity in that context. If we did get such an inquiry, we can easily slide those folks over to our anonymous tool.

[n_a_gordon]

The WebPA peer assessment tool is designed to meet this kind of requirement: see WebPA

there are LTI integration for it: though it requires that an institution host WebPA itself.

For information on the LTI facility for WebPA see

•http://www.spvsoftwareproducts.com/php/webpa-lti/ which means that WebPA could then be used as an external App.

[vrs07nl]

WebPA is a leader here in UK for peer assessment etc

It may be free BUT it is resource heavy in that you need a data base and host site, someone to manage it etc - for us that is IT who do not have the resources to even help out with a pilot at present... 

We are using surveys that we embed into courses but then extract data vis csv files etc... all time consuming

SO I am all for this feature idea

[BKINNEY]

Yes, the hooks are there, but only if you have API access, since LTI grade passback isn't useful in a context where the logged-in student is NOT the one receiving the grade. We (University of Delaware) built a standalone Peer Evaluation Tool we've been able to hook into Canvas using the API + LTI. It does everything Shana has described, except that the PE grade is not automatically incorporated into the project grade, but rather appears as it's own gradebook column, which you can do with as you will.

Regardless, I don't think the fact that it is possible to build a third party solution should prevent Instructure from considering this request. I have been unable to share my app outside our university for a variety of reasons, so the fact that our tool exists does not help anyone else. Please consider this request. Canvas has such a great set of collaboration tools, it is really a shame not to close the loop.