[Peer Review] Peer Evaluation for Group Projects

[sgreen10]

Every semester we need the students to create a peer evaluation for their group projects. We would like for students to be able to rate their group members contribution to the group project. This feature should allow the instructors to set up the peer evaluation for students to either rate themselves or students to only rate their peers. The instructor should be able to choose the type of evaluation students can give their peers like numerical rating, likert scale rating, text comments etc. There should also be an option to include the peer evaluation as a graded assignment.

Another nice option would be to automatically add the peer evaluation to group assignments. This would allow points to be deducted from the group assignment grade automatically. Usually, we had to divide the total points for the group project across two assignments: 1) the project and 2) the peer evaluation. With the option just described the peer evaluation would be included into the group project and there would no longer be a need for two assignments.

We have done the peer evaluation in survey builders like Qualtrics in the past but it is not the best option for several reasons. Student's get confused when they have to leave Canvas to complete assignments on other websites. Their activities don't get tracked in Canvas so they don't know if it will be graded. Qualtrics (at least the survey version we have at our University) is not designed to capture feedback in a round robin format needed for peer evaluations. We have to export the results from Qualtrics, clean the data, do some calculations then input the grades in Canvas. It becomes a really tedious process.

[ltscal2]

There's another one: iPeer. I believe this is locally developed by the University of British Columbia and is also open source.

Im also considering looking at Fluid Surveys. Its data is stored in Canada so it by-passes the need for consents and all that business. Plus it looks pretty slick

[ltscal2]

Hi Canvas Community,

I was wondering if there is an update on this feature request. I am currently looking for a solution and could possible be a commercial product. However, if Canvas is going to build this feature request in, it would be most idea than to try to build a LTI. In my search, I have found VERY few commercially available products that will do Peer Evaluation (as opposed to Peer Review). In addition, it would seem that most schools have created their own proprietary system for their own institutions. All of these proprietary systems are actually the same thing. Each one from what I have gleaned upon in my research is actually a different version. Basic needs include:

+ anonymous feedback

+ ability to review all group members through a single page

+ integration into the LMS

+ output reports and analytics in CSV format

...etc etc....

The other question is : is this area oversaturated that it is not viable for commercialization. Normally I would be looking for "free" open source software that we can use so we dont have to go through the legal hassle of consent and privacy schedules. However, support is very resource intensive as someone said about WebPA.

Hopefully there can be a unifying solution such as Moodle Workshop or BB Peer Assessment that can be implemented into Instructure Canvas or if someone can suggest a viable commercial product for us would be great. So far I can only see that CATMe may be the ONLY solution because Teammates app not only is it located in another country but it requires a google account to use which violates all our local privacy legislation.

[kmeeusen]

Hi  @ltscal2 

I am concerned by this statement, "Its data is stored in Canada so it by-passes the need for consents and all that business." How does it not violate student rights regardless of where data is stored? That is a slippery slope I would not recommend.

Correct me if I have made a misconception somewhere in this.

KLM

[ltscal2]

In the context where I am concerned, our legislations is that we can by-pass consent forms by signing a Privacy Schedule Agreement with a vendor if the data is stored in Canada. I am not sure what the legislation is in Ontario, Quebec or other states. But as far as the context I am working this is what I have been told.

[mcsmith]

We so appreciate everyone's dialogue. Seeing the different use cases really helps us understand the various needs and use cases. We do not have the capacity to work on this in the next 6 months so it will be archived at this time. However, we've seen peer evaluation ideas pop up consistently which gives us great data for prioritizing. 

[sjuve]

This is an important tool to develop for Canvas! That the students can evaluate group projects, how the quality of contributions are and how group project contribution can be improved is important, not only for the individual student but for the group process itself. Really hope this will be developed soon!

[ltscal2]

This is a high need for us that our institution is going to create a Canvas integration with an external product within the year through a series of grant funded projects. 
Its quite amusing though that a lot of academics dont know the differences between peer review and evaluation (group peer evaluation).
Although this project is internal, I would be quite interested to hear what other institutions needs are around peer evaluations in addition to what has been mentioned. One of our requirements is to produce a research project/paper/presentation after the grant.  Feel free to contact me.
 

[n_a_gordon]

The LTI connector I mentioned in my earlier post enables integration - the issue is hosting WebPA. You may like to join the WebPA community discussion  site [JISCMail - WEBPA List at WWW.JISCMAIL.AC.UK ] to get further input and potential collaborations. I have published a few articles around WebPA, and there are a few case studies etc. at WebPA  

[ltscal2]

@Neil Gordon:  Thank you for the information. I have joined the WebPA mail list and they have been very helpful. For our institution local hosting works for us (we even host our own version of Canvas!)

I dont know the technical side, but I am told that the LTI doesnt work 100% -- whether that is because of our install of Canvas or something else. However, integration one part, we are looking for other items as well (gradebook integration, Single sign-in) and perhaps we can put that into the open source and share it.

[ltscal2]

Anyone here used iPeer and can offer some UXD opinion?