cancel
Showing results for 
Search instead for 
Did you mean: 

Peer Evaluation for Group Projects

Peer Evaluation for Group Projects

(10)
Every semester we need the students to create a peer evaluation for their group projects. We would like for students to be able to rate their group members contribution to the group project. This feature should allow the instructors to set up the peer evaluation for students to either rate themselves or students to only rate their peers. The instructor should be able to choose the type of evaluation students can give their peers like numerical rating, likert scale rating, text comments etc. There should also be an option to include the peer evaluation as a graded assignment.

 

Another nice option would be to automatically add the peer evaluation to group assignments. This would allow points to be deducted from the group assignment grade automatically. Usually, we had to divide the total points for the group project across two assignments: 1) the project and 2) the peer evaluation. With the option just described the peer evaluation would be included into the group project and there would no longer be a need for two assignments.

 

We have done the peer evaluation in survey builders like Qualtrics in the past but it is not the best option for several reasons. Student's get confused when they have to leave Canvas to complete assignments on other websites. Their activities don't get tracked in Canvas so they don't know if it will be graded. Qualtrics (at least the survey version we have at our University) is not designed to capture feedback in a round robin format needed for peer evaluations. We have to export the results from Qualtrics, clean the data, do some calculations then input the grades in Canvas. It becomes a really tedious process.

49 Comments
kmeeusen
Community Coach
Community Coach

This is a great idea Shana.

Peer evaluation is both work, and a part of the learning process. I have strong pedagogical feelings about providing students with meaningful feedback, and that feedback should be easily available for peer evaluations of group projects - both the feedback from the peers and the feedback from the instructor.

I am a also strong proponent  of using rubrics, so it would be excellent if this functionality could include assigning a grading rubric to the process.

Not sure where we will get with this, but count me in as a supporter.

Agent K.

Renee_Carney
Community Team
Community Team

 @sgreen10 ​

Thank you so much for your submission.  This is a great idea, and one that we have seen in a previous voting period. 

This idea can be a little complex, which is why we asked the original poster to break out.  Would you please take a look at these three archived ideas...

Do they capture the essence of what you are seeking?  If so, these ideas were archived due to low votes, but you can resubmit them.  I would be happy to help!  Let me know what you think!

kmeeusen
Community Coach
Community Coach

Thanks, Renee!

sgreen10
Community Participant

Hi Renee,

I think only C- Option to Include Peer Evaluation into Grade captures part of what I was saying. It addresses the problem of having to split the peer review and group project into two assignments.

Maybe I don't understand the other two posts (parts A and B) but I don't think those address the main points I was making about allowing students to review their peers work/effort for group projects.

scottdennis
Community Team
Community Team

Hi  @sgreen10 ​,

Sorry to be slow on the uptake but are you requesting that peer review and group projects by split in such a way that you could have students work in a group, including doing peer review on each other's work but then ultimately have it such that one same grade given to any group member will be assigned to all other group members as well?

Thanks,

SD

sgreen10
Community Participant

No. I will try to explain.

The instructor creates one assignment called Project X and assigns a peer evaluation to Project X.  Say there are 4 group members. They work on project X together. They should all be able to provide feedback about the contribution of their group members to project X. For example, Sally may have slacked on the project X and the 3 other members would like the instructor to know that. So, they complete a peer evaluation for only their group members. Let's say project X was earned 100 points. Based on the peer evaluation, Sally should only earn 80 points, whereas everyone else in the group earned 100 points.

I think it would be ideal if the peer evaluation and group project were all captured into one grade in Canvas.

However, if that cannot be done. It would be fine if a peer evaluation was incorporated in Canvas itself.  I will explain this scenario, too.

The instructor creates an assignment called Project Y (worth 80 points) and another assignment called Project Y Peer Evaluation (worth 20 points). The same groups are assigned to complete Project Y and Project Y Peer Evaluation. The students will complete the peer evaluation for their group members only. Let's say John slacked. His group members indicate this on the Project Y Peer Evaluation. Project Y earned all 80 points, but based on the Peer Evaluation John earned 0 points. So, across both assignments John only earned 80 points whereas his group members earned 100 points.

Essentially, the latter way is how I have been going about this. However, the peer evaluation has to take place outside of Canvas. I am asking that the peer evaluation is included as an assignment type. For example, it could be available under Quizzes. Right now, we have the options for Quizzes and Surveys. With this, we could have Quizzes, Surveys and Peer Evaluations.

Does that make sense?

scottdennis
Community Team
Community Team

Oh, OK, that makes more sense to me now.   I think part of my confusion came from the fact that in Canvas we use the term "peer review" to refer to a process by which students submit work that is then evaluated by other students as opposed to a method by which group work participants can evaluate each other on the effort they put into the group project.  If I understand correctly you want students' grades to be partially based on the ultimate product of the group work and in part based on how students rate the contributions of other group members.

In trying to think about this within the structure of Canvas, I'm thinking that the way this could work, potentially would be to have an assignment group for the project with one assignment being the project score and one assignment being the aggregate of the other group members rating of an individual with the teacher determining the appropriate weighting within the group.  The specific feature idea ask would then be a method for capturing and averaging the group member ratings and applying them to the assignment grade.  Would you agree?

By the way, it occurs to me that all the hooks are there for this need to be met by an LTI integration - third party tool by which students could evaluate each other, averaging magic would happen and a grade be written back to the gradebook.

biray
Community Champion

Thank you for your further explanation,  @sgreen10 ​. There's also another idea here: (which might prove helpful, as well). This idea was, unfortunately, also archived. Since re-submissions of archived ideas are accepted, we will move forward with this idea and it will be open for voting in the upcoming voting cycle.

Smiley Happy

biray
Community Champion

This idea has moved to the next stage and will be open for voting among the Canvas Community, from Wed. April 6, 2016 - Wed. July 6, 2016.

Check out this doc for additional details about how the voting process works!

lmd5adm
Community Participant

We have a tool that we built at PSU World Campus but do not have the time to make it dynamic enough to be used in all of our courses at Penn State.  It does not tie back into the gradebook and only provides a summary download for the instructors.

Here is what we do:

Introduction

For years within World Campus we've had a homegrown tool that allows for peer evaluations within courses.  This current tool will do the following:

  • Evaluation of Teammates or Classmates.  It interfaces with Canvas to grab the Groups every time the tool is launched so that a student will be reviewing ONLY those members on his or her team. Teams are grabbed from Canvas at the time the script is run, so a student clicking the peer eval button gets the most up-to-date Group info.
  • Multiple Evaluations. You can set up as many evaluations throughout the course as necessary.
  • Your choice of Evaluation Criteria. You can use as many questions or criteria categories as you want.
  • A numeric scale of your choosing.  You can set whatever likert scale you like.
  • The option of open-ended comments. 
  • Automatic averaging of numeric scores for each individual.

Setting Up Your Peer Evaluation (2 of 17)

Setting up your Peer Evaluation

  • From the Edit view of your section record, check the Peer Evaluations checkbox and SAVE.
  • Once that has been checked (and saved), allow for the sync process to take place (2 minutes average).  Do NOT use the link from within the Quickbase Section Record.
  • Go to Settings -> Navigation -> enable the WD Peer Eval link in the course navigation
  • Save
  • Selecting WD Peer Eval in the course navigation will take you to the Instructor Portal of the Evo Peer Evals tool. Please note, this link refers to your role in the Canvas course and directs the user to the appropriate portal; therefore, when students select this link, they will be directed to the Student Portal. Instructors will also use this link to access the Peer Eval Results.

  class="center"

administration portal 

To Add an Evaluation (3 of 17)

To Add an Evaluation

  • In the bottom row of the Existing Peer Evaluations table, select New.
  • Enter a Title for your peer eval.
  • Optional: set Start and End dates for this Peer Evaluation. If these dates are set, the students will not see the Peer Evaluation through the student portal until the Start Date.

peer eval instructions box 

Adding Questions (4 of 17)

Adding Questions

To add a question:

  • Make sure you are in the edit mode of the Peer Evaluation.
  • Select Add Question. This will enable the question types for you to choose from.
  • Select which type of question you would like to add. (See the screen shot below)
  • When you select the question type, the question editor will automatically open for that question.

More details on the types of questions will be discussed on the next page.

Question Types (5 of 17)

Question Types

There are 4 types of questions to choose from: Rated, Comment, Numeric Score, and Percent Score.

Rated

This question type provides a scale to have students rate their peers on various criteria such as cooperation or contributions.

 

To set up a rated question:

  • If you'd like the student to also rate themselves on that question, check the Self-Review box.
  • Enter the Start Rate of the scale.
  • Enter the End Rate of the scale.
  • Enter the instructions for the question in the text box.

Tip - be sure to explain the criteria of the scale such as "On a scale of 1 - 10 with 10 being strongly agree and 1 being strongly disagree, ..."

Comment

 

This question type provides a text box for the student to provide information and feedback.

To set up a Comment question:

  • If you'd like the student to also rate themselves on that question, check the Self-Review box.
  • Enter the instructions for the question in the text box.

Please note, in order to submit the evaluation, the student must enter something in that box, even if their response is "No Comment". You may want to include that in your instructions for that question.

Numeric Score

 

A numeric score question allows a "TEAM TOTAL" for this one question to be dispersed among the team members. The students should be asked to divide the team total among all of their team mates.

To set up a Numeric Score question:

  • If you'd like the student to also rate themselves on that question, check the Self-Review box.
  • Enter the End Limit (Team Total).
  • Enter the instructions for the question in the text box.

Please note, the students filling out the Peer Evaluation do not need to disperse all of the available points for this question. For instance, if the End Limit is 100, a student could choose to only give 3 students 20 points for a total of 60/100.

Percent Score

 

This question type allows the student to give a score to each student up to 100% for each student.

To set up a Percent Score question:

  • If you'd like the student to also rate themselves on that question, check the Self-Review box.
  • Enter the instructions for the question in the text box.

Please note, a student can not give a student a score higher than 100% in this evaluation.

Self-Review Option

Each question has the ability to include "Self-Review" as an option.  This option allows the students to include a review of his/her own work.  With this option selected, students will see his/her name on the list for that question.  If this option is not selected, student will not see his/her own name but just those of their teammates.

To Edit an Evaluation
  1. Choose which Peer Eval you wish to edit from the Existing Peer Evaluations Sections.
  2. Choose "Edit" from the Action(s) column of the Administration Page.
  3. After completing your edits, save the record.
To Copy an Evaluation

In many cases, you may want the exact same evaluation for different projects or given at various points throughout the semester. Copy allows you to copy an evaluation, saving you time in re-creating it.

  1. Choose which Peer Eval you wish to copy from the Existing Peer Evaluations Sections.
  2. Choose "Copy" from the Action(s) column of the Administration Page.
  3. The copy of the evaluation will open in editing mode. Be sure to change the main title of the evaluation so that you can differentiate between that one and the original.
  4. Make any additional edits you would like.
  5. Save.

To Delete an Evaluation
  1. Choose which Peer Eval you wish to delete from the Existing Peer Evaluations Sections.
  2. Choose "Delete" from the Action(s) column of the Administration Page.
  • From the list of student's submissions, check the box next to the student's name(s)
  • Select "Delete Selected Review" to delete just that student submission(s)
  • Select "Delete All Submissions"

Student View (12 of 17)

Student View - I've not had the chance to document this section yet.  But here are some quick screen shots.

Screen Shot 2016-05-02 at 2.56.38 PM.png

Screen Shot 2016-05-02 at 2.57.33 PM.png

Screen Shot 2016-05-02 at 2.58.55 PM.png

Screen Shot 2016-05-02 at 2.59.34 PM.png

That gives a quick view of what the student would see.  It doesn't show each question type but I can come back later and add more.

(edited to fix some text that didn't copy right)

iys2
Community Participant

Lisa, thank you for sharing this process as I miss incorporating this for group projects. I really hope that this or something similar to this could be incorporated into Canvas.

lmd5adm
Community Participant

Thanks Louise.  I've edited my post.  I noticed that some of my information did not copy over properly.

alh245
Community Participant

Another great feature of this (and what I would like to see in Canvas) is the self-evaluation piece. 

Each member of the group evaluates their team mates as well as themselves based on their contribution to the project. 

Ideally this should go back into the gradebook directly.

Working well on teams is an important part of our curriculum, gives experience similar to real-life, and leads to enhanced learning.

Thanks,

Amy

iys2
Community Participant

Amy, this is a great point and I would like to see this as well. This was also an option in the Peer Evaluation tool that Lisa presented. I truly miss this tool and am hoping that something could be integrated into Canvas. This is a chance for members of a team to speak up and provide feedback about the work they have completed as well as others on their team. This is also another way that instructors can collect feedback about the assignment itself and could be potentially used to make improvements in the actual assignment. Great point - thanks for mentioning this.

BKINNEY
Community Contributor

I'm voting for this, for about the third time.

We have a solution in place through LTI that does everything you have described,  including (optionally) pushing a new gradebook column back into Canvas, but we can not share it easily for a wide variety of reasons. Even to put the code on GitHub, we'd have to make significant changes to the code base. We actually applied for a Canvas grant to do it a few years back, but we didn't win. So, we're happy with our solution, but still hoping that Instructure will come through with something.

One thing you might want to think about adding to your request is anonymity. Most of my users would not be taking advantage of our system if individual evaluations were not anonymous, in the sense that evaluatees don't know who has said what about them. Instructors do. Since the existing Canvas Peer Review is not anonymous, you shouldn't assume anonymity would be part of any group-enabled option Instructure might add.

RobDitto
Community Champion

 @BKINNEY , thanks for all you have been doing to keep this need alive as a feature idea!

Out of curiosity, what has been UDel's experience with the "Peer Reviews Appear Anonymously" setting?  While the notifications were finally anonymized, for Wharton what remains is that reviewers can still download submissions and might be able to spot the submitter's name in document properties.

BKINNEY
Community Contributor

 @RobDitto ,

I don't have any personal experience to report with regard to the built-in tool's anonymity. I have no reason to suppose it behaves any differently for us than it does for you. We recommend our own inhouse tool for faculty looking for peer evaluation (which is always anonymous), and the Canvas tool for those who want peer review. For the most part, I see the former, since I created the inhouse tool. Doesn't mean I never field questions about Peer Review, but I've never heard from anyone who was concerned about anonymity in that context. If we did get such an inquiry, we can easily slide those folks over to our anonymous tool.

n_a_gordon
Community Participant

The WebPA peer assessment tool is designed to meet this kind of requirement: see WebPA

there are LTI integration for it: though it requires that an institution host WebPA itself.

For information on the LTI facility for WebPA see

http://www.spvsoftwareproducts.com/php/webpa-lti/ which means that WebPA could then be used as an external App.

vrs07nl
Community Contributor

WebPA is a leader here in UK for peer assessment etc

It may be free BUT it is resource heavy in that you need a data base and host site, someone to manage it etc - for us that is IT who do not have the resources to even help out with a pilot at present... 

We are using surveys that we embed into courses but then extract data vis csv files etc... all time consuming

SO I am all for this feature idea

BKINNEY
Community Contributor

Yes, the hooks are there, but only if you have API access, since LTI grade passback isn't useful in a context where the logged-in student is NOT the one receiving the grade. We (University of Delaware) built a standalone Peer Evaluation Tool we've been able to hook into Canvas using the API + LTI. It does everything Shana has described, except that the PE grade is not automatically incorporated into the project grade, but rather appears as it's own gradebook column, which you can do with as you will.

Regardless, I don't think the fact that it is possible to build a third party solution should prevent Instructure from considering this request. I have been unable to share my app outside our university for a variety of reasons, so the fact that our tool exists does not help anyone else. Please consider this request. Canvas has such a great set of collaboration tools, it is really a shame not to close the loop.

About Idea Conversations
In the Canvas Community Ideas space, you can share, converse, and rate idea conversations related to software improvements to Canvas products.