[Peer Review] Peer Evaluation for Group Projects

[sgreen10]

Every semester we need the students to create a peer evaluation for their group projects. We would like for students to be able to rate their group members contribution to the group project. This feature should allow the instructors to set up the peer evaluation for students to either rate themselves or students to only rate their peers. The instructor should be able to choose the type of evaluation students can give their peers like numerical rating, likert scale rating, text comments etc. There should also be an option to include the peer evaluation as a graded assignment.

Another nice option would be to automatically add the peer evaluation to group assignments. This would allow points to be deducted from the group assignment grade automatically. Usually, we had to divide the total points for the group project across two assignments: 1) the project and 2) the peer evaluation. With the option just described the peer evaluation would be included into the group project and there would no longer be a need for two assignments.

We have done the peer evaluation in survey builders like Qualtrics in the past but it is not the best option for several reasons. Student's get confused when they have to leave Canvas to complete assignments on other websites. Their activities don't get tracked in Canvas so they don't know if it will be graded. Qualtrics (at least the survey version we have at our University) is not designed to capture feedback in a round robin format needed for peer evaluations. We have to export the results from Qualtrics, clean the data, do some calculations then input the grades in Canvas. It becomes a really tedious process.

[kmeeusen]

This is a great idea Shana.

Peer evaluation is both work, and a part of the learning process. I have strong pedagogical feelings about providing students with meaningful feedback, and that feedback should be easily available for peer evaluations of group projects - both the feedback from the peers and the feedback from the instructor.

I am a also strong proponent  of using rubrics, so it would be excellent if this functionality could include assigning a grading rubric to the process.

Not sure where we will get with this, but count me in as a supporter.

Agent K.

[Renee_Carney]

 @sgreen10 ​

Thank you so much for your submission.  This is a great idea, and one that we have seen in a previous voting period. 

This idea can be a little complex, which is why we asked the original poster to break out.  Would you please take a look at these three archived ideas...

Do they capture the essence of what you are seeking?  If so, these ideas were archived due to low votes, but you can resubmit them.  I would be happy to help!  Let me know what you think!

[kmeeusen]

Thanks, Renee!

[sgreen10]

Hi Renee,

I think only C- Option to Include Peer Evaluation into Grade captures part of what I was saying. It addresses the problem of having to split the peer review and group project into two assignments.

Maybe I don't understand the other two posts (parts A and B) but I don't think those address the main points I was making about allowing students to review their peers work/effort for group projects.

[scottdennis]

Hi  @sgreen10 ​,

Sorry to be slow on the uptake but are you requesting that peer review and group projects by split in such a way that you could have students work in a group, including doing peer review on each other's work but then ultimately have it such that one same grade given to any group member will be assigned to all other group members as well?

Thanks,

SD

[sgreen10]

No. I will try to explain.

The instructor creates one assignment called Project X and assigns a peer evaluation to Project X.  Say there are 4 group members. They work on project X together. They should all be able to provide feedback about the contribution of their group members to project X. For example, Sally may have slacked on the project X and the 3 other members would like the instructor to know that. So, they complete a peer evaluation for only their group members. Let's say project X was earned 100 points. Based on the peer evaluation, Sally should only earn 80 points, whereas everyone else in the group earned 100 points.

I think it would be ideal if the peer evaluation and group project were all captured into one grade in Canvas.

However, if that cannot be done. It would be fine if a peer evaluation was incorporated in Canvas itself.  I will explain this scenario, too.

The instructor creates an assignment called Project Y (worth 80 points) and another assignment called Project Y Peer Evaluation (worth 20 points). The same groups are assigned to complete Project Y and Project Y Peer Evaluation. The students will complete the peer evaluation for their group members only. Let's say John slacked. His group members indicate this on the Project Y Peer Evaluation. Project Y earned all 80 points, but based on the Peer Evaluation John earned 0 points. So, across both assignments John only earned 80 points whereas his group members earned 100 points.

Essentially, the latter way is how I have been going about this. However, the peer evaluation has to take place outside of Canvas. I am asking that the peer evaluation is included as an assignment type. For example, it could be available under Quizzes. Right now, we have the options for Quizzes and Surveys. With this, we could have Quizzes, Surveys and Peer Evaluations.

Does that make sense?

[scottdennis]

Oh, OK, that makes more sense to me now.   I think part of my confusion came from the fact that in Canvas we use the term "peer review" to refer to a process by which students submit work that is then evaluated by other students as opposed to a method by which group work participants can evaluate each other on the effort they put into the group project.  If I understand correctly you want students' grades to be partially based on the ultimate product of the group work and in part based on how students rate the contributions of other group members.

In trying to think about this within the structure of Canvas, I'm thinking that the way this could work, potentially would be to have an assignment group for the project with one assignment being the project score and one assignment being the aggregate of the other group members rating of an individual with the teacher determining the appropriate weighting within the group.  The specific feature idea ask would then be a method for capturing and averaging the group member ratings and applying them to the assignment grade.  Would you agree?

By the way, it occurs to me that all the hooks are there for this need to be met by an LTI integration - third party tool by which students could evaluate each other, averaging magic would happen and a grade be written back to the gradebook.

[biray]

Thank you for your further explanation,  @sgreen10 ​. There's also another idea here: (which might prove helpful, as well). This idea was, unfortunately, also archived. Since re-submissions of archived ideas are accepted, we will move forward with this idea and it will be open for voting in the upcoming voting cycle.

[biray]

This idea has moved to the next stage and will be open for voting among the Canvas Community, from Wed. April 6, 2016 - Wed. July 6, 2016.

Check out this doc for additional details about how the voting process works!

[lmd5adm]

We have a tool that we built at PSU World Campus but do not have the time to make it dynamic enough to be used in all of our courses at Penn State.  It does not tie back into the gradebook and only provides a summary download for the instructors.

Here is what we do:

Introduction

For years within World Campus we've had a homegrown tool that allows for peer evaluations within courses.  This current tool will do the following:

• Evaluation of Teammates or Classmates.  It interfaces with Canvas to grab the Groups every time the tool is launched so that a student will be reviewing ONLY those members on his or her team. Teams are grabbed from Canvas at the time the script is run, so a student clicking the peer eval button gets the most up-to-date Group info.
• Multiple Evaluations. You can set up as many evaluations throughout the course as necessary.
• Your choice of Evaluation Criteria. You can use as many questions or criteria categories as you want.
• A numeric scale of your choosing.  You can set whatever likert scale you like.
• The option of open-ended comments. 
• Automatic averaging of numeric scores for each individual.

Setting Up Your Peer Evaluation (2 of 17)

Setting up your Peer Evaluation

• From the Edit view of your section record, check the Peer Evaluations checkbox and SAVE.

• Once that has been checked (and saved), allow for the sync process to take place (2 minutes average).  Do NOT use the link from within the Quickbase Section Record.

• Go to Settings -> Navigation -> enable the WD Peer Eval link in the course navigation

• Save

• Selecting WD Peer Eval in the course navigation will take you to the Instructor Portal of the Evo Peer Evals tool. Please note, this link refers to your role in the Canvas course and directs the user to the appropriate portal; therefore, when students select this link, they will be directed to the Student Portal. Instructors will also use this link to access the Peer Eval Results.

  class="center"

 

To Add an Evaluation (3 of 17)

To Add an Evaluation

• In the bottom row of the Existing Peer Evaluations table, select New.
• Enter a Title for your peer eval.
• Optional: set Start and End dates for this Peer Evaluation. If these dates are set, the students will not see the Peer Evaluation through the student portal until the Start Date.

 

Adding Questions (4 of 17)

Adding Questions

To add a question:

• Make sure you are in the edit mode of the Peer Evaluation.
• Select Add Question. This will enable the question types for you to choose from.
• Select which type of question you would like to add. (See the screen shot below)
• When you select the question type, the question editor will automatically open for that question.

More details on the types of questions will be discussed on the next page.

Question Types (5 of 17)

Question Types

There are 4 types of questions to choose from: Rated, Comment, Numeric Score, and Percent Score.

Rated

This question type provides a scale to have students rate their peers on various criteria such as cooperation or contributions.

To set up a rated question:

• If you'd like the student to also rate themselves on that question, check the Self-Review box.
• Enter the Start Rate of the scale.
• Enter the End Rate of the scale.
• Enter the instructions for the question in the text box.

Tip - be sure to explain the criteria of the scale such as "On a scale of 1 - 10 with 10 being strongly agree and 1 being strongly disagree, ..."

Comment

This question type provides a text box for the student to provide information and feedback.

To set up a Comment question:

• If you'd like the student to also rate themselves on that question, check the Self-Review box.
• Enter the instructions for the question in the text box.

Please note, in order to submit the evaluation, the student must enter something in that box, even if their response is "No Comment". You may want to include that in your instructions for that question.

Numeric Score

A numeric score question allows a "TEAM TOTAL" for this one question to be dispersed among the team members. The students should be asked to divide the team total among all of their team mates.

To set up a Numeric Score question:

• If you'd like the student to also rate themselves on that question, check the Self-Review box.
• Enter the End Limit (Team Total).
• Enter the instructions for the question in the text box.

Please note, the students filling out the Peer Evaluation do not need to disperse all of the available points for this question. For instance, if the End Limit is 100, a student could choose to only give 3 students 20 points for a total of 60/100.

Percent Score

This question type allows the student to give a score to each student up to 100% for each student.

To set up a Percent Score question:

• If you'd like the student to also rate themselves on that question, check the Self-Review box.
• Enter the instructions for the question in the text box.

Please note, a student can not give a student a score higher than 100% in this evaluation.

Self-Review Option

Each question has the ability to include "Self-Review" as an option.  This option allows the students to include a review of his/her own work.  With this option selected, students will see his/her name on the list for that question.  If this option is not selected, student will not see his/her own name but just those of their teammates.

To Edit an Evaluation

1. Choose which Peer Eval you wish to edit from the Existing Peer Evaluations Sections.
2. Choose "Edit" from the Action(s) column of the Administration Page.
3. After completing your edits, save the record.

To Copy an Evaluation

In many cases, you may want the exact same evaluation for different projects or given at various points throughout the semester. Copy allows you to copy an evaluation, saving you time in re-creating it.

1. Choose which Peer Eval you wish to copy from the Existing Peer Evaluations Sections.
2. Choose "Copy" from the Action(s) column of the Administration Page.
3. The copy of the evaluation will open in editing mode. Be sure to change the main title of the evaluation so that you can differentiate between that one and the original.
4. Make any additional edits you would like.
5. Save.

To Delete an Evaluation

1. Choose which Peer Eval you wish to delete from the Existing Peer Evaluations Sections.
2. Choose "Delete" from the Action(s) column of the Administration Page.

• From the list of student's submissions, check the box next to the student's name(s)

• Select "Delete Selected Review" to delete just that student submission(s)

• Select "Delete All Submissions"

Student View (12 of 17)

Student View - I've not had the chance to document this section yet.  But here are some quick screen shots.

That gives a quick view of what the student would see.  It doesn't show each question type but I can come back later and add more.

(edited to fix some text that didn't copy right)