cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
erinhmcmillan
Instructure Alumni
Instructure Alumni

Canvas Release: Anonymous Moderated Grading (2018-07-14)

This document introduces Anonymous Moderated Grading as noted in the Canvas Production Release Notes (2018-07-14)

The current anonymous and moderated grading features have been updated to provide improved assignment flexibility with fully anonymized, multiple grader anonymity, and moderated options. Additionally, moderated grading has been simplified for a better user experience.

 

 

Feature/Integration Setup

Feature Options

Anonymous Moderated Grading involves up to two feature options: Anonymous Grading and Moderated Grading. These features can be enabled individually or together to provide multiple options in the assignments page.

 

When one or both feature options are enabled, the assignments page displays several options according to each feature.

 

Existing Moderated Grading Functionality

Currently moderated grading is included in all assignments by default, but this update allows it to be enabled for the entire account or on a course-by-course basis.

 

Existing Anonymous Grading Feature Option

Accounts or courses that have set the current Anonymous Grading feature option to On will have the new Anonymous Grading feature option turned on by default. However, accounts or courses that have set the feature option to Allow must reset the Anonymous Grading feature option to Allow; by default it will be turned off.

 

Account Management

Permissions

The following permissions are available with each corresponding feature option:

 

Anonymous Grading

  • Roles: Account, Course
  • Permission: View audit trail
  • Default: no user roles

 

Moderated Grading

  • Roles: Account, Course
  • Permission: Select final grade for moderation
  • Default: enabled for TA- and Teacher-based roles

 

Course Management

Course Settings

If the required feature options have been set to Allow at the account level, individual feature options may need to be enabled at the course level. Graders and moderators may need to confirm the feature options are enabled in Course Settings. 

 

Course-Level Features

Assignment Setup

Anonymous Grading

In anonymous grading, graders cannot identify students associated with each submission, which helps remove grading bias.

 

When anonymous grading is enabled in a course, content creators can create assignments with the option to hide student names from graders.

 

Notes:

  • This option can be edited and changed at any time by the final grader or an admin. Any edits made to the assignment will be tracked. Auditing tools will be added to help with accountability in a future release.
  • Currently, enabling this option displays SpeedGrader submissions as read-only and does not allow annotations or comments in DocViewer.

 

Anonymous grading assignment option

 

Anonymous Instructor Annotations

Content creators can choose to anonymize instructor names in DocViewer-supported assignments. When the Anonymous Instructor Annotations option is enabled for an assignment, all annotations and comments made by instructors in SpeedGrader are anonymized. This option does not apply to annotations previously made to an assignment in SpeedGrader.

 

Note: Anonymous Instructor Annotations is not associated with the Anonymous Grading feature option and displays in all assignments. This option can be enabled by any content creator.

 

Anonymous Instructor Annotations option

 

Moderated Grading

In moderated grading, anyone with assignment creation permissions specify an assignment submission to be reviewed by one or multiple graders, review the grades, and select which grade should be set as the final grade for each student. For each moderated submission, the user set as the moderator is the only user who can specify the final grade for each assignment. Any grader can be specified as the grader who can determine the final grade as long as they have the corresponding permission, but the moderator does not have to personally grade the assignment. Once the assignment is created, only the moderator and admins will be able to change the assigned final grader.

 

When moderated grading is enabled in a course, content creators can create assignments with moderated options:

  • Grader number—the number of graders who should grade the submission. This number does not have a limit. However, the page displays a message if there are not enough course users with grading permissions to fulfill the desired number of graders.
    • A moderator does not have to specify which graders can grade submissions. If a course includes a surplus of graders higher than the desired number of graders, any grader can assess an assignment submission until the specified number of grades has been achieved. When an assignment has reached the specified number of graders, remaining graders will not be able to access the assignment.
    • The moderator is never included in the number of available graders since the moderator does not traditionally grade individual submissions but may provide a custom grade.
  • Grader comment visibility—the option that allows graders to view each other’s comments. This option is selected by default.
  • Final grader—the moderator, or the grader who determines the final grade. The Select Grader menu displays the names of course graders who have been given permission to determine the final grade. This permission is assigned to one or more roles at the account level.
  • Final grader visibility—the option that allows the final grader to view the names of other graders. This option is selected by default.

 

Moderated assignments cannot be edited by any grader except the moderator. Any edits made to the assignment will be tracked. Auditing tools will be added to help with accountability in a future release.

 

Moderated grading assignment option

 

Anonymous Moderated Grading

When anonymous grading and moderated grading are both enabled, content creators can create assignments that hide student names from graders and include moderated options. All options for both anonymous grading and moderated grading are displayed for the assignment.

 

When the moderated grading option for graders to view each other’s comments is enabled, the Anonymous Grading options list also includes a second checkbox option: Graders cannot view each other’s names. This option allows graders to view comments from other graders but not view the name associated with each comment.

 

Moderated and anonymous grading assignment options

 

Gradebook Functionality

Both anonymous and moderated assignments are shown in the Gradebook in a read-only state with gray columns. Additionally, both assignment types are automatically muted.

 

Note: The Anonymous label supersedes the Muted label. Assignments that are both anonymous and moderated display as anonymous, though the Grade Detail Tray displays the assignments as being muted.

 

Moderated and Anonymous assignments in the Gradebook cannot be unmuted until grades are posted

 

Anonymous Assignment Details

  • Grade cells do not display any content, including the Needs Grading icons.
  • In the New Gradebook, columns display an Anonymous label in red text.
  • In the New Gradebook, the Grade Detail Tray also displays options in a read-only state and shows the assignment as being muted.

 

Grades are not displayed in the Gradebook until the assignment is manually unmuted. Additionally, to remove anonymity from each submission, the assignment must be edited and changed so that graders can view student names.

 

Moderated Assignment Details

  • Grade cells display dashes until grades have been posted.
  • In the New Gradebook, assignment columns display a Muted label in black text.
  • In the New Gradebook, the Grade Detail Tray also displays options in a read-only state and shows the assignment as being muted.

 

Grades cannot be edited in the Gradebook before grades are posted. However, after grades are posted by the moderator, grades can be edited in the Gradebook by any grader.

 

Grader Workflow

For grading, moderated and anonymous assignments are treated the same as any other assignment type. However, when an assignment is anonymous, some or all user name data is hidden.

 

Note: Anonymization and moderation options can be enabled or disabled at any time during grading, but audit logs will record the action and the user involved. Audit log visibility will be available in a future release.

 

Dashboard

In the Dashboard, all graders receive a To-Do notification when submissions have been received by students and are ready to be graded, regardless of assignment type.

 

To Do List for Graders

 

SpeedGrader

All existing functionality is supported in SpeedGrader for both anonymous and moderated assignments.

 

The following restrictions apply to accessing assignments for each assignment type:

 

Anonymous Assignments

  • When a grader accesses a SpeedGrader link, the first student displayed in the student list is always randomly generated for every assignment. Additionally, each assignment includes a random ID that cannot be traced back to any specific student.
  • Unless an assignment is moderated, assignments can only be graded by one grader per current SpeedGrader functionality. If a submission has already been graded, other graders can still access the submission and change the grade.
  • Comments in DocViewer display Grader instead of the grader’s name.
    • Note: Anonymity features do not hide any student identification that may be included directly in the submission.

 

Anonymous grading example in DocViewer comments

 

Moderated Assignments Grading Limit

  • For moderated assignments, when an assignment reaches its maximum number of graders, the grading notification will be removed from the Dashboard for graders who have not yet assessed the assignment. If a non-assessed grader opens the assignment directly, the SpeedGrader link will not display in the sidebar. If the non-assessed grader clicks the SpeedGrader link from the Gradebook, the assignment will redirect back to the Gradebook with a message that the maximum number of graders has been reached for that assignment.

 

Moderator Workflow

For moderating grades, moderated assignments mimic the existing moderated grading workflow but with simplified functionality.

 

Dashboard

In the Dashboard, the moderator receives a To-Do notification when at least one submission has been graded for a moderated assignment.

 

To Do List for Moderator

 

Grade Summary Page

The moderator can use the grade summary page to view all grades awarded by one grader and compare them to grades awarded by another grader. Grades are grouped by grader name.

 

Note: An assignment cannot be moderated until grades have been submitted.

 

Assignment with Moderate button

 

If the assignment is anonymous, or if the moderator cannot view other grader names, the names in the grade summary page will be appropriately anonymous.

 

Grade Summary page with two graders

 

In the final grade column, the moderator can select a grade from the menu from any grader. The moderator can also open the menu and type a custom grade for the final grade.

 

Grade Summary page selecting final grade

 

When grades are final, the moderator can post the grades to the Gradebook.

 

Note: After grades are posted by the moderator, grades can be edited in the Gradebook by any grader. However, grades in the Grades Summary page cannot be changed.

 

Grades posted to Gradebook

 

Community Resources

Future Enhancements

This document outlines existing functionality included in the initial release of this feature. Anonymous moderated grading will continue to receive additional feature enhancements over future releases as indicated in Release Notes. Please follow the release notes for future functionality updates.

 

Customer Feedback

Community feedback for anonymous moderated grading is welcome per Canvas Community Feedback Guidelines, which outlines general feedback, broken functionality, and feature enhancements.

 

For anonymous and moderated grading, feature enhancements should include the anonymous&moderated tag. Before submitting a new idea, please review all existing anonymous moderated grading feature ideas.

 

Feature Documentation

All feature functionality will be updated in the Canvas Guides on July 14. Comments are welcome for functionality clarification. Lessons will also be updated with all future enhancements as indicated in release notes.

 

Known Behaviors

The following notable behaviors have been identified in anonymous moderated grading. Known behaviors are being triaged by the Canvas engineering team—feature ideas do not need to be submitted regarding their functionality.

 

Information in this document will be updated as made available by the Canvas product team:

 

Audit Functionality

  • Full audit functionality that displays information associated with both anonymous or moderated assignments will be added in a future release. This functionality will ensure fair grading practices.

 

Mobile Apps

  • Anonymous grading is currently not supported in the mobile apps. Both the Student and Teacher apps need to implement the new feature options into their code, which will be available in a future release. However, the Teacher app includes a local option to enable anonymous grading on a per-assignment basis.
  • Moderated grading is not supported in mobile apps. No timeline for feature support is currently available.
Labels (1)
65 Replies
james_trueman
Community Contributor

Whilst I am really pleased that this AMG development has moved forward - I do have some concerns.

One is that there still appears to be a mismatch of understanding between what the workflow is (the specification) and the product that has been built - sad and surprising given how much effort has gone into consulting with Instructure on this (particularly from the UK HEI Sector needing this)?  For example, the opening line to Moderated Grading does not reflect the workflow that I'm aware of either locally, or from working with colleagues at other UK institutions 'In moderated grading, a moderator can specify an assignment submission to be reviewed by one or multiple graders, review the grades, and select which grade should be set as the final grade for each student.' This may just be semantics, but it conveys confusion.  Commonly there are two 'graders' involved, a 'marker' and a 'moderator' (moderated marking), or a 'first marker' and a 'second marker' (double marking), the 'moderator' does not usually make the decision described above, and the decision process to the final grade is also not reflected above.

I do not quite understand the thinking behind the controls and effect around when grades can be changed (i.e. not before posted but after, grades in the Gradebook can be changed but this does not affect the Grade Summary view)?

However, my biggest concern is that it seems an 'anonymity leak' still remains.  Below is a screenshot of the presented Gradebook view with AMG enabled: 

AMM Gradebook

It shows that the student name column shows, well... student names. I understand that the Gradebook could hold a range of assignment outcomes, formative, moderated, double marked (not blind / anonymous), fully anonymous. Maybe not all on the same course / module, but you could easily have formative, non-anonymous and anonymous assessment in the same course site / module, so would have reason to want to see the student names in some, but not in others.

I am concerned that this design leaks anonymity where there are few students submitting, particularly one.  This is particularly because at the moment, Canvas only allows us to filter students in the Gradebook by Section, and this is actually a useful way to divide students into marking groups and filter down to them.  It is also a meaningful way to handle students with extensions, where we need to assign them a later Due Date and possibly (for various reasons) a different marker.  Under AMG (as links in the Gradebook go to random student papers in Speedgrader), it is a way of ensuring that markers can easily locate the individual students that need marking.

Below is an example Gradebook

Gradebook

By using the filter option in the top right I can filter down to the Extensions students (could be Extenuating Circumstances, a student resubmitting under Appeal, or any other reason, including extra-ordinary submissions that have to occur, or that only one student failed their first attempt and is submitting a second attempt paper/assessment).

Extensions section

And as a result, I can now see who that student is, whether or not it was an 'anonymous' assignment

Extensions section showing student name

Because at the moment we can only un-mute per assignment, I know that some colleagues are setting up new/duplicate assignments for events like extensions, so that they can better control the un-mute period.  Such workflows will also present the same issue, as one student in an 'Extension Assignment' column will clearly have their submission in the same row as their name, whether or not the Section filtering I've described is used.

Thoughts?

BW

0 Kudos
Tasha_Biesinger
Community Contributor

Some great improvements here!  We currently tell our faculty to stay far away from moderated grading because of all the 'gotcha's. We may start reconsidering when these changes role out.

Once you are able to incorporate anonymous DocViewer comments into this process, I know a lot of our faculty want all of the grader comments to appear to students (not just from the grader whose score is selected - or from neither if the instructor types in an average), for both completed rubrics to appear to students, and some faculty want the ability to post average scores of graders (without having to manually calculate). Is any of this being considered as future work?

Or...perhaps pull it all into a additional option:

When the moderator is selecting between grader 1 score, grader 2 score, or manual entry, add an additional option which is the "Average" option.  The average option will include all comments and rubrics from the graders. After posting scores, the instructor/moderator can remove any comment they choose, add their own rubric, and adjust scores to their heart's content.

A separate pain point: once a moderator chooses a grade, they can no longer see any comments/rubrics from the non-selected grader(s). This information should remain viewable to the moderator somewhere.

I'm happy to throw this stuff into feature requests, just want to plant a seed now while you all are really digging into this project just in case you can start building the hooks for the features now.

erinhmcmillan
Instructure Alumni
Instructure Alumni

Hi, Tasha,

This feature isn't fully completed and will include a few more phases. Our team is aware of comments not being able to be seen by non-selected graders, and that functionality will come a little later when the teams implement full auditing functionality.

All feature ideas need to be submitted in our Canvas Studio space as indicated in the document, otherwise our product team won't see them. We appreciate your feedback!

Thanks,

Erin

natalie_norton1
Community Champion

Hi - great to see this functionality coming!  I did some initial testing last week with no issue, but this week cannot see the moderated grading box correctly (testing in google chrome and safari - same view) 

281042_moderated grading options.png

0 Kudos
soberoi
Instructure
Instructure

Hi Natalie, 

I'm glad that your initial testing went well with no issues associated with it! The issue that you present for the checkbox for Moderated Grading not appearing certainly seems odd. I have checked this myself and have had several other members on my team check it and we have been unable to reproduce it thus far. Are you still experiencing this issue? 

Best,

Sidh

0 Kudos
hesspe
Community Champion

Hi erinhallmark   

Regarding:

...a moderator can specify an assignment submission to be reviewed by one or multiple graders, review the grades, and select which grade should be set as the final grade for each student. For each moderated submission, the user set as the moderator is the only user who can specify the final grade for each assignment.

Two things:

1) Re: "...a moderator can specify an assignment submission to be reviewed by one or multiple graders..."  

It looks one does not have to be  a Moderator (or someone with "Select final grade for moderation"permission)  to be able to select a moderator of an assignment from the list of those with grading privileges.  As a Course Designer creating a new assignment, I was able to  choose from the drop-down list the "Grader that determines the final grade."  

2) And aren't the "Grader who can specify the final grade" and the Moderator really one and the same?  If so, I'd find it clarifying if the label on the drop-down list read "Grader that determines the final grade  (the Moderator)."

I don't mean this as criticism; just checking my understanding.

0 Kudos
james_trueman
Community Contributor

Hi Peter, 

It is interesting wording, which seems to be predicated on the model that the 'professor, module/course leader, course convener etc.' is going to be the 'moderator' - e.g. that they will get other people to do the 'grading' and then check that by doing the 'moderation' (be the arbiter of whether the grading was good enough?).

Admittedly, that does happen (at least the 'leader' can sometimes be the moderator), but it's also frequently likely that when it comes to the marking load, the 'leader' is just one of the gang, who mucks in with everyone else.  It's even more common that each marker has a colleague acting as their 'moderator' - and highly probable that the 'leader' isn't doing any of either marking or moderation.  

I'm sure this could be styled a little more intuitively (and dare I say with better fidelity to practice?).

Smiley Happy

g_bowie
New Member

Hi all,

Just to echo other feedback, thank you for the developments in this area. Overall we feel they're great strides forward.

Of course during testing we have discovered some things that are probably worth highlighting - apologies if they've been covered already, and let me know if they need to go to an alternative forum / mechanism!

  • Integration with Plagiarism Detection Framework - its not possible to create an Assignment with a 3rd Party Plagiarism tool selected. I think this has been previously noted, but its not listed in the current known issues. Is there an indicative timescale for this to be rectified?
  • Old Gradebook - Assignment Groupings - If you use Assignment Groups, and only place a single assignment under that group, whilst the mark awarded to the student is hidden under the assignment, the score for the overall grouping is shown, making it relatively simple to identify the student.
  • Old Gradebook - Export CSV - This shows the student names and IDs. We're not overly concerned about this, but thought the wider community may wish to know.
  • Anonymous Instructor Annotations - When this is the only option selected on the Assignment creation page, it doesn't appear to hide or anonymous anything?
  • Moderation / Double Blind Marking Workflows - If the grader who 'Determines Final Grade' is also one of the markers, then they get to see the Speedgrader Comments from the other markers.
  • Analytics - Shows the mark awarded to individual students, regardless of the anonymous state. Again, not overly concerned about this, but the wider community may like to be aware.
  • New Gradebook - Message Students Who - Clearly shows the student name and the marker awarded - another way to identify students. Once again, not overly concerned, but the wider community may like to be aware.

Finally, is there a way to get to the student specific submission in Anonymous Mode using the APIs - we've got some custom functionality where we'd like to drop someone to a specific submission? Any advice would be gratefully received.

chriscas
Community Champion

Hi  @g_bowie ‌,

  • Integration with Plagiarism Detection Framework - its not possible to create an Assignment with a 3rd Party Plagiarism tool selected. I think this has been previously noted, but its not listed in the current known issues. Is there an indicative timescale for this to be rectified?

On my beta instance, I can do this without any visible issues right now.  Are you sure your plagiarism detection provider is setup in the Beta instance (I don't know if it automatically copies from production or not).

-Chris

natalie_norton1
Community Champion

Thanks Greg, really helpful!

  • Moderation / Double Blind Marking Workflows - If the grader who 'Determines Final Grade' is also one of the markers, then they get to see the Speedgrader Comments from the other markers.

This could be problematic for us for double blind marking as the grader who 'Determines Final Grade' will almost always be one of the markers. At the point of marking they should not see the other marker's comments (or annotations in future release) 

james_trueman
Community Contributor

Agreed - would be important to design that out methinks Smiley Wink

soberoi
Instructure
Instructure

Hi all, 

Thank you so much for your feedback here. 

  • Moderation / Double Blind Marking Workflows - If the grader who 'Determines Final Grade' is also one of the markers, then they get to see the Speedgrader Comments from the other markers.

In regards to this, we did take this into consideration which is why we added an additional option of anonymity to make the identity of graders hidden to the final grader. This option is available underneath the area where you select the final grader - if you have the option for "Final grader can view other grader names" deselected, then the moderator (or additional marker) will be unable to view the identity of the other graders.  @g_bowie ‌  @natalie_norton1 ‌  @james_trueman ‌. 

Anonymity option for hiding grader identity from final grader (moderator)

  • Old Gradebook - Assignment Groupings - If you use Assignment Groups, and only place a single assignment under that group, whilst the mark awarded to the student is hidden under the assignment, the score for the overall grouping is shown, making it relatively simple to identify the student.

Yes, for assignment groupings where there is only one assignment within the assignment group, there can be an issue inside the Old Gradebook. 

  • Old Gradebook - Export CSV - This shows the student names and IDs. We're not overly concerned about this, but thought the wider community may wish to know.

This is something that we're aware of and will be fixing moving forward. 

  • Anonymous Instructor Annotations - When this is the only option selected on the Assignment creation page, it doesn't appear to hide or anonymous anything?

This feature is not doing anything currently and is unrelated to this work. However, will enable instructor annotations to be anonymous from students in an upcoming release. This is just showing the UI for how this will be enabled in the future. Sorry for any confusion that you may have. 

  • Analytics - Shows the mark awarded to individual students, regardless of the anonymous state. Again, not overly concerned about this, but the wider community may like to be aware. 

Thank you for reporting this - it's something that we'll evaluate. 

  • New Gradebook - Message Students Who - Clearly shows the student name and the marker awarded - another way to identify students. Once again, not overly concerned, but the wider community may like to be aware.

Thank you for reporting this issue - it's another one that we'll evaluate. 

I hope this helps clarify some of the questions and comments that have been posted! 

Best,

Sidh

g_bowie
New Member

Thanks Chris - can confirm it didn't work on another Course/Module where the AMM flags are not enabled, so suspect this is more to do with the configuration of the tools on Beta. 

0 Kudos
g_bowie
New Member

Thanks for the updates  @soberoi   - much appreciated.

Not sure the workflow for 'Determine Final Grade' is what is required, but perhaps we need to discuss that as a UK Community and document something  @natalie_norton1 ‌?

Also, anyone I can speak to about the API to take an application to a specific submission?

0 Kudos
erinhmcmillan
Instructure Alumni
Instructure Alumni

Hi, Peter,

We've clarified the wording for the user who can select the final grade. And you're right, the wording comes down to semantics; we tried to keep the term general to help with translation and just kept all references as Grader.

Thanks!

Erin

0 Kudos
erinhmcmillan
Instructure Alumni
Instructure Alumni

Hi, all,

We've updated this document with a few wording clarifications, but mostly we wanted to let you know that the Anonymous Instructor Annotations option displaying for all assignments in beta right now will be supported as of our release this weekend! (Previously we noted that DocViewer documents are only supported in read-only mode). When that assignment option is selected, comments and annotations in DocViewer will display the instructor's name as Grader

Thanks!

Erin

[EDIT: Please see clarifying comment: https://community.canvaslms.com/docs/DOC-14924-canvas-release-anonymous-moderated-grading#comment-12...]

kblack
Community Champion

Woo hoo!  That is awesome news, Erin!  

0 Kudos
natalie_norton1
Community Champion

Thanks for the feedback  @soberoi ‌ 

Agree - I am not sure that workflow quite meets the requirements for double blind marking - the actual comments need to be hidden at the point of second marking, rather than the identity of the marker. Is that possible? I'll start a doc with  @g_bowie ‌ to capture requirements if that would be useful

0 Kudos
erinhmcmillan
Instructure Alumni
Instructure Alumni

Hi, everyone!

Even though Canvas release notes indicate features for full web browsers, we added a known issue with mobile apps as a reminder that moderated grading is not supported right now. Additionally, our engineers discovered a small behavior with anonymous grading that they need to correct by releasing new app versions, which should be out shortly. In the meantime, anonymous grading is not functioning correctly, but the apps do have a local workaround of enabling anonymity on a per-assignment basis.

Watch the mobile release notes for more updates on these features.

Thanks!

Erin

0 Kudos
kblack
Community Champion

I'm checking a moderated assignment marked with the checkbox "Anonymize all annotations made by instructors..." and unfortunately I've noticed that graders still do not have the ability to make annotations on a submitted assignment in SpeedGrader:

Speedgrader editing bar

Just in case this only happened to moderated assignments for some reason, I unchecked that box to make it just a "regular" assignment, and still no luck.  erinhallmark‌, I assumed the above comment applied solely to the mobile platform, but perhaps that applies to the desktop version, as well(?).

0 Kudos
hesspe
Community Champion

 erinhallmark   The document says for moderated grading the default permissions set for TAs includes "Select final grade for moderation."  If we would prefer to allow only Teachers to become moderators, but allow TAs to be Graders ( with the "Grades - Edit" and "Grades - view all grades" permissions), would that work?  Another way of asking this is: do TAs acting as graders require either the "Select final grade for moderation" or the "Grades - moderate" permissions?  Thank you,  -Peter

0 Kudos
rdudley
Community Participant

Hi Erin,

Once the new anonymous marking process is in place, we are aware that no grades for an assignment are displayed within the GradeBook until the assignment column is unmuted.

Before grades get released to students, it is imperative that our team (as admins of the system) and our Exams team can see and download this data.

How will this be possible? Will there be a particular permission setting for us to activate?

Kind regards,

Richard.

0 Kudos
erinhmcmillan
Instructure Alumni
Instructure Alumni

Hi, Peter,

Yes, as long as you don't give TAs permission to select the final grade for moderation purposes, TAs will be treated as any other grader and cannot view the moderated page.

Thanks!

Erin

0 Kudos
erinhmcmillan
Instructure Alumni
Instructure Alumni

Hi, Richard,

I hadn't seen that you had also posted in this document; I replied to your similar question in the full release notes document: https://community.canvaslms.com/docs/DOC-14928-canvas-production-release-notes-2018-07-14#comment-12... 

Thanks!

Erin

0 Kudos
kmatson
Community Member

I am a little disappointed that we don't have anonymous grading anymore. Your change makes the annotations anonymous but the SpeedGrader comments are still signed. Was there something I missed? I have set this to Anonymous Instructor Annotations. The annotations are "grader" but SpeedGrader states my name.

281963_pastedImage_1.png

AND 

When I make it anonymous then I don't have any tools to write Anonymous Instructor Annotations. I have this set to Anonymous Instructor Annotation and Anonymous Grading and I have lost my annotation tools. My student names are anonymous but we often need the grader name to by anonymous. 

281964_pastedImage_2.png

Thank you in advance for help and clarification. Maybe this is something you are still tweaking.

Karen Matson

University of Oregon

soberoi
Instructure
Instructure

 @natalie_norton1 ‌ Sorry that I missed your question here! So being able to view the comments of the other marker is controlled by the "Graders can view each other's comments" selection inside of the Moderated Grading selection area. If that is deselected, then the actual comments would be hidden during the second marking. 

 @g_bowie ‌ I'd be happy to chat with you about the API to take it to a specific submission. Could you clarify what you're looking to do here? 

g_bowie
New Member

Thanks  @soberoi ‌ - and sorry, reading back now I can see how my request isn't very clear!

We've not problems with retrieving all the student submissions - what we'd like to do is display a clickable link to take them to that submission in Speedgrader so they can start marking. We have this working in the 'traditional' assignments world, but notice it doesn't work when the New Anon Marking is switched on. Any tips?

Heading to Colorado tomorrow, so happy to try and speak to you at InstructureCon next week if that helps?

0 Kudos
soberoi
Instructure
Instructure

 @g_bowie ‌ Let's plan on speaking next week at InstructureCon! One of the engineers that worked on this project will be there as well and may be able to help navigate a solution. 

I look forward to seeing you next week! 

0 Kudos
rdudley
Community Participant

We have just discovered that all annotations for all previous submissions across Canvas are now hidden from view with the new anonymous marking tool turned on. Can this easily be fixed as we have students in need of viewing their feedback and currently they cannot see anything. We have raised a case with Canvas Support but is everyone else experiencing this?

0 Kudos
g_bowie
New Member

Hi,

Just to confirm we've checked our settings in Beta and can confirm we've lost annotations.

We haven't enabled in Live, and as such annotations are still there - but we're very concerned about someone accidently enabling. We've contact Canvas Support #03274982).

Cheers,

Greg

rdudley
Community Participant

We already had anonymous grading on but turning it off has not brought back the annotations either. Everything has remained anonymous. It's all very concerning.

0 Kudos
natalie_norton1
Community Champion

Can confirm same here - only enabled in beta but seeing same issue. 

0 Kudos
kblack
Community Champion

 @kmatson ‌ not having the annotation tools is exactly the issue that we are still having, as I commented above.  I filed a case with our IT department to forward it along to Canvas.  I haven't heard anything yet.  I'm sort of glad to hear that we're not the only ones experiencing this issue, though.  

And yes, I saw right away that the grader's name was included in the student's view of such an assignment when I was testing this myself.  I guess I was not as concerned with that, specifically, since I figured that ultimately someone is responsible for the final grade and the student should know who that is for the sake of transparency.  The bigger concern, for me, is the lack of annotation tools that was promised.

0 Kudos
kblack
Community Champion

I just verified with a spot-check in some SPRING courses that the annotations are still there in our case.  However, I set Anonymous Grading to "Allow" at my institution, rather than "ON."  Perhaps that made the difference.  I didn't check any current courses, but given people are saying ALL courses, we appear to be okay with the "Allow" setting.

0 Kudos
rdudley
Community Participant

We've tried all options available and the annotations are still not there.

0 Kudos
soberoi
Instructure
Instructure

rdudley‌ Thank you very much for reporting this issue and thank you  @g_bowie ‌ and  @natalie_norton1 ‌ for confirming that you're experiencing the issue. We're very sorry for any inconvenience or issues this may have caused in testing it out. We have our team looking into it immediately and investigating what the cause may be. Thus far they have been unable to reproduce the issue, but we'll dig deeper into the support cases and will reach out for any additional info as needed. I'll report back here later in the day as we have more info. 

Also,  @kblack ‌ thank you for jumping in and spot checking. It's good for us to know that it is only effecting a certain subset of users. 

0 Kudos
soberoi
Instructure
Instructure

rdudley‌ Do you have a support case that you could link me to? Or DM me the links to courses where this issue is occurring? Also, are your issues on Production or in beta? 

0 Kudos
rdudley
Community Participant

Hi Sid, many thanks for looking into this promptly. The case we raised was #03274907 and our issues are in Production as we were already making use of anonymous grading throughout this academic year. All annotations created on assignments that were marked anonymously are no longer appearing.

natalie_norton1
Community Champion

Just to confirm, in case it helps,  we only have anon and moderated grading features on 'allow' in beta but seeing the issue. 

0 Kudos