Post Policy Updates Feedback

[jfenton]

Thanks for your feedback! We've made several improvements in the last few Canvas releases that are now in your production environment.

Hey everyone, 

We're grateful for all of the feedback we've received on Post Policies. The team has been working hard to address the most pressing issues and we've released several bug fixes over the last few weeks. And there's more work to come. To that end, we're looking for feedback on some proposed changes to Post Policy. Our goal with these changes is to make the feature more intuitive and reduce confusion. We also do not want these changes feel too disruptive to people who have already been using Post Policies. And we're anxious to get feedback on if we're heading in the right direction. 

Ok, let's get into it. 

1 - Updated Iconography 

The first proposed change is to the icons that we're using. Our goal here is to make the icons more streamlined between Gradebook and SpeedGrader, while still providing users the information that they need. 

For anyone who is unfamiliar with the new Post Policy feature, in New Gradebook you’re now able to set the policy for a course or an individual assignment that governs if grades are made available to students immediately as they are entered or if they are hidden until explicitly posted by the teacher. An assignment that has grades hidden by default has a “manual” policy, while the default behavior that makes grades available immediately to students is called an “automatic” policy. 

After grades are entered for students using an automatic post policy, they can be hidden if necessary; any new entered grades or changed grades are identified as being hidden (automatic hidden status). When using a manual policy, new entered grades or changed grades are also identified as being hidden (manual hidden status). Hidden grades must be posted before they can be made available to students.

This is what the new icon chart looks like:

Here’s a mockup of what the icons would look like in Gradebook headers:

Let me talk through the changes. First, you’ll notice that we’ve added a dot to the eye icon. We believe that most of the confusion around the eye icon has come from us trying to convey two different bits of information in a single icon. By using the eye icon to indicate the post policy for the assignment and a dot to indicate if there are grades actively being hidden, we hope to remove ambiguity and make the icon much more intuitive. The crossed out icon always indicates a “manual” post policy, while the dot will always indicate that a grade is hidden. We’ll also be updating the color of the badge in the individual cells to match the blue you see in the mockup. If an institution uses a custom color scheme, the badge will take the primary color.  

We’ve streamlined the icon between Gradebook and Speedgrader to make its meaning more consistent. In both places you’ll be able to see not only the post policy for the assignment, but also if there are grades that are ready to be posted to students. 

We’ve also changed the icon in the total column to more closely match what we’re trying to convey there, which is the total score includes at least one score that is hidden from the students. 

2 - Simplify posting to automatic assignments 

We’re proposing getting rid of the "Graded" option when posting to automatic assignments. If the assignment is set to post grades automatically, any posting that happens will have the same result if you post to everyone or graded. Since those two actions yield the same results, we can get rid of one of them to reduce any confusion.

3 - Posting to “everyone” sets policy to Automatic

When posting grades to everyone on a manual assignment, we're proposing also changing the policy for that assignment to automatic. This will get rid of the manual icon in the header and feels more in line with what users expect that action to do. We’ve added some wording to the post description to inform users that the policy will be changed:

We will also updated the confirmation alert with similar language. Post to a specific section or only to students that have been graded will not change the policy to automatic. 

We’re hopeful that these changes reduce confusion and make this feature even easier to use. Let us know what you think in the comments!

__________________________________________________________________________________________________________

UPDATE September 4th

Wow! Thanks for all the feedback. We’re so lucky to have such a passionate community of users and we genuinely appreciate everyone taking the time to chime in. 

While most of the comments so far have focused on the icons, it does sound like we’re on the right track with #2 and #3 mentioned above. Watch the release notes to see #2 and #3 be implemented in a future release. We’re excited about these changes!

I wanted to take a minute to give a bit more context for the icon proposal that we made and talk through some of our thinking. We’re proposing 2 icons for Post Policy. 

Eye Icon

First, an “eye” icon. This will most often appear with a slash through it and will indicate when an assignment has a manual policy, which means grades and comments are hidden from students until they are explicitly made available. As has been pointed out, an eye with a slash through it conveys something not being visible. We agree. The eye is meant to convey to faculty that the grades for an assignment will not be visible to students as they are entered.

It has been suggested that a different icon be used to indicate the policy on an assignment. This is certainly a possibility. However, no matter what icon we use, there will need to be some learning for faculty initially. We feel like the slashed eye icon does convey that grades entered for that assignment will be hidden from students. 

Dot Badge

The second icon is a colored badge. As has been mentioned in the comments, this is used elsewhere in Canvas to denote when something needs the user’s attention or when something new has happened. In a similar way, the dot here is meant to let the instructor know that something is requiring their action. There are grades that are ready to be posted as soon as the faculty is ready to make them available to students. If they post all of the grades that have been entered the dot disappears, and reappears when there are more grades that need to be made available to students. The blue dot in the header will also have consistency with the individual cells in the gradebook, as well as the hidden count in the tray.

UI Help

One common suggestion has been that we provide an easy way for users who are unsure of what the icons denote to learn more beyond the user guides. We’re exploring repurposing the current keyboard shortcuts model to be a more general “help” section. We could then include a key to these icons (as well as the keyboard shortcuts) directly in the UI.

Here’s a different view of the icon table above: 

Thanks for your thoughts!

____________________________________________________________________________________________________________

UPDATE September 9th

Hey everyone! 

Back with another update. First off, let me say one more time how grateful we are for everyone who takes the time to share their thoughts and feedback with us. We’ve been reading and discussing every single comment. 

It’s clear that using the eyeball icon to convey the policy state is confusing. It’s also clear that it will be the most intuitive to have two totally distinct and independent representations of the policy and the current visibility state of submissions. With this feedback in mind, we’ve got a new proposal that we hope will address many of your concerns. 

First, we’re proposing that we use the eyeball with a slash through it to indicate that there are graded submissions that are hidden from students. It will only appear in the gradebook header if there are grades/comments currently hidden from students for a given assignment. 

Second, we’re proposing displaying manual post policy status where muted status used to be conveyed - right below the assignment title. This will put it front and center for instructors and remove any ambiguity associated with a new icon. Additionally, this placement will feel familiar to users who used mute functionality in the past. 

Here’s an example of what 2 assignments would look like, both with manual policies. The first doesn't have any graded submissions that are currently being hidden from students, while the second one does.

We will only show the policy state if it has been set to manual. Similarly, we only show the eyeball if there are graded submissions that are hidden from students.  

Let us know what you think in the comments! 

____________________________________________________________________________________________________________

UPDATE September 18th

Hello Everyone! 

I’m back with another update. As is always the case with this awesome community, we’ve been yet again privileged to receive so much great feedback on my previous update. We spend a lot of time reading and discussing each comment. Thank you for the time you take to engage with us and share your thoughts. 

Let me also take a moment to say thank you for your patience as we take in and process feedback and put a plan in place to more forward. We think the worst thing to do here would be to introduce changes that end up needing to be changed again down the road. We want to make sure we’re moving forward in a positive direction, and that admins can confidently train their faculty knowing that we won’t be ripping changes out in a few weeks.

OK - let’s get into it. 

Icons

Based on the feedback we’ve received it sounds like the most recent proposal is a step in the right direction. Whenever grades are hidden from students the eye with a slash will be present in the header. A manual policy will be indicated by the word Manual in the header. The slashed eye in the Total Column will remain and will be visible whenever the total score for that student includes one or more grades that are hidden from them. The “Hidden” pill will still be used in SpeedGrader to indicate a grade that is hidden from students. Watch the release notes for this to be released in the near future.

Goals

There have been questions about why this change was made in the first place. Some have indicated that mute/unmute worked well for them. We appreciate this perspective. I wanted to take a moment to share some of our thinking that went into this project and what our goals were. 

There were three main goals with this project:

1 - We wanted to make it easier for teachers to manage grade visibility preferences across an entire course. In the world of Mute/Unmute this had to be done for every assignment, one by one. With Post Policies we hoped to make it simple and quick to set something course-wide that would hide grades from students as they were entered. This can be done with a course level post policy. 

2 - We wanted to give teachers more flexibility around how they release grades. Mute/Unmute was all-or-nothing for an assignment. In order to give additional flexibility, we needed to separate the default behavior for grades that are entered from the current visibility to students. For example, I might want to post grades to section A before I’ve graded (or even received) submissions from section B. Even though grades are now visible for section A, I still need grades to be hidden by default as they’re entered for section B. This can be done by setting a post policy for an assignment (or course as mentioned above) and then posting grades to a subset of my class. 

3 - We wanted to create more intuitive language and iconography. As has been expressed in the comments here, the term “mute” can be offensive to some. Additionally, it does not translate well across all of the languages we support. On the icon front - the bell icon seemed ill-suited for what it was conveying. Now, as the bulk of this discussion attests, we missed the mark here with our initial release. We’re excited to be making improvements to the icons and taking a step forward in hitting this goal. 

Perspective

Now, if I may, I’d like to say something about perspective. Each institution has different practices and feature needs. It can be tempting to look at a feature through the lens of one’s own needs only. Let me give 2 examples from this project. 

First - There have been some comments suggesting that the policy state on assignments does not need to be indicated. This is actually how our initial designs were set up. The only thing that was readily visible to teachers in the gradebook was the current visibility of grades. In those early designs a teacher needed to open the Post Policy tray in order to see the policy status. However, in our user testing we received universal feedback that the policy state was critical for teachers to see. As they enter the first grade for an assignment, it needs to be very clear whether that grade would be visible to students or not. Based on the prevalence of this feedback we adjusted our designs to make the policy state visible in the gradebook. 

Second - Some have suggested that an all-or-nothing approach to posting grades is sufficient and that adding more granularity only creates unnecessary complexity to a formerly simple feature. It is true that posting grades now requires a few more clicks. It is also true that there are now 2 pieces to this feature instead of one. However, the desire to have more control over whose grades are released is a request we’ve heard repeatedly. Since releasing Post Policy we’ve received positive feedback from institutions who welcome this added flexibility. And there are some who would like even more. The Post Policy work sets us up to offer more granularity in the future, like posting/hiding for an individual student.  

Our overarching goal is always to deliver features that are flexible enough to cover the vast array of needs, while still being easy to use and understand. It’s a hard line to walk and we obviously don’t always hit the bullseye. Sometimes features aren’t flexible enough, other times they are overly complex. We’re always looking to adjust as we get feedback.

Thank you again for all your input. Have a great day!