Addressing Your Feedback on Recent Discussion Checkpoints Changes

[SamGarza1]

This morning, we gathered our product team to discuss your feedback on the recent Speedgrader changes that apply to all Discussions in the Discussion Checkpoints feature. We heard you loud and clear – the update disrupted your grading workflow, especially for those not using Discussion Checkpoints. We understand how frustrating this has been, and we’ve decided to disable Discussion Checkpoints effective immediately until we can respond to your feedback and needs. If institutions would like to utilize Checkpoints before it is reenabled, they can reach out to their CSMs.

Note: If your institution has enabled Discussion Checkpoints, it remains enabled. Your institution can disable it by adjusting the feature flag. When the Feature Flag is disabled, Checkpoint Discussions will show a banner message informing instructors that it was disabled. Instructors will need to edit the discussion to update the due date. Points and assigned users will remain unchanged.

Here are the main concerns you’ve shared and how we are addressing them:

Speedgrader Changes Impacting All Discussions:

• The Issue: The updated Discussion experience in Speedgrader, designed to optimize the grading experience for Discussion Checkpoints, was applied to all discussions, even those without checkpoints enabled. 
• Our Solution: While our initial group of testers felt that these features would be beneficial to all Speedgrader users, we are hearing now that many others disagree. We will work to incorporate this additional perspective into an update that will allow users to choose either version.

Accessibility Concerns:

• The Issue: Concerns have been raised about the accessibility of the updated Speedgrader interface. More specifically around the newly added Previous Reply and Next Reply buttons, their placement on the page, and how they behave when navigating an individual user’s posts. 
• Our Solution: We are committed to providing an inclusive and accessible learning environment for everyone. We’ll be re-testing the updated Speedgrader interface, making some UX changes, and quickly addressing any reported accessibility issues to ensure it meets the highest accessibility standards.

Addressing Issues in Large Courses:

• We are looking into the performance issues some instructors are experiencing with the changes to Speedgrader in large courses with high student enrollment. Our development team is actively investigating and optimizing the system to ensure smooth and efficient grading for all course sizes.

Benefits of the New Discussions Experience in Speedgrader:

The latest updates to the new Discussions experience in Speedgrader were developed in partnership with our users, who told us they were looking for these benefits:

• Enhanced Context: Instructors can now view student responses within the context of the entire discussion thread, including the original prompt and other replies, and reply directly from Speedgrader. This provides a more holistic understanding of student engagement and participation.
• Updated Navigation: The "previous" and "next" buttons allow for more efficient grading, especially for longer discussions with multiple participants.
• Seamless Integration with Checkpoints: The changes to Speedgrader provide a streamlined and intuitive grading experience for instructors using Discussion Checkpoints.
• Integration with Third-Party Tools: The integration with discussion checkpoints lays the groundwork for integration with tools like Turn-It-In or other AI detection tools to verify the authenticity of student work.

Discussion Checkpoints - Optional and Flexible:

• Discussion Checkpoints are completely optional. Admins can enable or disable them at the root level for your institution based on your teaching needs and preferences. The feature is turned off by default.

Moving Forward:

We are committed to addressing your feedback and continuously improving the Canvas experience. We will keep you updated on the progress of these improvements and provide clear communication about any upcoming changes.  Feedback will be collected through a Maze survey shared below. The survey is a follow-up to feedback given by Canvas Community members and will further explore issues related to how Speedgrader works with discussions, especially checkpointed discussions.

https://t.maze.co/337213163 

We believe that these changes have the potential to significantly enhance the grading workflow for discussions. We appreciate your engagement, patience, and understanding as we work to make Discussion Checkpoints a reality.

We encourage you to continue sharing your feedback and suggestions. Your input is invaluable in helping us shape the future of Canvas.

[jb2483929]

For those of us who do not have Canvas Data set up, can you tell us how we can find courses/discussions where this feature was turned on? What will this look like to teachers/students in Discussions where it was enabled; what about if some grading had already taken place?

[james_whalley]

@SamGarza1 When is “immediately”? We want time to make a decision and contact the CSM if we need to keep it on. This is our first week of semester and people are already building discussions with this feature.

[Nancy_Webb_CCSF]

Thank you for listening Instructure!  That's wonderful news.

@jb2483929, I tested turning off checkpoints in Beta, and all graded discussions retained their scores.  The discussion due date was lost though.  Will that be the same if Instructure disables it everywhere?

Also, once checkpoints had been chosen but disabled at account level in Beta, a message appeared at the top of the discussion about asking the Admin to turn it on, The message appeared to students as well, at least when I masqueraded as a student.  I hope that will not appear to students.

[mwolfenstein]

Thank you @SamGarza1 for the clear and concise update. Is there a reason why Checkpoints wasn't rolled out as a feature option at the course level? If there's any way that it could be, it would definitely provide the best option in the short run since you'd be giving faculty users the agency to decide whether they like the new SpeedGrader experience or are willing to deal with it as a trade-off.

Additionally, I'm a little concerned about any graded discussions that faculty may have created with the checkpoints feature in the meantime, especially if they have already entered grades. which is probably less common but definitely not impossible.

[James]

The survey is pretty good, but the first main prompt is basically so open-ended that I felt it was asking for everything that was wrong. I typed a lot, just to realize that you later asked about those very items I was complaining about in the first message (for example, having the "reply x of y" status). There is no back button to go back and fix the earlier prompt, but it would be nice to warn people about how it's laid out and what to expect. Then, at the end, it would be nice to have the "is there anything else?" prompt in case we forgot something at the beginning or the prompts along the way made us think of something else.

[olssond]

Great news! When this returns, you may want to consider this being a course-level feature option.

[james_whalley]

@SamGarza1 I found in my testing that shutting off the feature in beta cause all due dates from all Discussions using Checkpoints to be removed. Furthermore, the discussions I checked that already had posts before removing the feature then did not display current or future posts in the SpeedGrader. Discussions that didn't have posts and then received posts only after the removal displayed fine in the SpeedGrader. Re-enabling the feature brought the posts back to the SpeedGrader.

[AnnLoomis]

Thanks for this info @james_whalley. Our campus went ahead and disabled Checkpoints yesterday, and I also found that dates disappeared entirely from Discussions that had been using Checkpoints. However, I was not aware that "the discussions I checked that already had posts before removing the feature then did not display current or future posts in the SpeedGrader". If this is the case, this is a huge issue, and Instructure needs to address it immediately. 

[SamGarza1]

Hi All, @jb2483929 @james_whalley @Nancy_Webb_CCSF @mwolfenstein @James @olssond @AnnLoomis 

Thank you for your patience as I work to answer your questions.

If your institution has enabled Discussion Checkpoints, it remains enabled. Your institution can disable it by adjusting the feature flag.

When the Feature Flag is disabled, Checkpoint Discussions will show a banner message informing instructors that it was disabled. Instructors will need to edit the discussion to update the due date. Points, assigned users, and existing submissions will remain unchanged. Please report any issues to our support team.

We're looking into what reporting we can offer institutions. 

And thank you to those who've participated in our survey. Please feel free to share it out to others as it will remain open for the next couple of weeks. 

[james_whalley]

@SamGarza1 I did some further testing regarding SpeedGrader visibility of posts. If a student has posted to a discussion before the checkpoint is disabled and makes no more posts, those posts are not visible in the SpeedGrader. If they make more posts after the change, all posts show in the SpeedGrader.

[cheryltice]

Hello, @SamGarza1,

I appreciate the transparency in your post. At the same time, I'm concerned about an emerging pattern. This situation echoes issues with the Discussions Redesign rollout, where rushing to production without adequate workflow testing led to disruptions that still persist in the community today.

Now we're seeing SpeedGrader changes impact our already challenging discussions interface. Both cases suggest a need for more thorough testing and attention to community feedback before deployment.

Some questions:

1. How is testing done? Are instructors with diverse class sizes included in the process, or is testing primarily developer-driven? How are community comments incorporated into the testing process?
2. What concrete steps will Canvas take to more effectively integrate community feedback before major feature rollouts?
3. For future updates, how will you implement a rollout schedule with clear benchmark dates to protect existing workflows? 

While we value Canvas for its reliability and ease of use in teaching and learning and appreciate innovation, I think that stability and workflow preservation must be prioritized. I strongly encourage establishing a more rigorous pre-release testing protocol that gives appropriate weight to community concerns. 

Looking forward to hearing your plan to address these issues.

Thank you!

Cheryl

[ACook-RVCC]

@james_whalley , so based upon your tests in a Checkpoint-enabled Discussion, if every student were to just go back in and post one more time AFTER Checkpoints are turned back off, it will cause all previous posts to become visible again in Speedgrader?

[james_whalley]

@ACook-RVCC From my limited testing, yes.

[Nancy_Webb_CCSF]

@james_whalley, does the speedgrader problem (student posts not appearing) only apply to discussions where checkpoints was turned on?  If the discussion didn't use checkpoints, do all the posts appear?  Thanks!

[james_whalley]

@Nancy_Webb_CCSF My testing showed that discussions that never used checkpoints were unaffected and showed fine in the SpeedGrader. I didn't run tons of tests, but it makes sense to me that this would be the case.

[RebeccaLarge-Sw]

I find the new speedgrader version not very speedy. It is cumbersome and I would prefer to use the prior version. My co-teacher and I give feedback on every post. We have already read the discussion threads so the new version does not help us at all.

Rebecca Large-Swope

Manage School Libraries SCHL 5160

[jmurray6]

@james_whalley , @ACook-RVCC , @Nancy_Webb_CCSF 

Regarding the posts not appearing in speedgrader issue. At this time, I have at least one faculty member with 4 courses utilizing discussions with checkpoints. Thus far, we have NOT disabled the feature (we're currently polling our faculty to see what the consensus is). So, in these 4 classes - the checkpoint feature is active and she has enabled checkpoints on her discussions. She has her gradebook rules set to autopost 0s on assignments with no submissions. As her discussions closed on Sunday, she got several contacts from students asking her why they got 0s when they did complete the discussion. Upon investigation - speedgrader said those students had no submissions/posts for that discussion forum, but if one went to the forum and searched for the posts - they were there. So it appears that the "posts not appearing in speedgrader issue" may be more widespread (and somewhat random) that what has been discussed here so far.

It wasn't every student - only a few in each course, and thus far I can't identify a pattern that might give some insight into why.

[james_whalley]

@jmurray6 I was not aware that there were issues with posts not showing in the SpeedGrader when the Checkpoints feature is still enabled. I hope you submit that case to Canvas support. Please keep us informed as to how that unfolds.