Discussion Checkpoints: Beta Update and Next Steps

[SamGarza1]

We’re excited to share an update on the progress of Discussion Checkpoints! Before moving the feature into our beta environment, we had the privilege of working with four institutions that tested the functionality. Their feedback has been invaluable in helping us refine the experience and ensure this feature will be valuable to our users.

As we continue to address any UX pain points and reported bugs, we encourage you to keep testing the feature on beta and report any issues directly to our Support team. This will help us resolve any remaining concerns before the official production release in November.

While Checkpoints is still in beta, we’re paying close attention to two key areas:

Language on each checkpoint: We're exploring ways to ensure the instructions and expectations are clear for both students and instructors.

Rubrics with Checkpoints: We’re gathering feedback on how rubrics interact with each checkpoint, including whether separate rubrics for each stage of a discussion would enhance clarity or complicate workflows.

Your feedback is critical to the continued improvement of this feature. As you test Checkpoints, please share your thoughts by leaving a comment on this blog or reaching out to your Customer Success Manager (CSM). 

Thank you for your partnership and support as we move closer to the production release of Discussion Checkpoints.

[venitk]

@SamGarza1 Are you keeping track of the comments made in the release notes FAQ? https://community.canvaslms.com/t5/Canvas-Releases-Q-A/Features-Q-amp-A-Canvas-Release-Notes-2024-11...

[KellyMurray]

I know it is mentioned above, but I want to reiterate that the loss of functionality of the Rubric for grading purposes when Checkpoints are used in a Discussion is a major concern and creates a really unexpected, negative trade-off for users. Communicating to instructors that they will need to decide between using the long-awaited Checkpoints OR being able to use their Discussion Rubrics will be a challenging and frustration-inducing conversation for all involved. 

[rmartini]

First, I’m very excited about being able to add multiple due dates to discussions so thank you for the work that’s been done so far! That said, I spent a good part of last week testing checkpoints and came across a few issues that make me nervous about the November 16 production release (particularly related to group discussions, the Assign To button, and needs grading indicators). If I've opened a case about the issue, it's noted below.

Group Discussion Not Created

This took some time for me to untangle, but in short, if an instructor goes to create a new discussion and enables settings for checkpoints and group discussions at the same time, no group discussion is actually created. The result is a single discussion topic where all students can comment (the group discussion setting still showing as enabled). When this happens, the buttons in SpeedGrader to move to the next/previous replies also seems inoperable and the instructor is just presented with the full discussion board for grading. Definitely a big problem if this moves to production!

By comparison, if I create a group discussion (no checkpoints), save it, then go back in to edit the settings to add checkpoints, everything seems to function like I’d expect (except that the View Due Dates tray doesn’t show the correct number of required replies).

I opened a case about this already (case 11310688) and it’s currently with L2.

Additional Replies Required Overwritten When Using "Assign To" Button

When I create a discussion with checkpoints enabled, save it, and then go to edit something (like changing the Reply to Topic Due Date or availability dates), it overwrites the original “Additional Replies Required” setting back to 1. I observed similar issues if I also tried to differentiate the assignment (by giving one student different due dates) with the Assign To button/tray. After saving the differentiation from the tray and then going back to edit the discussion’s settings, it didn’t seem to save the differentiation (it resets the additional replies back to 1 and Assign To still only showed “Everyone”). I didn’t seem to have the same issues if I made those changes on the main settings page, but something definitely seemed off. I opened a case about this as well and it’s also with L2 (case 11304071).

Needs Grading Status Inconsistency

If a student only partially meets requirements for the discussion, I would expect there to still be a needs grading symbol in the grade book and for it to appear in the teacher’s To Do list. That doesn’t seem to be happening (or at least it’s inconsistent). For example, if a discussion requires 1 reply to topic and 3 additional replies, but the student only makes one initial post and 1 additional reply, the instructor does not see a Needs Grading indicator (in the overall gradebook or SpeedGrader drop down). By contrast, the To Do list will contain separate links for each "task". Teachers will see one link for “Reply to Topic” and another link to “Required Replies (#)”…only if the student meets the criteria for all of them. So in that same scenario where a student made 1 initial post and only 1 (of 3) required replies, the To Do list will say the teacher needs to grade 1 Reply to Topic but give no notification for "required replies." I opened a case about this but was told I should submit a feature request since the tool was still under development…which seemed wrong? The case is 11311206.

Heading Levels on Discussion's Settings Page

After the redesign, I assumed that all of the settings had used headings to separate the different sections since the text was larger and looked like headings. After looking at this with Checkpoints, that doesn’t seem to be the case. Most of the bigger words are just text (such as Options, Checkpoint Settings, etc.). When I did a quick automated test with axe DevTools, it flagged an issue with skipped heading levels because the title of the discussion is an H1 and Peer Reviews is an H3 (but none of the other text “headings” are designated as headings). I opened a case for this as well (11317051).

New SpeedGrader Workflow

I’m not a fan of the new grading workflow in SpeedGrader that displays the entire discussion. I understand wanting to see a comment in the context of a discussion (it’s definitely needed sometimes!), but I don’t understand why instructors are presented with the full discussion in SpeedGrader. There’s so much extra information that is displayed and is generally irrelevant for the immediate grading purposes. Instructors, TAs, and graders basically have no ability to quickly scan the selected student’s comments (like they currently can) and their only choice is to see the posts one at a time. It also takes some time for the next reply to load and be highlighted. The current (production) view nicely displays all of a student’s posts on the same page without having to navigate anywhere / any additional load time. I'm expecting a lot of complaints about this...

Behavior of "Next" Button After Last Comment

When using the Next button to navigate through a student’s responses and you get to the student’s last reply, the instructor is then brought back to the initial post. They basically end up in an endless loop. Since the interface on the left is constantly shifting to highlight new responses, it’s not always easy to tell when you’ve reached the last comment and started over since there isn’t an indication of the number of comments…like “1 of 5 replies from [Student]”. By highlighting only one post at a time and threading ending at a certain point, it’s not easy to take a quick assessment of this.

Accessibility Within SpeedGrader

This also seems like it’d be more challenging for an instructor who utilizes a screen reader. While this may not technically be inaccessible, it makes grading more tedious and challenging because instructors have to navigate through the entire discussion each time they want to grade a student. If I tab through the interface, I see extra “Previous in SpeedGrader” and “Next in SpeedGrader” buttons that seem intended to make navigation easier…but it takes time for the new entry to load and focus ends up on the Next (or Previous) button rather than the start of the post (so a screen reader user needs to know to arrow up or Shift + Tab up to the start of the student’s post). It was also difficult to navigate from reading a student’s post over to the right side of the screen to enter a grade. I could use the button to jump to SpeedGrader navigation but that actually brings me over to the the left arrow before the selected student’s name. And then tabbing some more brought me into the full discussion rather than the grade entry on the right. I’m not an accessibility expert, so I may be wrong, but this wasn’t the greatest experience for me when I briefly tested this with JAWS.

New SpeedGrader view for discussions in Beta with Checkpoints enabled

Extra White Space at Bottom of Discussion

There seems to be a lot of extra white space at the bottom of the discussion (in a separate container?). When I get to a discussion, I always see two scroll bars. In my screenshot, the left/red scroll seems to apply to the discussion topic; the right/blue dash scroll bar brings me down to a lot of white space. It’s unclear why this is there. I opened a case about this as well (case 11316260), and was directed to comment in the community.

Extra scroll bar and lots of white space below discussion in SpeedGrader

Student View (from Grades)

It seems like the change in workflow also affects student’s view. When a student has received a grade and there are comments from the instructor, they can click the “comment icon” on the Grades page to see comments. But they can also click the name of the discussion to view additional helpful details like their submission (to view their replies), the rubric, etc. In production, that view currently looks like this:

Example of what a student sees after selecting the title of the discussion from Grades

In the new view in Beta, students are presented with the entire discussion forum. On the initial page load, they see a highlight of their initial comment but they don’t have any of the navigation tools that the SpeedGrader/teacher view has. To find their posts, it will require a lot of extra clicks...and I imagine most would just give up rather than trying to search for their specific posts.

Example of what a student will see after selecting the title of a graded discussion (in beta)

Late Status Not Automatically Applied

Related to the gradebook, I was hoping to see that the late status flag would automatically be applied to the “Reply to Topic” and “Required Replies.” That doesn’t seem to be the case and must manually be entered. Ideally, I’d like to see some kind of indicator that something was late. Currently in SpeedGrader, you’d see a little pill that says “late” which can be helpful for grading. In this new version of SpeedGrader, there’s no similar indication that one or both of the checkpoints were submitted after the due date.

I don’t have a ton of experience with using grade policies, but it seemed to deduct the expected number of points once I entered a score and applied the late status (which was good!). 

Using Rubrics for Grading

I saw the note in the FAQ document that rubrics can’t currently be used for grading but still wanted to test how this would function. Like @KellyMurray said, I think this will be frustrating for instructors. It’s very convenient to just select the boxes and have the scores autopopulate. In my testing, I could still add a rubric to the discussion and check the box to use it for grading (which I wasn’t expecting). When I actually got to SpeedGrader, I could use the rubric to make selections that would be shared with students (so they could see the criteria I selected) but I had to manually update the numerical score separately. I know this is still being worked out. But I’d second @KellyMurray 's comment that the current experience will be confusing for instructors.

I lean towards preferring one rubric to grade discussions in SpeedGrader but I know that can get complicated to figure out how to allocate points for each checkpoint with only one rubric.

Shifting Dates After Importing to New Course

Surprisingly, this worked well! In a new course site, I selected to import content by copying a Canvas course. I used the setting to adjust dates and it shifted everything as expected. So although the checkpoint dates aren’t currently pulled into the Edit Assignment Dates page, it seems to correctly shift the dates when importing for a new semester…which is GREAT news! But I would really like to see those dates on the bulk edit page someday too!

[hollands]

I have not had a ton of time to test this out yet but it seems like Rubrics can work in the sense that you can add a rubric to a discussion with checkpoints, but when clicking to assign criterion but it doesn't apply the point value (if there is one associated) with the criterion.  For our faculty it's likely more of a frustration point to have an extra setup, but if I guess correctly it's something they'll trade off to have checkpoints. 

My question is why? Is there a reason this was separated out? It doesn't seem like adding points to posts/replies creates two point fields so a rubric wouldn't have to differentiate between the two. Is it because rubrics are going through a redesign as it is and it makes more sense to wait? I know I'll get these questions so I'd love to have some information. Thanks! 

[sbing]

We have nursing faculty who have an anxiously awaiting checkpoints for years and I have them working in beta and here is their response:

In the speedgrader I get a message “Rubrics do not auto-populate grades for checkpoints”,

So I can grade with a rubric for the main post but not replys? The rubrics we have criteria for replies also. So do you have to enter the reply grades separately?

They also stated that separate rubrics for replies would create more work. All they really need is the second date for reply’s. 

[SamGarza1]

Hi @rmartini , 

Sorry to hear you were having issues posting your comment. Thank you for providing detailed feedback. 

• Group discussions not being created is a known issue that we're investigating. It's a tricky one as it doesn't seem to be happening consistently for all users but we are working to resolve it.
• Thank you for sharing your support case numbers. I believe this has been experienced by a few others and our team will be looking into it.
• I'll raise the grade book needs grading status issue with the team. The To Do list not having a link to grade it if a student hasn't met the required number of replies is the expected behavior but it sounds like it's not matching what you would expect. Just to confirm would you expect it to always show? Or only after at least one reply?
• Thank you for reporting this as it is an accessibility issue. I'll make sure this gets ticketed and resolved. 
• I appreciate you sharing your feedback and raising the issue of how students view it. I've captured the additional information you shared in this post to share with my team. 
  
• This is a known bug that our team is working on. 
• Thank you for identifying this issue. I'll make sure our team addresses it. 
• On using rubrics with checkpoints is your preference to have only one for the entire discussion? Or one per checkpoint?
• Glad to hear it! We will be working to support bulk edit assignment dates after the release.

[SamGarza1]

@venitk Yes and apologies for the delay. 

@KellyMurray @hollands @sbing 

Hi, responding to you together since you had similar questions and concerns about rubrics.

The current experience is that you can still have a rubric associated with a checkpointed discussion in the same way that you could with normal graded discussions. For checkpointed discussions though it can't be used to auto-populate the grade after marking the rubric. This is due to a technical issue that we're investigating. 

We're evaluating different solutions for this because the current experience isn't ideal and does currently require instructors to do things twice. The option we're evaluating based on technical limitations and feedback from our early testers is allowing for a rubric to be associated with each checkpoint. This would then allow rubrics to auto populate the grades for checkpoints.  I encourage you to share feedback on how this would work for your institution and what the ideal workflow would be. 

And please understand this may change if we run into an unexpected issue or have a eureka moment. 

[venitk]

@SamGarza1 

Forgive me if this doesn't make sense, I'm trying to quit caffeine and am struggling on this dark, rainy morning.  

I'm responding to your question "would it be more valuable to have a single rubric or one per checkpoint?" from the Release Q & A discussion because I figure that release Q&A page will be hard for me to get back to after the release. 

If there were two rubrics (one for the original post and one for the responses):

Here's an example of the undergraduate rubric. To have a rubric associated with the checkpoint, we could simply separate the response post criteria and associate that with the second checkpoint, but that would lead to confusion when the instructor grades the timeliness and writing quality, which would presumably apply to the original post and the responses. If writing quality is only associated with the original post, students would understandably think that only the original post was evaluated for grammar (when in fact they would both need to be).  

So, to use rubrics associated with checkpoints, we would either need to repeat all the items for both checkpoints (so that would mean 6 criteria total would be assessed = more clicks for the faculty), or, to maintain the number of criteria at 4, would need to condense timeliness and content into one criteria, then have writing quality stay as its own (we're required by the college to assess writing as 10% of the grade, so that would have to stay separate). That would mean the original post and the responses would have two criteria: Timeliness/content & writing quality. And I don't love the idea of combining timeliness with content--it's helpful for the instructor to split those out. (as a side note, I don't know anyone who uses the automatic deduction to take off for late points.) 

Either solution would probably not be worthwhile for us to just be able to use the checkpoints. 

undergraduate discussion rubric

Here's an example of a graduate rubric. You can see how it considers the original response and the replies as a whole. We'd have a similar problem here, except in this case, to make two rubrics, we'd need 8 total criteria. 

graduate discussion rubric

If it were one rubric 

I can't picture how this would work. How would Canvas know how to divide up the points? By percentage? Would just one item in the rubric be associated with the checkpoints (see my original point above that that would lead to confusion)? How would students know which criteria in the rubric were for the original post and which for the replies? 

I don't have any use cases in HE in which it would be helpful to have a separate grade awarded to the checkpoints when there's a rubric involved. To work with this system, any rubric would either be holistic (which none of our courses use), be pretty basic, or have duplicated criteria. 

I'd love to be wrong, though! How would other schools use rubrics attached to the checkpoints? 

It would be easier for us to continue to use pages with to-do dates to indicate when the responses are due. The best thing for us would have been an additional due date added to the calendar, to-do list, and module that linked to the discussion, but I'm sure there were people asking for checkpoints to be more robust than that.  

[KellyMurray]

Thank you for the reply, @SamGarza1. I appreciate the attention to - and consideration of - this issue! In reply to your question regarding having 1 rubric or 2 that can be used to auto-populate the score for grading, I have follow-up questions are about how this might work that would influence my answer:

• Would having 2 rubrics (1 for initial post and 1 for follow-up) require opening SpeedGrader twice for each student? If so, I would NOT support this option as this will likely feel to instructors that this is doubling the grading time/effort.
• Would it be possible for the 2 rubrics can be within a single SpeedGrader window for a student? If so, then I think this could work since there are points associated with each checkpoint (initial and follow-up) and the 'populating' of the score from the rubric would go to the proper checkpoint (initial versus follow-up). This could require some duplication of criteria when creating the rubric, potentially (as described by @venitk ) if the instructor wants to assess certain things overall/across both initial and follow-up. As an instructor, this seems a bit annoying but doable. However, in this situation (where criteria have to be repeated), there may not be a time-savings, ultimately, versus having to enter the points manually as it functions currently.
• Can a single rubric can be used in a way that indicates which points are for the initial post and which points are for the follow-up posts - and then those points can auto-populate the score for those items? This seems like the best option to me. This would require options while creating the rubric to indicate which points go where. I have no idea if/how that could be accomplished.

I spent many years as an instructor of large classes and I developed rubrics to help increase efficiency of grading Discussions. The ability to quickly click on the rubric as a means to provide some feedback while also auto-populating the score was a key to this efficiency. With the current process (not auto-populating scores) the Checkpoints are going to diminish grading efficiency. This will be most evident to instructors of large classes - especially since the rubric no longer auto-populates AND the instructors will now have to spend time partitioning and entering 2 separate scores.  

[arovner]

I want to thank @rmartini for their very thorough/ detailed and thoughtful post. You captured so many things that are not really "production" ready that are causing inaccessibility and challenges for our students and faculty. I'm glad you have submitted tix for some and I really hope Instructure folks are listening! 

[tom_gibbons]

Ooof. This is rough. 

For 12 years, instructors have been asking for the ability to assign multiple due dates for discussions. I think most of us have found reasonable workarounds for the grading questions: noting the point values in the prompt, labeling rubrics for clarity, and so on. 

It's been nearly two years since the discussion redesign was announced as basically complete, and users were like, "But what about checkpoints?" January 2023, iirc. 

So, now we have beta checkpoints to play with. 

The minimally viable product was literally just multiple due dates for a single discussion. But now we have additional issues with sectioning off the grading, getting rubrics to work correctly, what does the UI look like for students, etc. 

I get that splitting off the grades is nice to have. I get that there was probably a strategic decision to use the existing components to link two assignment items together with one discussion item (rather than the current 1-1 linkage between a discussion and a graded assignment to create a graded discussion), which would allow multi-component assignments in other contexts of Canvas. I really do understand the business rationale. 

But still. Ooof. It's not hard to identify the relevant user stories re: how discussions are used instructionally. Adding on the split grading component--again, nice to have--just muddies the waters around the primary use case for multiple due dates: nudging students for engagement. 

[rmartini]

@SamGarza1 - My answers to your questions are below. Thanks for following up on the feedback and sharing it with the team!

Regarding Needs Grading Status: If a student partially meets requirements, I would ideally like to see some kind of needs grading indication if there's at least one reply. It would be up to the instructor as to whether or not to give partial credit, but I can see many scenarios where multiple replies may be required for full points but an instructor will give half the points for half of the required replies (or some other kind of partial credit). Right now, though, an instructor would not see anything to indicate that there's anything to grade unless the student submits at least the required number of replies. Also, I think it's important to consider that students may make mistakes and accidentally reply "incorrectly"...and I'd want to avoid instructors then accidentally overlooking something that should be graded. Basically, if there's any kind of submission, I would want to see a needs grading indicator as the instructor. 

Regarding Rubric(s): Ideally, I'd probably prefer one rubric for grading. Like @venitk pointed out, to take an existing rubric and separate out the initial post points from reply posts is more work just to change the rubric and in the end likely adds more clicks for faculty when actually using them. And like @KellyMurray pointed out in the last bullet point, I'm not sure of an easy way one could designate where points flow within a single rubric (if the Reply to Topic and Required Replies fields are kept separate). When I was working as an ID, there were often minimum posting requirements in our courses (usually 1 initial reply and at least 2 reply posts). Some rubrics intentially separated out points for the initial post from points for the reply posts, while others didn't. But in all cases, instructors and TAs grade initial posts and replies at the same time (other than the exceptions where students post after the original deadline, etc.). We always built a single rubric for a discussion...so creating two rubrics for a single discussion would be an unexpected change. I don't know if there's some middle ground that could be created by having a checkbox where instructors could designate that they want to use two separate rubrics (if they want to use two separate fields in SpeedGrader) or use a single rubric (and just see the single, autopopulated total when grading with a rubric). In most use cases, if I needed to separate the initial and reply posts, I would use the rubric to create that separation rather than expecting there to be two separate grade entry fields when grading.

The question about how rubrics behave prompted me to take a closer look at how checkpoint grades appeared for students. I missed it in my initial testing, but it looks like students can select an arrow to expand information about discussions which displays separate rows for the initial post/reply to topic and replies. Note, though, I think the language here needs to be changed where it says "Reply To Entry (#)" instead of "Required Replies (#)".

Student Grades view with checkpoint grades expanded

On some level, I can see two rubrics making sense here...but if I'm trying to think as a student, I can't think of a situation where I'd expect two rubrics to be associated with a discussion since it's all part of the same assessment. I do like being able to see the status flag/indicators for each of the "tasks." That can be helpful. But since those aren't automatically displayed (unless the section is expanded), I'm not sure students will be likely to notice it...one click on the arrow, one click to access the rubric for the initial reply, then another click to open the other rubric... The screenshot below is what students see when they initially open the Grades page (they don't see the late/missing flag and won't see the due/submitted times):

Default view of Grades page with the discussion (with checkpoints) collapsed and no due date, submission date, or status.

I'd also second much of what @tom_gibbons commented on in that instructors really wanted a quick way to connect a discussion with a second due date to get into the calendar (without needing to create an extra page or assignment). The biggest benefits I can see with checkpoints is that students will see two calendar dates and also see the "required replies" date automatically crossed off when they meet the requirements. But there are definitely some trade offs at the moment.

[SamGarza1]

Hi @venitk @rmartini @KellyMurray 

Thank you so much for the detailed feedback and screenshots. I'll be documenting these so we can reference them as we move forward. 

And in case you haven't seen, I posted a blog today letting users know that we're releasing Checkpoints as a feature option instead of turning it on for all users. 

[callinger]

Many have already pointed out the rubric issue, but I haven't yet seen the exact solution that I would prefer to see be an option.

Please, can there be an option to have separate due dates for checkpoints but NOT separate point values??

Essentially, we want an option to have a required number of replies (and corresponding due date) WITHOUT impacting the point value/grade at all. 

For example, this rubric looks at discussion participation more holistically rather than as separate grades for initial posts and replies. Having the option to parse out points between posts and replies can be great for folks who want that. But many people JUST WANTED TWO DUE DATES: one for the initial post and one for replies to peers. 

It's like Instructure said, "Hey, here's this thing you've been asking for over more than a decade, but you can only use it if you change your entire approach to grading online discussions." Thankfully, this will work for some folks. It's just a shame that we can't just get the part we have been asking for.

[tmalone1]

@callinger I agree, that option would be excellent. 
Faculty should be able to create a discussion with a Post date and Reply date with our without separate points. 

[hansonav]

Hi @SamGarza1  ,

Thank you for your information and engagement on this topic as checkpoints are working on being enabled. I want to echo a number of the points and concerns listed from various community members here as well. One, in particular, I want to highlight from @rmartini is both the accessibility concerns of navigating the discussion posts with a new view in SpeedGrader as well as elaborate concerns broadly for the impact and function for instructors. The tabbing and double scrollbar/white space issue is both an accessibility concern and a usability concern in general.

Within Beta, regardless of enabling checkpoints, it looks like the discussion view within SpeedGrader is changing from how it currently is in production, is that correct? The accessibility needs are a priority to address, but additional considerations with the design impact what instructors use the view they have in SpeedGrader to do - speed up their grading. Now, an instructor will have to make multiple clicks to utilize the "next reply"/"previous reply" buttons, individually search for student names, or spend additional time scrolling and seeking. For example, if an instructor has 40 students and each student needs to make one reply to the topic and 3 replies to peers, that adds 160 clicks to their process for navigation rather than just scrolling to see the posts from one student at a time. Add in with checkpoints, if they use a rubric, they will still need to click to fill in the rubric AND add two separate scores since the rubrics cannot auto-populate currently.

In isolation, that doesn't feel like a lot, but then double those numbers or multiple by multiple sections of a course, across multiple weeks in a course, and that changes the workflow for an instructor. Currently, there is a link at the top of SpeedGrader in a discussion that would open the entire discussion if someone wants to view it in SpeedGrader. In my own courses, there are times that even after reading and engaging in the discussion with my students prior to using SpeedGrader, I still need to review the context of their posts and use that option, but overall, when I am ready to grade, I am looking for the efficiency of reviewing their posts.

Are there considerations or possibilities to have different ways to view a discussion when grading? If this change goes into effect even without checkpoints enabled for a discussion (as it seems to be in Beta currently), I'm worried that this change near the end of a term when instructors are going to be heavily reliant on SpeedGrader will cause a lot of disruption.

Thanks!

[dbrace]

Just so everyone knows, the release (even as a feature flag / option) of multiple due dates (checkpoints) in discussions has been delayed.

Canvas Release Notes (2024-11-16)

Features Q&A: Canvas Release Notes (2024-11-16)