Canvas Rubric Redesign: Progress, Challenges, and Next Steps

[RaviKoll]

We know you have all been waiting in anticipation for the all new rubric redesign. First and foremost, we want to express our gratitude to each and every one of you who has shared your thoughts, suggestions, and feedback with us. We're here to share an update on our progress and some adjustments to our timeline.

A Complex Redesign: As we dive deeper into the redesign process, we've encountered some unforeseen complexities. Redesigning the Rubrics functionality is no small feat – it's a multifaceted endeavor that requires careful planning, testing, and iteration. While we've been working tirelessly to bring you a revamped Rubrics experience, we've encountered some hurdles along the way that have led to a slight delay in our timeline.

An Extended Timeline: We understand that delays can be frustrating, especially when you're eagerly anticipating new features and improvements. That's why we want to be transparent about our timeline adjustments. The Phase 1 of Rubrics redesign will be released in the beta environment in Q2’24, with a production rollout due with the subsequent release cycle. The release will be implemented under a Feature Flag, giving you the flexibility to enable or disable it according to your preference.

Changes in the Pipeline: In addition to the revised timeline, we also have some updates regarding the features that will be included in the June production release. While the majority of the enhancements will still be part of Phase 1, we've made the decision to push the implementation of Student Self-Assessment to Phase 2, which we understand isn't ideal. However, the complexity involved in its implementation, including modular inter-dependencies and architectural considerations, has led to these delays. Despite this, we didn't want to hinder the overall release progress. Our goal remains to get the redesign into your hands as early as possible, even if it means a slight delay for Self-Assessments. 

We've received numerous requests for Rich Content Editor (RCE) integration into the Speedgrader experience, and we want to assure you that it's a key part of our roadmap. While we don't have a specific implementation timeline just yet, we're actively working on it. Rest assured, we'll keep you informed and updated as we make progress in this direction. Stay tuned for further updates!

Numerous enhancement requests have been received, and we want you to know that we're actively evaluating each one. Our team is committed to carefully considering all your valuable feedback during the redesign process to ensure that Canvas Rubrics aligns with your needs and exceeds your expectations. However, we may need to prioritize certain enhancements over others, and some suggestions may require further evaluation before implementation. 

Despite the challenges we've faced, we're more excited than ever about the future of Rubrics within Canvas LMS. We're confident that the changes we're making will streamline your workflow, enhance the assessment process, and ultimately empower both educators and students.

Before we sign off, we want to extend a heartfelt thank you to each and every member of the Canvas Community.

We're excited to offer you an exclusive preview of our ongoing progress with the rubric redesign initiative. Check out this brief video for a sneak peek at what's in store!

[venitk]

@James_Kocher_UF 

James, we have the same concerns about the rubric redesign. I'm at the Distance Teaching and Learning Conference and spoke to a CMS here, about the rubric redesign and she encouraged me to 1) find allies who have the same concerns and 2) get them to contact their CSM. So please share these concerns with your CSM and not just here, because she said they tested the rubric redesign extensively and these issues didn't come up. 

[DMSDave]

With the new rubric design rolled out, I would like to request a couple of features which some may find helpful like myself when utilizing the rubric to grade an assignment:

- The ability to disable this, or return to a "Classic" view
- The ability to resize the rubric pop-out when grading an assignment
- The ability to pop out the rubric to it's own window

The difficulty I personally have with this change is due to the complexity of some of the assignments I grade which may include 30+ images, and my rubric contains around 20 fields. I typically comment on each image in the "comments" section when going through the images, and then use that as a guide for completing the rubric, but it's now so large that I cannot see my comments. Happy to elaborate further!

[James_Kocher_UF]

@venitk  Already in talks with them. And of course, we're in the middle of a CSM shuffle as well!  Here's a list of issues we're having that I'm sending on to them:

• Can’t use traditional view if a criterion has more than 5 ratings.
• Students can’t see rating descriptions anymore (with L2 support)
• Teachers can’t edit rubrics completed by TAs or other teachers. Was a previous functionality. Now they must grade with a whole new rubric to change one rating.
• Filling out a new rubric replaces the existing one. Previously, you would be able to see all rubrics.
• Rubrics completed by others are marked “peer review” at the top.
• Rubrics slide out in speed grader now. Traditional view overlaps the assignment area, so grades can’t see the whole submission. There’s not a way to adjust the window sizes as you could previously. The horizontal/vertical view looks too compact, and the text looks slightly cut off.
• With criterion that have ratings that are all zeros, no matter which rating you click on, it will default to the far most left.
• When “I’ll write free-form comments” is enabled, students aren’t able to complete peer reviews.

[RaviKoll]

As always, I want to emphasize that your comments and feedback are invaluable and insightful. Thank you! Here's a quick update on some of the enhancements we will be rolling out immediately as refinements to the existing rubric redesign:

• The ability to lock or unlock the new rubric redesign experience, allowing instructors to test the new rubrics.
• Improved experience on SpeedGrader for the redesigned rubrics, ensuring better cohesiveness and usability.
• The ability to save rubric inputs on SpeedGrader without having to submit them. 

Please reach out to your CSM if you observe any unintended behavior in your instances. We will address it as soon as we can. We understand that the release isn't perfect; this is the initial version, and we hope you'll continue to support us as we navigate through the bugs and inconsistencies. We appreciate your patience as we work through these issues.

We have several other items on our roadmap, including incorporating the new rubric experience into the assignment section, enabling the sharing of rubrics across courses, importing and exporting rubrics, enabling student self-assessments, and more! Stay tuned as we work through these enhancements, driven primarily by your insights and feedback.

[James_Kocher_UF]

@RaviKoll Thank you for that update! Will you post here when they go live? Also, is there a page here that has roadmap and timeline, etc? I wasn't able to find one of there is.  Thanks!

[RaviKoll]

We will soon publish the timeline for upcoming releases. 

[tmalone1]

Greetings,

@simone_laughton 
@pual_fynn
@JamesSekcienski
@ldavenport4015

I am also VERY concerned about Outcomes. 

We have system-level outcomes that faculty import into their courses and add to their Rubrics. These outcomes are NOT edited by the faculty. These are ungraded criteria that help Cal Poly meet accreditation requirements. 

We have many years of outcome data and have plans for future data. Any changes to this process will be catastrophic. 

If you want to learn more about how we are using outcomes and rubrics, please feel free to reach out. 

Tonia

[tmalone1]

Greetings,

This morning, I had a chance to review the NEW Rubric redesign in BETA with our Outcomes.

What I found was not quite what I hoped. 

Although I can search in the course for the system outcomes and add them, then in the NEW Rubric include those system outcomes...

When picking NOT to use the criterion for scoring...

...it still has point on the Rubric.

This is an important part of our process. Faculty do NOT want points for Outcomes. 

On a side note about Outcomes...

It would be nice to have an area to list the criterion description to help faculty choose the correct option. Now, we have to add the full paragraph in the name. Not ideal. 

Tonia

[mhower1]

@venitk @James 

Hello, I've been out of the loop on this update until today, when one of my faculty tried to import a rubric using the Rubric Importer Canvancement.  The button no longer appears.

Any suggestions?  Our entire outcomes-based assessment model is contingent upon the ability to import rubrics generated by spreadsheet. Unless this gets resolved, we need to redesign our entire process (110 courses, 30000 students, ugh...).

[James]

@mhower1 

I responded in the other place you posted. Generally, Canvas doesn't care that their code broke my script. They cannot let forward motion be slowed by customizations other people have made.

Until I can get things resolved, you can still manually create the rubrics within Canvas or import them through the API.

[mdthomas]

Thanks @tmalone1 for posting your concerns about the points displaying even though you selected to not use criterium for scoring. This was an recent observation that I had as well. We are about to use Outcomes for our new General Education assessment and would like both of what you reported to be adjusted in the new design:

1. Remove the points when Use criteria for scoring is unselected, and
2. Ability to list descriptions for the criteria in Outcomes instead of having to put it into full descriptions.

[SuzanneHICKTON]

I just read @tmalone1 's post and omg I feel sick. We are using this to meet our accreditation requirements and we absolutely cannot have points associated with the Outcomes. The entire university is also currently undertaking a project that looks exceedingly looks like it will need to leverage Canvas Outcomes at scale across the institution with no points attached to Outcomes. This is not good 😞

[danaleeling]

I guess I am confused, but in my experience points were removed from outcomes at the point that a rubric was attached to an assignment. As far as I can tell, this is still the case. One has to click on the pencil icon in a rubric attached to an assignment to open the checkbox list that includes removing points from a rubric. I do not recall an option to build a rating scale for outcomes with no points specified. At least not in the institutional bank of learning outcomes. In the image below Optimal was originally 5 points, Sufficient was 4 points, Suboptimal was 3 points, and No evidence was 0 points. I must be missing something about the issue of not being able to not have points associated with learning outcomes.

[danaleeling]

This is a follow-up to a question I asked earlier in this topic as to whether faculty can edit rating scales for institutionally banked outcomes. As far as I could determine, instructors cannot edit rating scales for outcomes pulled from an institutional bank of learning outcomes held in the admin root account. In very limited testing, instructors cannot edit the rating scale attached to an outcome pulled from an institutional bank of outcomes in either Outcomes or Rubrics. In rubrics a pop-up notes that the outcome is locked and cannot be edited. This inability to edit and alter outcomes is necessary so that when outcomes results are exported to external tools the rating scale and values are those that were specified in the institutional bank. For me, this answers a question I posed far earlier in this topic.

[ldavenport4015]

@danaleeling I know exactly what @SuzanneHICKTON is referring to, and I, too, would be devastated if we cannot remove the Ootcomes scoring as part of the rubric. You can remove all the points from the rubric at the point of using it on the assignment, BUT when you align an Outcome to an existing rubric used to grade an assignment and you want to mark the aligned Outcome for student program acheivement,  you can choose to "Not use Outcome for scroring" so as to not effect the student score when 'grading using the rubric" is chosen. 

[tmalone1]

@danaleeling 

After I open the Outcomes in a course and Import some system outcomes....

In the NEW redesign (using TEST) Rubric ...I can...

1. click on the Create from outcomes

2. Click on the outcome name
3. Unselect Use criterion for scoring 
4. Import

...but when it is imported into the Redesigned Rubric...the score appears. 

...I do not see the option to "Remove Points from Rubric". 

The other point ["As far as I could determine, instructors cannot edit rating scales for outcomes pulled from an institutional bank of learning outcomes held in the admin root account. "]...In our PROD system (OLD Rubrics)...

Our faculty are ONLY choosing to add system outcomes as NO "Use criterion for scoring."... for example, this Rubric in PROD has only 5 total points for one criterion that is not an outcome. 

Tonia
CP - SLO

[mbmacdonald]

@RaviKoll - is there a plan for this Feature to be enforced at some point, as recently occurred with Discussions Re-Design? If so, is there an approximate timeline for that? Thanks!

[mbmacdonald]

One of my favorite features of this re-design is the "Locations Used" in the Rubrics index page. This is especially useful for Rubrics built at the sub-account level that are used in multiple courses. It would be great if, in the list of locations used for a given rubric, it could provide more info about the course, such as Term, and Instructor name. This would help program coordinators quickly confirm that their department-wide rubrics are being adopted by all instructors. As it is, many instructors teach courses that have the same name, and continue to have the same name term after term, so the list is not as helpful.

[mcarruth]

The 7/19/2024 that forced vertical layout for the rubric made the rubrics unusable for some faculty.  Faculty need to be able to choose traditional, horizontal,l or vertical for all rubrics.

[kara_atkinson]

From a teacher's perspective, this redesign is awful for workflow and efficiency. I can no longer see the entire rubric with descriptions without clicking on each number, and I cannot see two descriptions at the same if I am going back and forth and really need those descriptors to help score. For team calibration and consistency across classrooms - ESPECIALLY with new teachers who have not yet internalized the scoring rubric - it is IMPERATIVE that teachers can see the full description for each criteria  at the same time when scoring.

We have used Instructure's Mastery Connect since 2021, and we all abhorred the rubric scoring for this exact reason. We begged and pleaded with our administration to be able to use Canvas for rubrics (and Canvas Outcomes instead of aligning standards on Mastery Connect, but that's an entirely different issue). But now, Canvas is updating some of their best features to mirror the headaches that we saw in Mastery Connect. 

As a building leader and classroom teacher, I am extremely disappointed.

[RaviKoll]

We want to assure you that the initial release of the enhanced rubrics is not the final version. Development is ongoing, and our primary aim is to gather as much feedback as possible to ensure that our valued users have a seamless experience. Your concerns about workflow efficiency and usability are important to us, and we are actively working on improvements. Specifically, we're addressing issues related to viewing full descriptions without needing to click on each number and the inability to see two descriptions simultaneously. These fixes are in progress, and our goal is to enhance the experience based on your feedback.

We also understand the critical role rubrics play in team calibration and consistency. We are committed to ensuring that enhanced rubric supports these needs effectively. Next week, we will be launching a dedicated user space in the Canvas Community to streamline feedback and provide more effective communication. I will share the link once it’s available, and we encourage you to continue sharing your thoughts there.

[mbmacdonald]

Hi,

I've just looked at Rubrics Re-Design in beta after the most recent release, and I see that at least one issue has been resolved: you can now view the rubric (in trad view) and the assignment at the same time.

I have read through ALL the comments in this blog post, and I can't seem to figure out what the problem is with Outcomes. I have decided not to roll this out in Production until I understand how this will affect Rubrics with Outcomes... but it doesn't seem like there is any real problem.

Can one of my fellow community members who has done some testing summarize for me (and all of us), what are the potential problems re: Outcomes that this enhanced feature could introduce?

[tmalone1]

@mbmacdonald 

Can you review my post...

‎07-29-2024 05:48 PM from tmalone1

If you have any questions, please let me know. I could Zoom with you and show you if that helps. 

Tonia

[RaviKoll]

We’re excited to announce that we’ve created a dedicated Feature Preview User Space for Enhanced Rubrics on Canvas! This space is designed to foster collaboration, share experiences, and provide feedback as we continue to refine and expand this feature.

You can join the conversation, ask questions, and stay updated on the latest developments by visiting the Enhanced Rubrics Feature Preview User Space.

[TimPacker]

There is just one thing that is holding our organisation back from using this tool in a widespread way which is the inability to highlight a 'partial' statement. So for example a student might get '3' points for something, but my instructors want the ability to highlight a small portion of the statement to indicate that it was not always clear cut (this can be achieved with other workarounds I know, but it remains a sticking point that could potentially be fixed easily?)