I'm Ben Friedman, Product Manager for the Canvas LMS Outcomes Team. I will be keeping track of this thread and answering questions as they come up. Thanks to those of you who have posted comments and questions!
@bbennett2 , @Sopko , and @nathanleesSSD Currently, the Improved Outcome Management feature is paired with the Account and Course Level Outcome Mastery Scales feature, and both “Feature Previews” will need to be enabled. Once the Account and Course Level Outcome Mastery Scales feature preview is enabled for your account, sub-account, or course, mastery scales and calculation methods are separated from individual outcomes and become consistent across that context. The Outcomes page then displays Mastery Scales and Calculation Methods as tabs (see screenshot below). Please note that we’re going to be doing some additional work to bring the capability of setting these at an individual Outcome level back over the next few months within Improved Outcome Management. Stay tuned if that is important to you!
@bbennett2 In terms of adding an Outcome to an Assignment rubric, that can still happen with the new feature enabled. I love that you are giving granular feedback to students through rubrics - that is really taking advantage of the capabilities Canvas offers!
@nathanleesSSD You can indeed customize Mastery Scales in the tab by setting the exact number of levels you would like, changing the names or “descriptions” and “points” for each, and even adjusting the color paired with each level. See more information here.
@nelsonreg You can find the full impact of making a change here. Enabling the Account and Course Level Outcome Mastery Scales feature preview will affect current account data. If a rubric with aligned outcomes has been used to assess an assignment before Account and Course Level Outcome Mastery Scales are enabled, however, the assignment is not affected. Any assignments that have not been assessed by the rubric are affected if mastery scores are adjusted. Please connect with your CSM if you’re considering this so you can talk through the details.
Yesterday our CSM team turned on the Improved Outcomes Management within our "Beta" environment so that we could start to do some testing on our own. We already had already started building what we call "Career & Life Skills" into the "Outcomes" area of Canvas...prior to turning on the Improved Outcomes Management feature preview. We have these stored at the Canvas account level. Here are a couple things I've noticed so far...
- For our rating scale that we used for our C&LS (4, 3, 2, 1, 0), the descriptions didn't fully come over on the "Mastery" tab. For example, the description for 4 is "Meets Expectations - demonstrates all required elements", but when Improved Outcomes Management was turned on for us, the description only showed "Meets Expectations " (including the space after the word "Expectations"). The same held true with the descriptions for 3, 2, and 1 ... where there was a dash between two words within the description. So, it seems that Canvas stripped out everything from the dash to the end of the text, "- demonstrates all required elements". I had to re-add the missing text.
- Related to the first bullet point, after I updated the descriptions of those ratings and clicked the "Save Mastery Scale" button, I was prompted with the message, "This will update all account and course level rubrics that are tied to the account level mastery scale and have not yet been assessed." I clicked "Save". Then, I navigated to my own sandbox course where I had several rubrics already built long before Improved Outcomes Management was enabled. I went to my "Rubrics" screen and found a rubric that also included at least one Outcome that had already been added to it. For example:
When I clicked "Edit Rubric", the second row displaying the outcome changed rating scales on me and replaced the 5, 3, 0 scale with the 4, 3, 2, 1, 0 scale.
This is slightly concerning to me. I understand why this happened...because I chose to "Save" the changes I had made to the descriptions on the "Mastery" tab. But, what if the instructor had wanted to make some kind of change to the original outcome row...and wanted to keep the 5, 3, 0 scale. Those edits still need to be made via the "Outcomes" screen, but it wouldn't currently be able to use the 5, 3, 0 scale since the 4, 3, 2, 1, 0 scale is set as the default. Or, let's say that the instructor instead needed to make an edit to the first row of the rubric (not the row with the outcome). When editing the rubric, the 5, 3, 0 values still change in the outcome row even though the instructor really wants to make an edit to the row above it. This would surprise and confuse me as instructor wondering what happened to the 5, 3, 0 values in the second row and why it was replaced with a completely different scale and descriptions...and potentially changes the overall total points of an assignment. In this case, my first screenshot shows a total of 6 points for the rubric while the second screenshot shows a total of 5 points for the rubric.
