cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Community Member

Best practices in setting up Outcomes

Hi Canvas Community,

Our institution is just getting started in Canvas, migrating over from Moodle.

We are looking to implement Canvas Outcomes in order to gather the Mastery data associated with it. As we do this we see that there could be Institution, Program, and Course Outcomes that could potentially be measured by linking Outcomes and Rubrics and attaching that rubric to an Assessment. What suggestions do you have for a new institution setting up Outcomes? Do we focus on Program outcomes in order to draw the most robust data on that program and student OR should we go ahead and start with Course Level Outcomes and gather data that way? Any suggestions would be greatly appreciated. We want to be sure we gather the most appropriate/useful Mastery information.

Thanks!

11 Replies
Highlighted
Surveyor

Hello,

I know this post dates back to 2015. However, our university is trying to dive into using Outcomes. We are trying to determine best practices as well as finding someone that has retrieved data from Canvas Outcomes.

0 Kudos
Highlighted

Hi Bryan, 

You provided a very helpful write up on how your institution set up your Outcome structure. 

I know this is an older post, but here's a question: Does this Outcome group structure mirror your Institution's overall Sub-account structure?

Thanks,

Ben

Highlighted
Surveyor

My institution is new to Canvas, and I'm trying to dig into Outcomes as it shows remarkable potential. We've set up a large enrollment course to pilot outcomes, but I have run into one question. Our outcomes are built using a 0, 1, 2, 3 rating system, with mastery at 2. However, the rubrics are used for grading, and we needed to have it be a 5-point value for each of the outcomes. What I've noticed is that the value of 5 is not only used for grading that particular assignment but even passes along into the Learning Mastery grade book. Now, students who received 5 in the rubric are showing 5 in the Learning Mastery for that outcome, which could drastically distort their achievements. Has anyone else reported this? What is the best way to handle outcomes that are used to score assignments? Is there a better way to be handling this process?

Highlighted
Surveyor

This is very helpful - thank you! And I may need to go to InstructureCon. Smiley Happy

A question - how do you do program-level reporting on Outcomes? I know how to find the Learning Mastery report for one course, but we are interested in looking at outcomes across our entire curriculum.

Thanks,

Emily

Highlighted

Thanks a million!!

Highlighted
Community Coach
Community Coach

Deactivated user​ (Thanks for chiming in bhauf@aihm.org​)

If you want more information about outcomes, here is a compilation of all the sessions at InstructureCon 2015 that are related to outcomes and rubrics:

Verifying Student Learning with Outcomes

A Collaboration in Measuring Student Competencies and Outcomes in Canvas

Woodstock via the LMS: Canvas for American Studies

Herding Fish: Encouraging Faculty to Use Subaccount-Generated Outcomes and Rubrics

Orchestrating Instructional Design at the System Level

Also, if you (or anyone else reading this thread is not at InstructureCon this year, no fear... every session is being recording and will be available after the conference for viewing).

Hope that helps!

Smiley Happy

Highlighted

Oh perfect! Thank you! This is so helpful. I will certainly watch the recording and appreciate you getting me some items in "writing" as well Smiley Happy I am hoping to make it to CanvasCon next year as our Institution will be about a year into using Canvas.

Highlighted
Community Advocate
Community Advocate

Deactivated user​ Here is the write-up. I left out using questions banks for scoring outcomes as that will take longer to explain. I will attempt to add screenshots for some of the descriptions below as I am able to.

Overview

Our institution recently realigned program competencies and outcomes to assess student performance across the curriculum. Here is brief description of the process:

One of the schools at our University was searching for a better solution to assess competency through various assessment methods, such as quizzes, papers, oral presentations and examinations. At the course level, faculty outline competencies on their syllabi and map them to assessments that are delivered throughout the semester, enabling students to see how competencies are introduced and reinforced throughout the curriculum.  The school was looking for consistency in how students were assessed across programs, so program level competencies and outcomes were introduced and used for aligning with assessment items. This approach made it easy for us to collect data on competencies and outcomes that were used across the program to see how students are progressing.

The Setup

We started at the highest account level in Canvas for creating outcomes. At the time, this would make it easier to update outcomes and import them into sub-accounts and courses so that all the data rolled up to the highest level.

Structure

We structured, or grouped, the Outcomes as follows:

  • School > Program > Concentration > Competency > Outcomes
  • Bold indicates a Group/Folder created within the Outcomes Tool
  • These Outcomes were then imported into the School’s sub-account in Canvas so that they could be assessed by faculty enrolled in courses organized in this sub-account.

The structure was created for faculty to easily navigate to and select outcomes to import into their courses. We used the competency and outcome alignment matrix developed by the school to define this structure.Group/Folder structure is also important for the Student Learning Mastery gradebook. Depending how you organized your Outcomes in the course level outcomes tool will determine how outcomes are organized in this view. For example, we imported the Concentration folder, competencies folder, and outcomes for each competency so that they were grouped by competency in the student learning mastery view. This enables you to be able to see how many outcomes a student is mastering within a competency.

Nomenclature

Again, the nomenclature was created based upon the structure. Each outcome was named as follows:

  • School.Program.Concentration.Competency#.Outcome#
  • Example: SPH.MPH.EOHS.C1.LO1
  • The reason for this nomenclature is due to the Outcomes report tool in Canvas. The report tool does not include the group structure, so we had to create a name that allowed us to see where this particular outcome resides.

Rubric Scale

For easy implementation, our school agreed upon a 5 point scale with easily identifiable descriptions for every learning outcome. This varied slightly from the default rubric in Canvas (5,3,1) as we saw the need to allow faculty to score students who fell between outstanding, acceptable, and unacceptable. Since learning outcomes in a particular program are assessed across multiple courses, it was important to implement a method that was both easy to read and use for both students AND faculty. We the used the API to update outcomes with the rubric after they were created.

There are also some calculation methods to consider. Best score, highest score, decaying average. See here: What calculation methods are available for Outcomes?

Be sure to decide on a calculation method as this will impact the learning mastery gradebook and cannot be adjusted after an outcome is scored.

Alignment

Since outcomes were created at the account level, they need to be imported into the course. Things to note:

  • Make sure your outcomes are grouped so that they are organized in the student learning mastery view.
  • Faculty cannot delete outcomes from their course once they are imported (admins can), but they can delete outcomes that are in groups. We decided to import outcomes into the course for the faculty so that the folder structure was correct and persistent through all courses.
  • If you import any outcomes that will not be assessed, delete those from the course since they will display as "not met" in the learning mastery areas. We import into the course-level based on concentration and then delete those that are not needed so that the folder structure is retained.
  • Outcomes are aligned using the rubric tool that can be attached to assignments, quizzes, and discussion boards. Outcomes can be imported onto the rubric to either grade the student or not impact the grade at all. We chose the latter so that outcome scoring was separate from traditional student grades (A, B, C, D, etc).
  • Scoring outcomes occurs in speedgrader.

I hope this is helpful. I'll try to clean this up since it is essentially an information dump from my head.

Community Advocate
Community Advocate

Deactivated user​, I am doing a presentation at InstructureCon15 on this very topic. We (UNT Health Science Center) have been using Outcomes for a little over a year now. The session should be recorded and made available after the conference. I will attempt to write our best practices and experiences; it is just a copious amount of information that may take a bit to put together.

Highlighted

Thank you!

0 Kudos