[ARCHIVED] Institutional Assessment for Accreditation Using Outcomes and Rubrics?

[phineas]

Are there other institutions who are actively trying to implement institutional assessment using outcomes or are interested in doing so? 

At the University of Wyoming we are working on a plan to assess student learning on a department/college/institution wide level using outcomes and rubrics. This process is part of ongoing accreditation reviews where the reviewers need to see specific learning artifacts which correspond with specific learning outcomes as examples.

One specific question is that we need to download specific learning artifacts associated with an outcome.  Say, all the papers submitted to the english department or across the institution which are connected to the outcome for critical thinking. You can already see a list in Canvas of "outcome artifacts" associated with an outcome with links to the student submissions but we need an easy way to download them all…Then we need a mechanism whereby reviewers can access and assess these outcome artifacts (ideally without individual student information).

It seems like the learning mastery grade book, outcomes and rubrics have been designed with this all in mind but there is currently no road map as to how to accomplish this in practice. The canvas guides speak to these capabilities:

"Grading student work automatically collects and compiles data on student progress for the Outcomes. The data is available for reporting to support teaching improvement, identify at-risk students, and support the accreditation process. This unified, streamlined approach dramatically reduces the amount of work required to implement Learning Outcomes through the intelligent reuse of assessment workflows in the grading process."

[hollands]

I have had experience with both, though most of it was for our school of Ed. My former institution swapped away from using Taskstream to all Canvas and as far as I know it went fairly smooth. There of course was a learning curve mostly because Tasksteam would automatically crunch numbers and in Canvas it required something to interpret the .csv

Essentially (and you'll have to forgive my brevity because I was not directly involved with this project), we had 3 people as account admins creating all of the outcomes. One of which was the accreditation coordinator, who interacted with the faculty to help setup rubrics for assignments. They also were able to go into courses to pull examples/samples as needed.

They were able to run the reports as needed, though for a while we would have to run it for them. Then it was a matter of graphically representing the data.

[dpc3]

Shaun would any of the "3 people as account admins creating all of the outcomes" be willing to describe the process they used. A white paper or How To, or share templates that were created to report out?  Thanks

[CSoran]

David - At Tacoma Community College, we have 3 people as account admins create all of the outcomes.  We did a presentation on it during the 2015 Instructurecon and you should be able to find the recording.  What we do is get exports from Cirricunet, where all of the college's outcomes are centrally tracked.  We massage the formatting in the Excel files a bit and then copy/paste them into Canvas.  We have all of the college's outcomes setup at the account level.  We have 3 main outcome groups, with many sub-groups - Courses, Degree Learning, and Programs.  Any instructor can navigate any of those groupings to get the specific outcomes they want. 

The one thing we still need more work on is versioning.  We are currently just keeping everything updated, but have considered naming outcomes v1, v2, etc or something to signify that it is an old version.

[randerson2]

I should let you know that depending on how immediate you need this that I was stopped after the presentation by an employee of Canvas (I have to get his name again from our CSM and will post it when I do so) that assured me that all of the issues I discussed as being problems in collecting the data that I want and the broken parts of Canvas involving test banks was going to be fixed.   He is new to Canvas and this was one of the duties he was specifically given. I have a meeting with our CSM Wednesday and I am going to ask to set up a meeting with him to see what kind of timeline they have in mind for fixing this.

Here is what I am asking them to do:

1. 1) Have each Outcome be a specific URL so that the links to the outcome won’t break even if you move a course that has a rubric tied to that outcome outside of the subaccount in which it was originally connected.    This is in response to our school’s decision to reorganize departments, which in turn made our SIS create new subaccounts where new courses were being housed.  Since the outcomes were built in the legacy subaccount, when we copied the courses with those outcomes we lost them.   I then moved the legacy subaccount one step higher in the hierarchy then re-copied the courses and the outcomes stuck.   The problem was that our SIS then moved the legacy subaccount back to where it had been. Everything still worked fine for several semesters, but this past spring the outcomes tied to rubrics in the fall courses did not copy over correctly to the spring.   The outcome tag stayed, but Canvas moved the outcomes from subaccount outcomes to course outcomes.   So I am having to go course by course to pull the data and then merge it myself between all the courses in that department.
2. 2) What Canvas needs to provide is an expanded querying capacity that allows you to choose what data you want pulled. Right now you can just pull for a single term and you get all the outcomes from that subaccount from that term (whether you want them all or not).   What needs to be built is a dashboard that allows users to select a single or multiple terms, a single, multiple, or all outcomes and a way to pull outcomes from a particular list of courses instead of simply getting every outcome from every course as a default.
3. 3) As you asked for, the system needs to be fixed to allow for outcomes to be pulled when question banks aligned with outcomes are taken by the same student more than once.  I also think that you should be able to align outcomes by specific questions in case you have no desire to use question banks and sets.

[ljimenezron]

Robert, did your request on the following elements answered? if so,would you share your insights?

1. Have each Outcome be a specific URL so that the links to the outcome won’t break even if you move a course that has a rubric tied to that outcome outside of the subaccount in which it was originally connected.    This is in response to our school’s decision to reorganize departments, which in turn made our SIS create new subaccounts where new courses were being housed.  Since the outcomes were built in the legacy subaccount, when we copied the courses with those outcomes we lost them.   I then moved the legacy subaccount one step higher in the hierarchy then re-copied the courses and the outcomes stuck.   The problem was that our SIS then moved the legacy subaccount back to where it had been. Everything still worked fine for several semesters, but this past spring the outcomes tied to rubrics in the fall courses did not copy over correctly to the spring.   The outcome tag stayed, but Canvas moved the outcomes from subaccount outcomes to course outcomes.   So I am having to go course by course to pull the data and then merge it myself between all the courses in that department.
2. 2) What Canvas needs to provide is an expanded querying capacity that allows you to choose what data you want pulled. Right now you can just pull for a single term and you get all the outcomes from that subaccount from that term (whether you want them all or not).   What needs to be built is a dashboard that allows users to select a single or multiple terms, a single, multiple, or all outcomes and a way to pull outcomes from a particular list of courses instead of simply getting every outcome from every course as a default.
3. 3) As you asked for, the system needs to be fixed to allow for outcomes to be pulled when question banks aligned with outcomes are taken by the same student more than once.  I also think that you should be able to align outcomes by specific questions in case you have no desire to use question banks and sets.

[Renee_Carney]

We have a new focus group that you may be interested in joining!  Canvas Focus Group: Accreditation

Come join the Conversation: How Can Canvas Improve your Accreditation Experience?

[kenneth_rogers]

Hi there, phineas!

Given that this question is from over two years ago, and that there have been many updates to Canvas during that time, I am going to mark this as "Assumed Answered". We'd love to hear from you and have your thoughts on some of the resources shared!