Outcomes and Accreditation: The Good, the Bad, and the Ugly

[LEASUSAN]

Tap/click map to enlarge

Mario Guerra​

Ryan Myers​

The University of Texas at Austin - HigherEd

UT Austin is embarking on a large-scale assessment project to determine student mastery for accreditation in the History Department. We are using this pilot to create a program of assessment to be used across the university. We’ll share pilot data, best practices, and let you know how it really went.

[rcb]

It seems from this session that there are several schools out there looking into how Canvas can help support assessment of General Education outcomes, especially for accreditation purposes.  We should find a way to get together and share ideas and resources.  I'm not sure how to get that going but if anyone is interested I am

Ron Bramhall

University of Oregon

rcb@uoregon.edu

[rcb]

There is an Accreditation community group on Canvas that might serve as a place to get together:

Accreditation

I'd also be interested in trying to pull together a conversation at InstructureCon this evening or tomorrow if anyone is interested.

[kyoung]

I'm in. We are working on that at TCC.

Kristina Young, Interim Honors Program Coordinator

and

Professor

Written Communications/Humanities

F1-20

6501 South 19th Street

Tacoma, WA 98466

Phone: 253.566.5330

Email: kyoung@tacomacc.edu<mailto:kyoung@tacomacc.edu>

[garth]

I too am interested in collaborating on  how we can leverage Canvas outcomes for accreditation reporting.

Let's see if we can get some momentum in the user group Accreditation​

I am all about the "open" message that has been a theme here at InstructureCon 2016​, and am very happy to share strategies that I have been working on.

[ltscal2]

Great presentation. Confirmed every single action that we did up until implementation. Deal breaker was data entry

When the ability for large enrollment management is available we are a go. Until then, would love to see how other schools would deal with the scaling up to large courses. It IS a bit deal for us.

[acatters]

I completely agree - but the assessment need to be more granular, whereas we can drill down to the question level...not just assignment.

Anna Catterson

Emporia State University

acatters@emporia.edu

[rcb]

Anna

Agreed.  I understand there is an ability to connect outcomes to quiz questions but I do not have experience with that and can't envision right now how it works.  Anyone doing that level of outcome assessment?

[kyoung]

You would create a rubric for the quiz, then tag questions in the rubric. It is not elegant, and requires some tweaking. I'm on a phone, so I can't show you. Part of the tweaking is whether the outcomes are manually created or imported from the institutional folder.

Sent from my iPhone

[ltscal2]

You cant really do that cleanly. But certain accreditation require question level data. In our testing what we ended up doing is, creating a no submission assignment and attaching the rubric to it in order to record that data for each and every question that needed to be recorded. Its super messy and I wonder if anyone actually implemented it....

[kyoung]

You would need to have dual assessments. Every question would have its own criteria which would be the outcome.

Sent from my iPhone

[garth]

Canvas will allow you to associate one or more outcomes with a question bank.

People have asked to be able to associate an outcome with a single question, but consider this:

• How much time would it take to associate one outcome with one question until all questions had been associated with the outcome?  Are you sure you have only one single question that is associated with one single outcome?

I think it is more likely that you will have multiple questions associated with an outcome.

Thus you create a question bank to organize those related questions, and associate the outcome with that question bank.

A benefit of this is you are able to randomize the questions on the quizzes, and still measure the outcome.

You can build a quiz from multiple question banks, allowing you to measure multiple outcomes in a single quiz.

Food for thought.

[kdemello]

We align course and division outcomes with both rubrics for assignments as well as question banks. The outcomes themselves are not editable for instructors, but they are free to align them with whatever assignment they believe measures that outcome. They can also include those divisional outcomes to a grading rubric, but uncheck the box to use it for scoring. This way there is still a grading rubric, but also the aligned outcome(s). We have pressed all faculty to create course-level outcomes in addition to the divisional ones, so they are accustomed to how that all works on rubrics and question banks.

I love that Canvas handles all the math for us, and the profs can get an easy visual of where the students are in the learning mastery grade book.

The trick with question banks is for the instructors to add the questions to the banks with aligned outcomes BEFORE adding those questions to the quizzes. There is a feature request out there now to add folders to question banks<https://community.canvaslms.com/ideas/5800>, so that it would be easier to manage all questions aligned with an outcome, sorted by the quiz that the question is measured on, for ease of import into quizzes.

Kate

[roush_94]

Keroleen,

Do you mean data entry from the instructor side? Having to use SpeedGrader and a Rubric versus using the grade book or being able to import data from a spreadsheet? I definitely see that as an issue, I just want to make sure I'm not missing some other pitfall .

We are piloting the Outcomes tool this fall on a relatively small scale...we need the data for accreditation, so we'll have to monitor faculty use of the tool to make sure it is being used which I'm not looking forward to at all.

[kelly1]

Why hasn't the video of this session been posted yet? It was one of the best for those of us interested in using Canvas to track outcomes...