[ARCHIVED] Measuring success: course design

[akenady]

We are into our second term of using Canvas, and have established as our goals consistent use of modules, the syllabus, and the grade book. Without paying for additional reports, and lacking the IT/Canvas admin staffing to create our own, is there another, perhaps indirect way, to measure these goals? For now I'm doing informal spot checks, which is fine, but I'd love it if I could roll out some numbers to demonstrate an improvement cycle.

Thanks in advance!

[kmeeusen]

You make this quite a challenging question,  @akenady  !

"Without paying for additional reports"

and

"lacking the IT/Canvas admin staffing to create our own"

Within those parameters, I am afraid I am at a loss as to how to help you. Out of pure desperation and a hope that any workable answer you receive might also be helpful to me, I am going to share your question with both the Canvas Admins‌ and Canvas Developers‌ focus groups, just because there are some very creative folks there.

Kelley

[a_craik]

Hi Ann

I imagine this is the golden goose for a lot of us, and honestly, even if you could put some code together the reporting would lack necessary interpretation.

So for example, with a bit of coding it wouldn't be too hard (for a coder) to run API calls that cycle through a set of courses and capture information about the modules in each (see Modules - Canvas LMS REST API Documentation). But that doesn't give you the 'real' picture (and note the technical limitations in the API doc - more calls are really required). What if the course also has Files and Pages visible, and this isn't consistent with your desired standard? Or what if the modules expected are there, but the nature of the content within is lacking? This is where only humans can truly interpret and measure quality. Not only that - we can recognise deviations that aren't 'wrong', but are in fact innovations, and we can learn from them.

To really measure success I honestly believe a course needs to be interpreted holistically by a knowledgeable human. I therefore believe the route to be able to measure success efficiently requires making more knowledgeable humans to be able to self or peer audit courses. This requires not just a change in knowledge, but also a change in culture.

And I'm afraid that is just as challenging to achieve as coding for non-coders is. But I do think it's the right challenge

[akenady]

Adam:

I love this answer! Thank you for confirming the difficulty of trying to do this with our current constraints, and for affirming the need for ongoing, meaningful review and revision cycles. It's on our radar, but we've a ways to go yet.

Being the "knowledgeable person" who cares most about measuring our goal achievement, I guess it's on me to track it for this first year.