Reports for Funding Audits from Canvas

[jbrown10]

My company is based in Australia and we have very strict reporting requirements when it comes to providing evidence of participation for students who receive state government funding. 

Canvas Reports and Analytics don't provide us with enough information. I'm looking for suggestions of how other canvas users use any third party tools or LTI's to gather greater insight into student activity within a course. 

My CSM did provide me with a report ('all version student submission report') that canvas can provide at a cost however it was not 100% what was needed. 

Our compliance team are very hesitant to trust Canvas is compliant in the Aus RTO space because the reports on evidence of participation are not robust enough to simply export a single report and provide this to an Auditor. 

Any suggestions of other third party analytics tools or even use of the Canvas API to pull a robust report on individual activity within canvas would be great.

Cheers

[jsavage2 · Accepted solution]

Hi Joel,

Do you have access to the Canvas Data Portal? If you have needs for specific reports that are beyond what’s normally available, you can get data down to individual page views as either CSV extracts or Red Shift data. That can be used to feed whatever your preferred reporting/BI tools are that you use to report on your other systems.

For those of us outside Australia, though: what are the specific reporting requirements. There is an abundance of information available through the API, but it’s difficult to recommend specific actions without knowing what your auditor will accept as adequate evidence of participation.

[jsavage2 · Accepted solution]

Hi Joel,

Do you have access to the Canvas Data Portal? If you have needs for specific reports that are beyond what’s normally available, you can get data down to individual page views as either CSV extracts or Red Shift data. That can be used to feed whatever your preferred reporting/BI tools are that you use to report on your other systems.

For those of us outside Australia, though: what are the specific reporting requirements. There is an abundance of information available through the API, but it’s difficult to recommend specific actions without knowing what your auditor will accept as adequate evidence of participation.

[jbrown10]

Hi Jay,

Thanks for the mention of Canvas Data, ill talk to my CSM and get access. It's likely what we need will be a report pulled from the API which is then combined with our records data from VetTrak (our student management system). 

It's a little difficult to explain the reporting requirements because our requirements are written based upon a traditional face to face & hand written or manually submitted assessments. In the event that 'records' are digitally entered, the requirements reference a teacher inputting these into the student management system, not an LMS. An LMS does not usually retain a 'competency' log which is which is what our SMS will have attached to each enrolment. 

After further exploration, I believe I can get around this using Canvas Data ensuring I have one report per student which includes:
Student ID, Assessment Name, Assessment ID, Submission Date, Outcome, Grader Name, Grade Date

This will be for all assessments that a student submits.

This will then be matched against an existing assessment matrix which details assessments per module, then how each modules map to each unit of competency. Ie, Assessments 1 2 and 3 are required for Module 1. Module 1 Starts 2 units of competency but only finishes one, the second is not completed until all work in module 1 and 2 is completed.

There is a push internally to have ALL of this information generated directly from Canvas (because our old LMS was both an SMS and LMS and could produce this) however I simply do not see how that can ever be done when Canvas is simply the delivery platform for assessments, it does not have any of the Competency Mapping interfaces that relate to the quality framework for the training package that the course is built upon.

if any of that needs explaining, this explanation of training packages on the ASQA site might help
https://www.asqa.gov.au/about/australias-vet-sector/training-packages

I had previously at another company explored ways of using rubrics or mastery paths to build competency mapping but neither I or the other canvas admin before me were ever able to successfully build the mapping. I also dont what kind of reports this would generate.

[adamwarecs · Accepted solution]

Feel free to bring discussions of this nature over to the Vocational Ed & Training user group.