Has anyone used Canvas for faculty grading a student during a lab practical?

[eshay]

I need for faculty to be able to pull up the "quiz" and score the student in real time, rather than a student taking the "quiz". Our students have to demonstrate competency in diagnosing a physical dysfunction (somatic dysfunction) and then perform several osteopathic manipulation techniques for treatment. Has anyone used Canvas and the rubrics for grading in this manner?  

We are experimenting with various platforms/mediums for grading Clinical Skills Assessments - CSA (lab practicals) for our medical students so as to improve efficiency and accuracy in grading (decrease steps in the process in which errors may occur).  Currently we are using rubrics on paper, then entering the data to several individual Excel worksheets for each student, all of which is very tedious for each professor. The subsequent stages of calculating the scores for each subsection of the CSA, the final score, then collecting all of the students’ scores and comments for entry into Canvas is even more time consuming for the course director and administrative assistant.

One of the difficulties we have found is that we need to have a “kill switch” for each section of the CSA.  The diagnosis must score a 70% and each of the treatments must score 80% proficiency in order for each section in order to pass. The “kill switch” is to determine “automatic re-testing” for that portion of the CSA.  Once the student has demonstrated proficiency in a re-test, then a combination of the scores is generated for their final score.

Key components needed:

Professor scoring the student's performance of the CSA online rather than the student completing the "quiz"

Multiple rubrics (sometimes duplicate rubrics, other times different ones)

Drop-down/pick-list for technique names for each rubric

"Automatic Re-testing" if-then logic for rubric scores below a set threshold 

Ability to enter formative assessment comments for each CSA

Ability to recalculate the final scores after a re-testing if necessary

I've attached a sample of part of our rubric, if this helps understanding what I'm asking.

I’ve searched through Canvas and cannot find any references to lab practicals and I’m not sure how else to search.  Is Canvas capable of this type of evaluation?

Thank you!!

osteopathic manipulative medicine

lab 

practical‌

labpractical‌

clinical assessment‌

grading rubrics‌

multiple rubrics‌

if-then logic‌

[kona]

 @eshay  , unfortunately I don’t believe Canvas will do exactly what you need. We do have faculty who grade hand skills and things like that using an assignment with a rubric. They pull up the assignment, go to speedgrader and navigate to the student, and then open up the rubric and fill it out and make comments while watching the student do something. Yet, that doesn’t address your other needs. 😞

Have you figured anything out that might work?

Kona

[thomasm1]

I do something similar in my lab classes.  I was just thinking about how quizes when graded and tied to a test bank tied to a outcome might meet the idea of a kill switch, but then I realized the instructor can't tie a quiz to a student and the student would have to log in and the instructor pretened to be the student.  Seems kind of awkward.  

When I do my assignment assessments of practical skills, I just put the appropriate learning outcomes into the rubric I'm using to assess.  AS I observe I just click on if the students met my expectations or base an outcome rating based on the scores they achieve.  Canvas won't pick the outcome rating automatically in an assignment (but it will in a quiz!) , but the instructor could just click on learning outcome rating that would be appropriate.  And you could endit them to make it just a pass/fail sort of situation or customize for what works. 

I think this is one of those things you try out on a small scale and see what works/doesn't work.  Waiting for one perfect solution often will stand in the way to incremental improvements.

[eshay · Accepted solution]

 @kona ‌ and thomasm1‌. We finally gave up on Canvas and figured out how to create rubrics in ExamSoft for our lab practicals. We still are unable to have a "kill switch" for automatic re-testing issues, but ExamSoft works much better for the remainder of our requirements. We've been using ExamSoft for the written examinations for over six years, so we have a strong database of categories for linking to questions for tracking statistics.

Thank you for both of your comments and insights. They were appreciated.

Eva