Community

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
eleclere
New Member

Access Report not tracking students' engagement with video files

Jump to solution

I'm teaching a course currently in which I want to gather data about student engagement with specific files - for example, lectures that I upload as .mp4 video files. Directions for additional assignments are embedded in these videos to prompt students to watch the videos.

I found it interesting to note that students who are clearly watching these videos (i.e. responding to questions that were posed in the lecture videos) are not tracked as such in Canvas.

For example, here are some anonymized data from a student's Canvas Access Report. According to this report, they have only watched two videos out of seven total lecture videos:

284300_access-report-by-student.png

I've also used the Firefox extension TamperMonkey and run the Access Report Data user script written by  @James ‌. These are data from the same student, which also indicate the student has only clicked on two of seven videos.

284301_access-report-by-student-datafile.png

My question is - am I analyzing the data incorrectly? As far as I can tell, there's no student work-around for NOT viewing the videos and still being able to participate fully in other course assignments, given that I give the instructions for the week in the videos.* According to these reports, students are overwhelmingly not accessing the video files.

A few possibly related details -

  • Many students have told me they access the videos and much of the course content via mobile device. Would this have any impact on the course analytics as they are presented in Canvas / Access Report Data user script?
  • How significant is it that I have organized the course videos as .mp4 attachments instead of embedding them within a third-party app like Vimeo or Canvas's own Media Gallery? Is there an easy fix that is related to course organization?

* I haven't eliminated the possibility that students truly aren't engaging with my videos... which would be disheartening. Given the student feedback I've received and the way I've set up the videos to present relevant course information, it seems unlikely that this is the full picture.

1 Solution

Accepted Solutions
James
Community Champion

 @eleclere ,

Saying that my script gives the same information as the Access Report page doesn't mean anything or add any credibility to the argument. My script downloads the same information that the Access Report page does, so you're really getting two different views of the same information, which is why they agree. The only place they should disagree is the number of views for quizzes. In the web page, Canvas takes one view away for the time they actually took the quiz, while I just report the information returned by Canvas.

The real question is about whether mobile is included in the Access Report. Measuring mobile access has long been an issue and there have been some recent advances. Canvas is just now getting around to counting Mobile access in the Page Views, but they said they hoped to have it in other places soon. Someone asked this question a couple of weeks ago on the page about my script and I referred to a feature request that hadn't been marked as completed, so I am thinking that mobile is still not incorporated everywhere. Since they are putting it in in pieces, I'm not sure that any of us has the definitive word about what is counted and what isn't.

In other words, it very well could be that mobile activity doesn't show up in the Access report -- or that some of it does while parts do not. But, once the mobile activity is included in Canvas, it will be included in my script -- automatically -- since I use the same data they do.

On the other hand, there is nothing to say that students aren't watching the video together or that one isn't writing down the information and sharing it. It might be that a student downloaded it and shared it with everyone. It could be that students are just figuring things out on their own without watching the videos. I thought I had a bunch of "essential" videos that students needed to watch to succeed. Unless it had a due date on it and appeared on the To Do list, I found most students never viewed them.

I would also watch out for just putting an .mp4 into Canvas. A stand-alone .mp4 almost certainly doesn't closed captions, so unless you're using open captioning, you're running into accessibility issues. Using a player instead of a raw .mp4 file allows you to upload captions that can benefit your students and keep the civil rights people happy.

We have recently started using Arc, another Instructure product, that shows us which parts of a video a student has viewed and it also allows students to comment on the video. During a presentation at InstructureCon last week, I saw one school that used a video player that had the whole transcript that was searchable and highlighted the words as they were spoken in the video. There's a lot of analytics possible that are possible, but the only one you have (and then perhaps not for mobile) when you put an .mp4 file into Canvas is whether or not the student accessed it.

View solution in original post

2 Replies
James
Community Champion

 @eleclere ,

Saying that my script gives the same information as the Access Report page doesn't mean anything or add any credibility to the argument. My script downloads the same information that the Access Report page does, so you're really getting two different views of the same information, which is why they agree. The only place they should disagree is the number of views for quizzes. In the web page, Canvas takes one view away for the time they actually took the quiz, while I just report the information returned by Canvas.

The real question is about whether mobile is included in the Access Report. Measuring mobile access has long been an issue and there have been some recent advances. Canvas is just now getting around to counting Mobile access in the Page Views, but they said they hoped to have it in other places soon. Someone asked this question a couple of weeks ago on the page about my script and I referred to a feature request that hadn't been marked as completed, so I am thinking that mobile is still not incorporated everywhere. Since they are putting it in in pieces, I'm not sure that any of us has the definitive word about what is counted and what isn't.

In other words, it very well could be that mobile activity doesn't show up in the Access report -- or that some of it does while parts do not. But, once the mobile activity is included in Canvas, it will be included in my script -- automatically -- since I use the same data they do.

On the other hand, there is nothing to say that students aren't watching the video together or that one isn't writing down the information and sharing it. It might be that a student downloaded it and shared it with everyone. It could be that students are just figuring things out on their own without watching the videos. I thought I had a bunch of "essential" videos that students needed to watch to succeed. Unless it had a due date on it and appeared on the To Do list, I found most students never viewed them.

I would also watch out for just putting an .mp4 into Canvas. A stand-alone .mp4 almost certainly doesn't closed captions, so unless you're using open captioning, you're running into accessibility issues. Using a player instead of a raw .mp4 file allows you to upload captions that can benefit your students and keep the civil rights people happy.

We have recently started using Arc, another Instructure product, that shows us which parts of a video a student has viewed and it also allows students to comment on the video. During a presentation at InstructureCon last week, I saw one school that used a video player that had the whole transcript that was searchable and highlighted the words as they were spoken in the video. There's a lot of analytics possible that are possible, but the only one you have (and then perhaps not for mobile) when you put an .mp4 file into Canvas is whether or not the student accessed it.

Thanks for clearing up these questions, James - this is my first semester using the learning analytics features of Canvas, and it's really intriguing to see how much context your script adds to the picture!

If some of the mobile activity isn't showing up, as you suggest, then I'll have to rethink "viewing lectures" as an appropriate metric for participation. From what I can infer, many of my students are using mobile devices to access course content via the Canvas app. It's an online class taken over the summer semester, so there's not a strong expectation that they'll have meaningful access to personal computers, computer labs, or even the Internet 24/7.

I still think the students are watching the videos, but I haven't considered that they may be watching the videos together (which Canvas analytics can't track) or are otherwise sharing information about the lecture content with each other (less ideal, but not particularly troubling). I'm not interested in capturing any data points other than whether a student watched the video or not, which I thought would be a relatively easy measurement to capture using Canvas analytics.

I'd be interested in learning about how Canvas measures mobile activity more generally - I'll look for the thread you referenced about the mobile feature request! Thanks for bringing this to my attention.