Having used Arc (now Studio) pretty extensively last year, I was hoping that with the new name and icon, we might see improvements in the functionality of the Insights area. Alas, such is not the case. Here are my complaints (and also my suggestions for improvement):
1. Students appear in the Insights area apparently in the order that they first accessed the first available video in that course. There is no provision to reorder them, sort them by groups, etc., and judging from others' comments, if they drop the course, they still appear in Insights.
2. In my experience, the viewing data collected is frequently inaccurate. I've had students report that they watched a video in its entirety despite Insights reporting they watched either only a portion or none at all. In some cases, I'm sure I was being lied to, but in others, students whom I trust implicitly also claimed to have watched the videos, and looking at their Canvas usage during the time period in question seems to have backed them up.
3. There is no easy way to grade students on having watched a given video. I already have more quizzes than I like in my class, and I don't want to embed another one for each video I post. So, my solution has bee to grade students on how many minutes of a video they watched divided by the total time, but that approach is problematic given the inaccuracy of the recorded data, which seems to be affected by what browser was used, whether the video was viewed on a mobile device, etc. There is also no provision for the viewing data to flow to the Canvas Gradebook - I have to open the course gradebook in one window and Insights in the other, then manually enter a score for each student, after having to hunt through the disordered names to find the data I need.
4. If a student goes back and watches a video after I've scanned the data and entered a grade (which I allow), there is no means for me to be automatically notified that is has happened. The only way I know is if a student lets me know, or else I go back and periodically review all the data for each video, which again is a very laborious process, given the random order of the students' names and the inability to tell by looking at their icons that there is new viewing data available.
Studio would be a lot more useful if the problems I described are fixed. Right now, if I could avoid using Studio at all, I would, but I haven't found any other viable option to deliver my self-produced videos.
As a footnote, I've discussed these problems with Canvas techs in the past, but so far, nothing seems to have changed. This shouldn't be a votable item, but rather a necessary bug fix.
Yes, viewer drop rate is always a concern. The quiz option is a useful approach but this is just one strategy.
Perhaps there could be an assignment type 'View Video' which shows complete/ incomplete based upon a set percentage of the video played and viewed by the student (e.g. 80%). Then again, in that scenario a student could press play for a video and subsequently step away to do something else.
Ironically, I was just now fielding an insights question as a direct link is not available under the embedded video for instructors if the in-video commenting feature is disabled.
You insights add tremendous value and if there are any 'gotchas' such as with the mobile viewers I believe the info should be summarized and added as notes to the page -
But I think it is odd that the mobile browser is the pain point and not the app. I would have assumed it to be the other way around.