Include Canvas Mobile App in Student Access Reports

[kent_ratajeski]

Apparently, mobile access does not show up on the Individual student access reports at the present time.  A Canvas support agent told me that this was because the mobile app uses access token rather than web sessions.  This means that the student access reports are not showing the entire story when it comes to student access of course material.  PLEASE FIX THIS!  This is the only way I will know exactly whether or not individual students have accessed course lectures, assignments, and other material.  

[KristinL]

At the account level, more mobile data is available (https://community.canvaslms.com/ideas/2005-include-mobile-app-page-views-in-page-views-on-user-accou... and, specifically, Erin's response... ) I think student activity from a mobile device should be visible at the course level for teachers as well. I'm going to share this idea with the https://community.canvaslms.com/groups/cmug?sr=search&searchId=3473ac7b-e91e-454a-bb12-623d6c23ad06&...‌ Group to try to attract some more votes and conversations.

[nlatimer]

Yes, I definitely agree that teachers need to be able to view this information at the course level as well. I work in a high school, and this has come up a number of times for us. Kids will say they have been accessing a course, but it doesn't appear that they have, because they were using their phones. While it's helpful for the Admin to be able to look up access information that includes an IOS or Android device, it would be great if the teacher could see this for themselves. It would also help avoid confusion when the teacher is not aware that the information they are seeing does not reflect mobile device access to their course.

[dherstad]

It would be very useful if we could access the logs generated by the Canvas app.  It's always been frustrating when attempting to help with app-related issues since these logs are unavailable to us.  Thanks for your consideration!

[rseilham]

I agree with this idea. The information should be available at the teacher level, not just the account/admin level. 

[Nancy_Webb_CCSF]

It also appears that module Requirements are not marked complete when they are executed in a mobile app.  This can prevent a student from moving to the next module (or item) even though the required work was done.  We had a case where the only student who didn't seem to have met the requirement had actually done it in the iOS app (thanks to looking at page views), which forced the teacher to remove the requirement so she could proceed.

Have any of you had experience with apps and requirements?

[rseilham]

Hi  @Nancy_Webb_CCSF ‌, 

Your issue is not related to this idea as this idea is focusing on Teachers viewing mobile activity in Student Reports, but I still wanted to address your concern:

I have not heard of any issues with Requirements on the iOS app. I just tested the Requirements for "Mark as Done" on Pages and it's working just like the web (see below). Can you describe the kind of requirements you have in your module? Thanks.

[Nancy_Webb_CCSF]

Thanks so much  @rseilham , for testing the App. Interesting to see that requirements worked for you. I may have leapt to the wrong conclusion that the problem stemmed from same source as Access Reports not updating from mobile Apps.

The requirements were to view two assignments before progressing to the next item in the module; submitting not required.  When the teacher checked module progress (not the access report) for the student, it indicated that the two items were not completed and the next item was not unlocked for her.  The student reported she couldn't move to the next item in the module and that there was no indication she'd viewed them. So I looked her up in page views.

The assignments were using an external tool (McGraw Hill Connect) that is configured to send grades back to Canvas. Although module progress indicated that the items weren't complete, my Admin view of Page views showed she had viewed the two assignments; at least the assignment number appears in the long API link in page views.  Am I misinterpreting this api link (school info left off)?
api/v1/courses/13841/assignments/83012/submissions/70825.js  
13841 is the course number,  83012 is the assignment, 70825 is the student ID.  There are other api URL's with the assignment number.  But perhaps due to the way the App works, the assignment was not actually visited and viewed?  

Would this one display the 83012 assignment?  /api/v1/courses/13841/assignments/83012?per_page=50

I thought that if the Access reports were not correct, that module progress might not be correct too, and the requirement not marked complete.  But maybe that's not true. Maybe I misinterpreted the urls .

Nancy

[rseilham]

Hi again! 

The requirements were to view two assignments before progressing to the next item in the module; 

Yes, I just tested this out with a standard and LTI (external tool) assignment, and it worked as it would on the web. 

The assignments were using an external tool (McGraw Hill Connect) that is configured to send grades back to Canvas.

I'm not familiar with McGraw Hill Connect, but it's possible this could be the issue. The mobile app might not be working correctly with the external tool, which wouldn't surprise me. 

api/v1/courses/13841/assignments/83012/submissions/70825.js
13841 is the course number, 83012 is the assignment, 70825 is the student ID. There are other apiURL's with the assignment number. But perhaps due to the way the App works, the assignment was not actually visited and viewed?  

This looks right, but just like I said before, this could be an LTI issue. 

Sorry I don't have any answers for you, but if it's related to an LTI, I would suggest you put in a support ticket with Canvas support. 

[j_c_turner]

hi, great idea, getting a lot of pain from staff here caused by the lack of clarity over the missing detail in analytics. We need this soon as. This seems to be an idea that has been talked about across the community for a long time, but this time, i have a feeling its going to happen.

[Nancy_Webb_CCSF]

Yes instructors need to monitor student progress, and many students are using mobile devices instead of computers, I'm surprised by how many.  Students could be dropped in error and other problems could arise due to incorrect access records.  It is very important that access records include mobile app activity.

[kburns6]

From an admin/ instructional design side - it would also be helpful for analytics on instructors as well in the Canvas Teacher app. Which courses are instructors spending the most time in, how long are their average sessions — overall instructor use, not just which canvas course have been published would be helpful to find out our Canvas all-star users. 

Currently, installing Google Analytics within your school’s canvas account only tracks user’s use on the Canvas website https://yourschool.instructure.com As far as mobile users the only helpful information this method provides is how many/which users access the website on their device versus using the mobile app.

[rseilham]

 @kburns6 ‌: This is all true. We get some good usage data out of Canvas Data (although it has quite a bit of noise), but that's not a practical solution for everyone. I've had some discussions with the Mobile Product Manager about this and he definitely agrees this needs to get better, and it's a group effort from Instructure. I get the idea that we'll see some movement this year (but don't quote me on that). 

[davids]

Quite a few of our courses award points for participation and penalize for not viewing required items, and our teachers use the Access Report to make those determinations of who viewed (or didn't view) what. It was quite disturbing to learn today, after students got penalized and complained that their view hadn't counted, that it isn't as reliable as we thought (& were told) it was. What good is an Access Report if it doesn't include the most common type of views?

As an admin, I'm also able to view IP address and browser/OS type, but only in the daily tracking log; I have the same issue in the Access Reports, which is where I went to investigate and validate the complaints from students.

A few years ago I could have understood inconsistencies in how mobile devices reported, but this should be normative now. The majority of our students do the bulk of their work on mobile devices, and that's a trend I'm seeing and hearing from colleagues everywhere. This is well-documented in the New Media Consortium's Horizon Report for Higher Ed for 2017 which identifies mobile learning as one of the top 2 developments in educational technology for higher education occurring within the year (i.e., now) - see pages 40-41. So in my mind I don't even see the lack of this ability as a "feature request," but rather as a bug or problem in existing applications which don't report properly and thus need to be fixed.

[Adrienne_J_Gaut]

A middle step so instructors are aware (they are not) is to have "mobile app data not included" ANYWHERE it's not included --- the Analytics views, the Access Report, etc.

We've had a few judicial affairs cases wrapped around this discrepancy -- the instructors have incomplete information from their view. That's not cool for them or for the students being accused of no-access or access, etc. 

It's also a burden on admins to then dig into Page Views for the students on the User Access Detail page...scrolling..scrolling...scrolling to find the rows/dates needed and then copy/pasting into something else (the csv only downloads a limited set of data).

[Nancy_Webb_CCSF]

That's a great idea,  @Adrienne_J_Gaut ‌.  Canvas, can that be easily implemented?

[davids]

I agree with  @Nancy_Webb_CCSF ‌ and really appreciate this idea.  At our institution, we've had several incidents where students lost points over it and complained (either to the instructor, the IT admins, or to deans or department heads), so it became a judicial matter involving admins, too.

It's still something I feel like Canvas needs to fix ASAP (as I explain elsewhere in this thread), but I realize implementing that could take a good 6 months to a year.  If a "mobile app data not included" statement could be included wherever tracking data is reported, that would save alot of grief and unnecessary confusion in the meantime.

[rseilham]

I periodically have discussions with the Mobile PM and lead engineers, and they are well aware that this is something that is important to their users. In fact, it was a priority at last year's Project Khaki. It's awesome to see the great ideas on here and throughout the community as I hope, just like everyone else here, that there is something available in the future. 

[Nancy_Webb_CCSF]

I hope I'm not jumping the gun, but it seems that with the new student mobile apps 6.3, iOS and Android, mobile page views are now visible in course Analytics and Access reports.  See the release notes and Cody's comment at the bottom here: https://community.canvaslms.com/docs/DOC-15152-canvas-student-release-notes-android-63?commentID=122... 

We'll need to tell students to download the latest version.  Fingers crossed this solves the course reports problem.

[rseilham]

Hi  @Nancy_Webb_CCSF ‌,

That is correct, it's now available, and students will need the 6.3 or above for this to be visible to admins in the access reports. What happens is when the application is closed (just switching out of the app, not force quitting it), all the page views are sent up to Canvas, so it's not "real-time" but pretty close. 

[ProductPanda]

Comments from Instructure

As part of the new Ideas & Themes process, all ideas in Idea Conversations were reviewed by the Product Team. Any Idea that was associated with an identified theme was moved to the new Idea & Themes space. Any Idea that was not part of the move is being marked as Archived. This will preserve the history of the conversations while also letting Community members know that Instructure will not explore the request at this time.