New vs. Old Analytics

Katrina-Hess
Instructure Alumni
Instructure Alumni
38
12663

Hello Canvas Community!

We will continue to iterate on improvements to course-level analytics. In the meantime, we've created a guide of the differences. Please let me know in the comments if we've missed anything and I'll be sure to add it to the list!

General Analytics

  New Analytics Old Analytics
Aggregate Period Weekly—per student, per section, per course* Daily—per student and course
Hourly Aggregate No* Yes—per student
Category Aggregates No* Yes
Account Aggregates No* Yes—across multiple courses in a sub-account
Data Display Table view
In development: Submission Date column in Student View data
Graphic Illustration view
Data Comparisons Course, Section, Student -
Data Customization Filter by assignments, quizzes, and graded discussions and enable shape representations -
Activity View
  • View specific students who viewed, didn't view, participate, or didn't participate
  • View specific students with missing or late assignments
  • Message students regarding status
  • View number of students with page views or participations
  • View number of students with missing or late assignments
Assignment Grade Statistics Mean score for the class, sections, and students
In development: median, high, and low scores per assignment grade
Median, high, and low scores per assignment grade
Rollup Resource Data View number of resources viewed with list of resources View number of resources viewed
Rollup for Student Assignment Status Status per assignment
In development: number of missing, late, on time per student
Missing, late, on time
Mobile Page View Data Included -
Student Messaging in Score Ranges Yes—via group message sent to students individually -
Student Messaging for Missing/Late Assignments Yes—via group message sent to students individually -
CSV Data Download Yes (all except Attendance) -

 

*improves overall performance; this functionality will continue to be improved

 

Attendance Report

  New Analytics Old Analytics
Attendance Report Yes -
Data Comparison Course, Section, Student -
Data Customization Course Access, Posts, Assignments, Collaborations, Conferences, Pages, Quizzes -
Aggregate Period Weekly or Daily
Account Aggregates Course only -
CSV Data Download No
Student Messaging No

 


Old Analytics Documentation

Existing analytics documentation is still available in the following guides:

Student Guide

How do I view my course analytics as a student?

Instructor Guide

How do I view Course Analytics?

How do I view analytics for a student in a course?

38 Comments
vanzandt
Community Champion

Thank you for putting this together. I notice that it is primarily listing things from the instructor perspective. I feel like I've also read about some of the student view differences/additions, that may not be captured here, but to be honest the specifics of those posts aren't coming to mind.

Looking forward to this being kept up to date as additional development occurs. Thank you again!

Maeve_McCooey
Community Contributor

Hi  @Katrina-Hess , thank you for posting this, it is very informative.

I am currently putting together a gap analysis between the two versions of analytics and if I find anything else I will post here. My initial thoughts on new analytics are that it provides greatly improved, meaningful stats and functionality and that the various CSV report downloads will be super beneficial to teaching staff.

Do you have any timelines around when the developments in italics above will be released? And/or any indication of when New Analytics will be fully refined? Could I also ask that you please consider the following ideas/questions raised please?

https://community.canvaslms.com/ideas/15553-new-analytics-filter-identifydifferentiate-sections-stud... 
https://community.canvaslms.com/ideas/15501-submission-date-missing-in-new-analytics  
New Analytics, Student View Submissions feature 

Eagerly awaiting the further iterations/updates of New Analytics and again, thank you for sharing!
 @Maeve_McCooey  

dennis_foung_wo
Community Member

Thanks for this! Very useful! 

I wonder if there are differences in counting page views / participation between new analytics and old analytics. It seems that the pageview in the old analytics is not the same as the new one, for the same course, same period of time and same student. The two numbers are highlighly correlated but they are really different. The old one is always higher than the new one too. 

Katrina-Hess
Instructure Alumni
Instructure Alumni

Hi vanzandt, we include differences/additions with the student view in the list above but please let us know if we missed something!

Best, Katrina

Katrina-Hess
Instructure Alumni
Instructure Alumni

Hi  @Maeve_McCooey ‌, please do post anything we may have missed here. 

Do you have any timelines around when the developments in italics above will be released?

  • Submission Date column in Student View data -- releasing soon
  • Median, high, and low scores per assignment grade -- releasing in 2020
  • Number of missing, late, on time per student -- releasing in 2020

And/or any indication of when New Analytics will be fully refined? 

Our plan is to iterate and refine New Analytics on regular basis. We will add new features based on requests and usage. 

Could I also ask that you please consider the following ideas/questions raised please?

  • New Analytics Filter - Identify/Differentiate Sections, Students, Assignments -- request received

  • Submission date missing in new analytics -- releasing soon 
  • New Analytics, Student View Submissions feature -- we are not replacing the submissions visualization but will be adding the Submission date column which will be releasing soon

Hope this helps! 

Best, Katrina

Maeve_McCooey
Community Contributor

Fantastic  @Katrina-Hess  ! Thank you so much for the update, so helpful Smiley Happy

schmi071
Community Novice

Hi Katrina, I am not sure this one is on the list. It is the VIEW SUBMISSIONS graph found on this website.( How do I view analytics for a student in a course? )

It is the most useful tool I know for quickly assessing how a student is performing in class. I can quickly see number of late assignments and more importantly HOW LATE these assignments were. I can also see what assignments are missing and quickly count them up (count all red squares). I can assess a students performance in about 3 seconds on this graph. I NEED THIS BACK!.

ataylor30
Community Contributor

I second this. This graph has been really helpful and it seems like a big lost function with New Analytics.

thompsli
Community Champion

I am also a major user of this feature and very concerned about its absence from New Analytics. I regularly use this view during parent-teacher and student-teacher conferences to talk about student habits around late versus on-time submissions, and it's the clearest way to get a visual of who is doing steady work versus working in bursts every few weeks. 

Katrina-Hess
Instructure Alumni
Instructure Alumni

Hi everyone, thanks for your feedback. We added a Submitted column in the Student View. You can sort the column headers. To see the Missing assignments, click on Status.

336653_Screen Shot 2020-01-30 at 5.13.35 AM.png

Katrina-Hess
Instructure Alumni
Instructure Alumni

Hi  @dennis_foung_wo ‌, new analytics provides a better representation on user activity than the old analytics. We removed extraneous requests to better align the data with student behavior.

schmi071
Community Novice

Hi Katrina, This is a step in the right direction, but not nearly as much information or as quickly reviewed as the graphic we are looking for. (View submissions graph in this website). I still have to do the math on how late each assignment was (submission date minus due date) and then try to determine if the late submissions were were clumped together during a stressful life situation or throughout the semester. I have about 75 assignments in the class so doing all this takes time. I teach two classes with 400 students in each class another factor in why I am so passionate about streamlining good information on my students.

I am not trying to be difficult here but why not just put in the old graphic in the new analytics as an option?

Thanks. _David

thompsli
Community Champion

Here's what I'm missing with New Analytics that I value with the current system. I can make a quick distinction between a student who "works in bursts" like this:

analytics screenshot with many yellow tails all on the same day with green dots interspersed

versus a student who is just sort of gradually falling behind the pace and dawdling along like this:

analytics screenshot with gradually more and longer yellow tails as a student falls behind

versus a student who is just always, for whatever reason, turning everything in about one day late and has been all term: 

analytics screenshot with interspersed greens and short-tailed yellows

Those all lead to different conversations about work habits, and the visual is really useful for those conversations.

Katrina-Hess
Instructure Alumni
Instructure Alumni

Thanks  @thompsli ‌, your examples are helpful! I will take this back to the team to discuss. We see the value of this visualization.

Katrina-Hess
Instructure Alumni
Instructure Alumni

Hi  @schmi071 , thank you for sharing details of your workflow! The old graphic isn't connected to the new data source so it's not as straightforward as putting the old graphic back in. I will share your feedback with the team as we continue our exploration on improving this chart. 

thompsli
Community Champion

I thought of one more example of something that was easy to spot with this visualization, but hard to spot in most other cases: The person who does the "burst work" thing, but well ahead of the deadlines. Example here:

analytics screenshot with many green tails all on the same day with green dots interspersed

This isn't always a problem, but can be if the quality of work or retention of ideas suffers. (The ones without tails for this particular student were my "check-in" assignments with very narrow open to/from dates - all actual math tended to be done in bursts and then the student would have a few weeks off while they waited for me to open up the next batch of work.)

In this particular student's case, this worked out just fine, but if a student comes in with lower grades on exams as a result of a strategy like this on online assignments then it's something I'd bring up in a "study habits" conversation. (My classes are not self-paced so the exams happen on a schedule to do with the term.) I wouldn't be able to see this pattern at all if I only looked at late versus not late assignments, so the tails were very useful here.

BradMoser
Community Contributor

What about outcomes? Any chance we can view this inside of the new analytics? 

Katrina-Hess
Instructure Alumni
Instructure Alumni

Thanks  @thompsli ‌, will add to your examples!

kirsten_ryall
Community Participant

Hi there, has there been any requests so far to add 'type of resource' to the 'resources' csv download? I can see resource name which is good, but would also like to see type as well in the report. I think it could be a really valuable report for learning designers. If I had a course that included loads of content pages for example and a few ungraded discussions, and I could see that those ungraded discussions were high in views and participations, perhaps I would include more discussions in future.

Kirsten

nancy_webb
Community Participant

Hello, our account rep mentioned that page views and participation metrics  include an aggregate across all devices, including Canvas mobile apps.  So, we will be able to get a count of which tools students accessed using specific devices (desktop, laptop, and mobile)?

Thanks,

Nancy Webb

griggsda
Community Explorer

David Schmidt hit the nail on the head. I found the graph most useful in 'old analytics'. 

I find 'new analytics' utterly useless.

ah2274
Community Novice

Yes, PLEASE take this back to the development team! I agree 100% with Linnea Thompson. This kind of visualization is incredibly useful!  I was shocked to learn it was dropped in the New Analytics.

anamraicu
Community Novice

It would be very helpful if we could find in New Analytics:

 

1.date of each submission is exceptionally important; personally I don't need the Due date in New Analytics, I already know it since I posted the Assignment

2.how many hours each student spent in CANVAS each day

3.how many hours each student participated in the online conferences each day

4.last day of activity in CANVAS, either attending online Conferences, Reading documents or posting assignments; our college expels students after a certain period of nonattendance; this date is critical both for students and for teachers since we would like to keep them enrolled

5.the number of words per each submission

If these functions are already in place, please advise where would be the best place to get instructions

 

Thank you for your efforts to improve CANVAS

Like  0Reply
Jeff_F
Community Champion

I am concerned with the aggregate period being weekly vs. more frequently so that may stall our adoption.   If we are to use an analytics tool for the purpose of instructors seeing what is happening then a weekly update is simply too long.  This is probably long past observed and debated.  

If this is not a big deal, would someone please explain?  

kathleen_gonoud
Community Member

I do NOT like New Analytics. It does NOT show the info I need to track student activity. Can we have the choice to use "old" analytics?

LloydH
Community Member

Hello,

I'm wondering if there has been any consideration for giving instructors the ability to apply analytics to daily activity. In the current academic environment, being able to determine what students have done on daily basis is an essential tool. The weekly activity report is just not very useful at all.

AshleyRobinson
Community Member

@LloydH I agree. We are in desperate need of the daily attendance feature rather than weekly. @Katrina-Hess Is there a way to turn off the new analytics? We are between a rock and a hard place when we don't have daily attendance listed. I know I can get it in the reports but it is a pretty cumbersome process for our instructors. 

ysmalls
Community Champion

At K-12 it is very important for us to be able to run daily attendance reports.  @LloydH we had to pay for a custom which loads data daily and allows the teacher to run daily attendance reports.  There is an Attendance Report on the Course navigation for them.  They can run the report by All Students, Course or Specific Student and they can select the date range.  We have had a few snags since this is new but this has been great for our teachers.  The report has a column with Attended on it with that has Yes or blank!

 

DanBurgess
Community Participant

@AshleyRobinson, for our folks that still need to access the data as presented in the old analytics, we have been using the workaround of appending "/analytics" after the course number in the URL:

your_canvas_domain/courses/course _number/analytics

Not sure how long this will continue to work. 

 

arthur_maturo_2
Community Participant

Weekly aggregate but not daily? TOTALLY screws up any chance of recording asynchronous attendance. I used the old analytics to check and record asynchronous attendance. It was integral! Now how do we check DAILY attendance on Canvas? How do I know WHICH DAY a student comes and does something? I don't care about weekly! Weekly does NOT help at all.  

Congratulations CANVAS you've done it again: made this LMS worse than it was before!

Stef_retired
Instructure Alumni
Instructure Alumni

@arthur_maturo_2  We hope you'll review the new Attendance in New Analytics feature and post your feedback and questions to  Releases Q&A: 2021-08-21 New Analytics Attendance.

DL1919___
Community Explorer

It would be great to see this at course level or sub-account level, I understand you are working on it. One thing we notice is that it doesn't show any level of participation data when it comes to active learning, so other LTIs or MS LTIs recently added. It is, for us, still quite a basic form of data gathering. Any plans to include third party tools integrations as an option? 

LoganGaz
Community Member

Hello @Stef_retired  and @Katrina-Hess, I am trying to make a case for our institution as to whether or not we turn on the New Analytics. 

is the top graph still the most up to date comparison between the standard and new analytics? The date is really out of date but I'm not sure if you update this when new features are added? 

thanks , Gaz

 

carlycurran
Instructure
Instructure

Hi, @LoganGaz

Yes, the chart at the top of this page is up to date. We've added to it as we've made updates to New Analytics 🙂 

erinhmcmillan
Instructure Alumni
Instructure Alumni

@LoganGaz 

Yes, this document is up to date. It doesn't show the last updated date, but it was 09-17-2021.

Thanks,

Erin

LoganGaz
Community Member

Thanks @carlycurran and @erinhmcmillan 

alissaarcline
Community Participant

Hi! I might just be looking in the wrong place, but I am wondering if it's possible to see how many times a particular file in a course has been viewed/downloaded (and if so, to get a report of which students have done so)?

cgaudreau
Community Contributor

One of the barriers to our full-scale adoption of New Analytics is the exporting of data via API. Leadership is very excited about the Attendance Report, but without the API connection, it isn't making any movement on campus.

I've looked for updates related to this feature roll out, but couldn't locate any. Are there any insights I'm missing elsewhere?