New vs. Old Analytics

[Katrina-Hess]

Hello Canvas Community!

We will continue to iterate on improvements to course-level analytics. In the meantime, we've created a guide of the differences. Please let me know in the comments if we've missed anything and I'll be sure to add it to the list!

General Analytics

	New Analytics	Old Analytics
Aggregate Period	Weekly—per student, per section, per course*	Daily—per student and course
Hourly Aggregate	No*	Yes—per student
Category Aggregates	No*	Yes
Account Aggregates	No*	Yes—across multiple courses in a sub-account
Data Display	Table view In development: Submission Date column in Student View data	Graphic Illustration view
Data Comparisons	Course, Section, Student	-
Data Customization	Filter by assignments, quizzes, and graded discussions and enable shape representations	-
Activity View	View specific students who viewed, didn't view, participate, or didn't participate View specific students with missing or late assignments Message students regarding status	View number of students with page views or participations View number of students with missing or late assignments
Assignment Grade Statistics	Mean score for the class, sections, and students In development: median, high, and low scores per assignment grade	Median, high, and low scores per assignment grade
Rollup Resource Data	View number of resources viewed with list of resources	View number of resources viewed
Rollup for Student Assignment Status	Status per assignment In development: number of missing, late, on time per student	Missing, late, on time
Mobile Page View Data	Included	-
Student Messaging in Score Ranges	Yes—via group message sent to students individually	-
Student Messaging for Missing/Late Assignments	Yes—via group message sent to students individually	-
CSV Data Download	Yes (all except Attendance)	-
*improves overall performance; this functionality will continue to be improved

Attendance Report

	New Analytics	Old Analytics
Attendance Report	Yes	-
Data Comparison	Course, Section, Student	-
Data Customization	Course Access, Posts, Assignments, Collaborations, Conferences, Pages, Quizzes	-
Aggregate Period	Weekly or Daily	-
Account Aggregates	Course only	-
CSV Data Download	No	-
Student Messaging	No	-

---

Old Analytics Documentation

Existing analytics documentation is still available in the following guides:

Student Guide

How do I view my course analytics as a student?

Instructor Guide

How do I view Course Analytics?

How do I view analytics for a student in a course?

[vanzandt]

Thank you for putting this together. I notice that it is primarily listing things from the instructor perspective. I feel like I've also read about some of the student view differences/additions, that may not be captured here, but to be honest the specifics of those posts aren't coming to mind.

Looking forward to this being kept up to date as additional development occurs. Thank you again!

[Maeve_McCooey]

Hi  @Katrina-Hess , thank you for posting this, it is very informative.

I am currently putting together a gap analysis between the two versions of analytics and if I find anything else I will post here. My initial thoughts on new analytics are that it provides greatly improved, meaningful stats and functionality and that the various CSV report downloads will be super beneficial to teaching staff.

Do you have any timelines around when the developments in italics above will be released? And/or any indication of when New Analytics will be fully refined? Could I also ask that you please consider the following ideas/questions raised please?

https://community.canvaslms.com/ideas/15553-new-analytics-filter-identifydifferentiate-sections-stud... 
https://community.canvaslms.com/ideas/15501-submission-date-missing-in-new-analytics  
New Analytics, Student View Submissions feature 

Eagerly awaiting the further iterations/updates of New Analytics and again, thank you for sharing!
 @Maeve_McCooey  

[dennis_foung_wo]

Thanks for this! Very useful! 

I wonder if there are differences in counting page views / participation between new analytics and old analytics. It seems that the pageview in the old analytics is not the same as the new one, for the same course, same period of time and same student. The two numbers are highlighly correlated but they are really different. The old one is always higher than the new one too. 

[Katrina-Hess]

Hi vanzandt, we include differences/additions with the student view in the list above but please let us know if we missed something!

Best, Katrina

[Katrina-Hess]

Hi  @Maeve_McCooey ‌, please do post anything we may have missed here. 

Do you have any timelines around when the developments in italics above will be released?

• Submission Date column in Student View data -- releasing soon
• Median, high, and low scores per assignment grade -- releasing in 2020
• Number of missing, late, on time per student -- releasing in 2020

And/or any indication of when New Analytics will be fully refined? 

Our plan is to iterate and refine New Analytics on regular basis. We will add new features based on requests and usage. 

Could I also ask that you please consider the following ideas/questions raised please?

• New Analytics Filter - Identify/Differentiate Sections, Students, Assignments -- request received

• Submission date missing in new analytics -- releasing soon 

• New Analytics, Student View Submissions feature -- we are not replacing the submissions visualization but will be adding the Submission date column which will be releasing soon

Hope this helps! 

Best, Katrina

[Maeve_McCooey]

Fantastic  @Katrina-Hess  ! Thank you so much for the update, so helpful

[schmi071]

Hi Katrina, I am not sure this one is on the list. It is the VIEW SUBMISSIONS graph found on this website.( How do I view analytics for a student in a course? )

It is the most useful tool I know for quickly assessing how a student is performing in class. I can quickly see number of late assignments and more importantly HOW LATE these assignments were. I can also see what assignments are missing and quickly count them up (count all red squares). I can assess a students performance in about 3 seconds on this graph. I NEED THIS BACK!.

[ataylor30]

I second this. This graph has been really helpful and it seems like a big lost function with New Analytics.

[thompsli]

I am also a major user of this feature and very concerned about its absence from New Analytics. I regularly use this view during parent-teacher and student-teacher conferences to talk about student habits around late versus on-time submissions, and it's the clearest way to get a visual of who is doing steady work versus working in bursts every few weeks. 

[Katrina-Hess]

Hi everyone, thanks for your feedback. We added a Submitted column in the Student View. You can sort the column headers. To see the Missing assignments, click on Status.

[Katrina-Hess]

Hi  @dennis_foung_wo ‌, new analytics provides a better representation on user activity than the old analytics. We removed extraneous requests to better align the data with student behavior.

[schmi071]

Hi Katrina, This is a step in the right direction, but not nearly as much information or as quickly reviewed as the graphic we are looking for. (View submissions graph in this website). I still have to do the math on how late each assignment was (submission date minus due date) and then try to determine if the late submissions were were clumped together during a stressful life situation or throughout the semester. I have about 75 assignments in the class so doing all this takes time. I teach two classes with 400 students in each class another factor in why I am so passionate about streamlining good information on my students.

I am not trying to be difficult here but why not just put in the old graphic in the new analytics as an option?

Thanks. _David

[thompsli]

Here's what I'm missing with New Analytics that I value with the current system. I can make a quick distinction between a student who "works in bursts" like this:

versus a student who is just sort of gradually falling behind the pace and dawdling along like this:

versus a student who is just always, for whatever reason, turning everything in about one day late and has been all term: 

Those all lead to different conversations about work habits, and the visual is really useful for those conversations.

[Katrina-Hess]

Thanks  @thompsli ‌, your examples are helpful! I will take this back to the team to discuss. We see the value of this visualization.

[Katrina-Hess]

Hi  @schmi071 , thank you for sharing details of your workflow! The old graphic isn't connected to the new data source so it's not as straightforward as putting the old graphic back in. I will share your feedback with the team as we continue our exploration on improving this chart. 

[thompsli]

I thought of one more example of something that was easy to spot with this visualization, but hard to spot in most other cases: The person who does the "burst work" thing, but well ahead of the deadlines. Example here:

This isn't always a problem, but can be if the quality of work or retention of ideas suffers. (The ones without tails for this particular student were my "check-in" assignments with very narrow open to/from dates - all actual math tended to be done in bursts and then the student would have a few weeks off while they waited for me to open up the next batch of work.)

In this particular student's case, this worked out just fine, but if a student comes in with lower grades on exams as a result of a strategy like this on online assignments then it's something I'd bring up in a "study habits" conversation. (My classes are not self-paced so the exams happen on a schedule to do with the term.) I wouldn't be able to see this pattern at all if I only looked at late versus not late assignments, so the tails were very useful here.

[BradMoser]

What about outcomes? Any chance we can view this inside of the new analytics? 

[Katrina-Hess]

Thanks  @thompsli ‌, will add to your examples!

[kirsten_ryall]

Hi there, has there been any requests so far to add 'type of resource' to the 'resources' csv download? I can see resource name which is good, but would also like to see type as well in the report. I think it could be a really valuable report for learning designers. If I had a course that included loads of content pages for example and a few ungraded discussions, and I could see that those ungraded discussions were high in views and participations, perhaps I would include more discussions in future.

Kirsten

[nancy_webb]

Hello, our account rep mentioned that page views and participation metrics  include an aggregate across all devices, including Canvas mobile apps.  So, we will be able to get a count of which tools students accessed using specific devices (desktop, laptop, and mobile)?

Thanks,

Nancy Webb

[griggsda]

David Schmidt hit the nail on the head. I found the graph most useful in 'old analytics'. 

I find 'new analytics' utterly useless.

[ah2274]

Yes, PLEASE take this back to the development team! I agree 100% with Linnea Thompson. This kind of visualization is incredibly useful!  I was shocked to learn it was dropped in the New Analytics.

[anamraicu]

It would be very helpful if we could find in New Analytics:

1.date of each submission is exceptionally important; personally I don't need the Due date in New Analytics, I already know it since I posted the Assignment

2.how many hours each student spent in CANVAS each day

3.how many hours each student participated in the online conferences each day

4.last day of activity in CANVAS, either attending online Conferences, Reading documents or posting assignments; our college expels students after a certain period of nonattendance; this date is critical both for students and for teachers since we would like to keep them enrolled

5.the number of words per each submission

If these functions are already in place, please advise where would be the best place to get instructions

Thank you for your efforts to improve CANVAS

Like • Show 0 Likes0Reply

 Actions 

[Jeff_F]

I am concerned with the aggregate period being weekly vs. more frequently so that may stall our adoption.   If we are to use an analytics tool for the purpose of instructors seeing what is happening then a weekly update is simply too long.  This is probably long past observed and debated.  

If this is not a big deal, would someone please explain?  

[kathleen_gonoud]

I do NOT like New Analytics. It does NOT show the info I need to track student activity. Can we have the choice to use "old" analytics?

[LloydH]

Hello,

I'm wondering if there has been any consideration for giving instructors the ability to apply analytics to daily activity. In the current academic environment, being able to determine what students have done on daily basis is an essential tool. The weekly activity report is just not very useful at all.

[AshleyRobinson]

@LloydH I agree. We are in desperate need of the daily attendance feature rather than weekly. @Katrina-Hess Is there a way to turn off the new analytics? We are between a rock and a hard place when we don't have daily attendance listed. I know I can get it in the reports but it is a pretty cumbersome process for our instructors. 

[ysmalls]

At K-12 it is very important for us to be able to run daily attendance reports.  @LloydH we had to pay for a custom which loads data daily and allows the teacher to run daily attendance reports.  There is an Attendance Report on the Course navigation for them.  They can run the report by All Students, Course or Specific Student and they can select the date range.  We have had a few snags since this is new but this has been great for our teachers.  The report has a column with Attended on it with that has Yes or blank!

[DanBurgess]

@AshleyRobinson, for our folks that still need to access the data as presented in the old analytics, we have been using the workaround of appending "/analytics" after the course number in the URL:

your_canvas_domain/courses/course _number/analytics

Not sure how long this will continue to work. 

[arthur_maturo_2]

Weekly aggregate but not daily? TOTALLY screws up any chance of recording asynchronous attendance. I used the old analytics to check and record asynchronous attendance. It was integral! Now how do we check DAILY attendance on Canvas? How do I know WHICH DAY a student comes and does something? I don't care about weekly! Weekly does NOT help at all.  

Congratulations CANVAS you've done it again: made this LMS worse than it was before!

[Stef_retired]

@arthur_maturo_2  We hope you'll review the new Attendance in New Analytics feature and post your feedback and questions to  Releases Q&A: 2021-08-21 New Analytics Attendance.

[DL1919___]

It would be great to see this at course level or sub-account level, I understand you are working on it. One thing we notice is that it doesn't show any level of participation data when it comes to active learning, so other LTIs or MS LTIs recently added. It is, for us, still quite a basic form of data gathering. Any plans to include third party tools integrations as an option? 

[LoganGaz]

Hello @Stef_retired  and @Katrina-Hess, I am trying to make a case for our institution as to whether or not we turn on the New Analytics. 

is the top graph still the most up to date comparison between the standard and new analytics? The date is really out of date but I'm not sure if you update this when new features are added? 

thanks , Gaz

[carlycurran]

Hi, @LoganGaz! 

Yes, the chart at the top of this page is up to date. We've added to it as we've made updates to New Analytics 🙂 

[erinhmcmillan]

@LoganGaz 

Yes, this document is up to date. It doesn't show the last updated date, but it was 09-17-2021.

Thanks,

Erin

[LoganGaz]

Thanks @carlycurran and @erinhmcmillan 

[alissaarcline]

Hi! I might just be looking in the wrong place, but I am wondering if it's possible to see how many times a particular file in a course has been viewed/downloaded (and if so, to get a report of which students have done so)?

[cgaudreau]

One of the barriers to our full-scale adoption of New Analytics is the exporting of data via API. Leadership is very excited about the Attendance Report, but without the API connection, it isn't making any movement on campus.

I've looked for updates related to this feature roll out, but couldn't locate any. Are there any insights I'm missing elsewhere?