New vs. Old Analytics

Instructure
Instructure
10 26 4,079

Hello Canvas Community!

We will continue to iterate on improvements to course-level analytics. In the meantime, we've created a guide of the differences. Please let me know in the comments if we've missed anything and I'll be sure to add it to the list!

 

  New Analytics Old Analytics
Aggregate Period Weekly—per student, per section, per course* Daily—per student and course
Hourly Aggregate No* Yes—per student
Category Aggregates No* Yes
Account Aggregates No* Yes—across multiple courses in a sub-account
Data Display Table view
In development: Submission Date column in Student View data
Graphic Illustration view
Data Comparisons Course, Section, Student N/A
Data Customization Filter by assignments, quizzes, and graded discussions and enable shape representations N/A
Activity View
  • View specific students who viewed, didn't view, participate, or didn't participate
  • View specific students with missing or late assignments
  • Message students regarding status
  • View number of students with page views or participations
  • View number of students with missing or late assignments
Assignment Grade Statistics Mean score for the class, sections, and students
In development: median, high, and low scores per assignment grade
Median, high, and low scores per assignment grade
Rollup Resource Data View number of resources viewed with list of resources View number of resources viewed
Rollup for Student Assignment Status Status per assignment
In development: number of missing, late, on time per student
Missing, late, on time
Mobile Page View Data Included N/A
Student Messaging in Score Ranges Yes—via group message sent to students individually No
Student Messaging for Missing/Late Assignments Yes—via group message sent to students individually No
CSV Data Download Yes No

 

*improves overall performance; this functionality will continue to be improved

 

 

Best, Katrina

 

 

Update 1-30-2020: New analytics provides a better representation on user activity than the old analytics. We removed extraneous requests to better align the data with student behavior.

26 Comments
Explorer III

Thank you for putting this together. I notice that it is primarily listing things from the instructor perspective. I feel like I've also read about some of the student view differences/additions, that may not be captured here, but to be honest the specifics of those posts aren't coming to mind.

Looking forward to this being kept up to date as additional development occurs. Thank you again!

Community Advocate
Community Advocate

Hi khess@instructure.com, thank you for posting this, it is very informative.

I am currently putting together a gap analysis between the two versions of analytics and if I find anything else I will post here. My initial thoughts on new analytics are that it provides greatly improved, meaningful stats and functionality and that the various CSV report downloads will be super beneficial to teaching staff.

Do you have any timelines around when the developments in italics above will be released? And/or any indication of when New Analytics will be fully refined? Could I also ask that you please consider the following ideas/questions raised please?

https://community.canvaslms.com/ideas/15553-new-analytics-filter-identifydifferentiate-sections-stud... 
https://community.canvaslms.com/ideas/15501-submission-date-missing-in-new-analytics  
New Analytics, Student View Submissions feature 

Eagerly awaiting the further iterations/updates of New Analytics and again, thank you for sharing!
m.mccooey@qub.ac.uk 

Surveyor

Thanks for this! Very useful! 

I wonder if there are differences in counting page views / participation between new analytics and old analytics. It seems that the pageview in the old analytics is not the same as the new one, for the same course, same period of time and same student. The two numbers are highlighly correlated but they are really different. The old one is always higher than the new one too. 

Instructure
Instructure

Hi vanzandt, we include differences/additions with the student view in the list above but please let us know if we missed something!

Best, Katrina

Instructure
Instructure

Hi m.mccooey@qub.ac.uk‌, please do post anything we may have missed here. 

Do you have any timelines around when the developments in italics above will be released?

  • Submission Date column in Student View data -- releasing soon
  • Median, high, and low scores per assignment grade -- releasing in 2020
  • Number of missing, late, on time per student -- releasing in 2020

And/or any indication of when New Analytics will be fully refined? 

Our plan is to iterate and refine New Analytics on regular basis. We will add new features based on requests and usage. 

Could I also ask that you please consider the following ideas/questions raised please?

  • New Analytics Filter - Identify/Differentiate Sections, Students, Assignments -- request received

  • Submission date missing in new analytics -- releasing soon 
  • New Analytics, Student View Submissions feature -- we are not replacing the submissions visualization but will be adding the Submission date column which will be releasing soon

Hope this helps! 

Best, Katrina

Community Advocate
Community Advocate

Fantastic khess@instructure.com ! Thank you so much for the update, so helpful Smiley Happy

Community Member

Hi Katrina, I am not sure this one is on the list. It is the VIEW SUBMISSIONS graph found on this website.( How do I view analytics for a student in a course? )

It is the most useful tool I know for quickly assessing how a student is performing in class. I can quickly see number of late assignments and more importantly HOW LATE these assignments were. I can also see what assignments are missing and quickly count them up (count all red squares). I can assess a students performance in about 3 seconds on this graph. I NEED THIS BACK!.

Surveyor

I second this. This graph has been really helpful and it seems like a big lost function with New Analytics.

Learner II

I am also a major user of this feature and very concerned about its absence from New Analytics. I regularly use this view during parent-teacher and student-teacher conferences to talk about student habits around late versus on-time submissions, and it's the clearest way to get a visual of who is doing steady work versus working in bursts every few weeks. 

Instructure
Instructure

Hi everyone, thanks for your feedback. We added a Submitted column in the Student View. You can sort the column headers. To see the Missing assignments, click on Status.

336653_Screen Shot 2020-01-30 at 5.13.35 AM.png

Instructure
Instructure

Hi dennis.foung.work@gmail.com‌, new analytics provides a better representation on user activity than the old analytics. We removed extraneous requests to better align the data with student behavior.

Community Member

Hi Katrina, This is a step in the right direction, but not nearly as much information or as quickly reviewed as the graphic we are looking for. (View submissions graph in this website). I still have to do the math on how late each assignment was (submission date minus due date) and then try to determine if the late submissions were were clumped together during a stressful life situation or throughout the semester. I have about 75 assignments in the class so doing all this takes time. I teach two classes with 400 students in each class another factor in why I am so passionate about streamlining good information on my students.

I am not trying to be difficult here but why not just put in the old graphic in the new analytics as an option?

Thanks. _David

Learner II

Here's what I'm missing with New Analytics that I value with the current system. I can make a quick distinction between a student who "works in bursts" like this:

analytics screenshot with many yellow tails all on the same day with green dots interspersed

versus a student who is just sort of gradually falling behind the pace and dawdling along like this:

analytics screenshot with gradually more and longer yellow tails as a student falls behind

versus a student who is just always, for whatever reason, turning everything in about one day late and has been all term: 

analytics screenshot with interspersed greens and short-tailed yellows

Those all lead to different conversations about work habits, and the visual is really useful for those conversations.

Instructure
Instructure

Thanks thompsli@hsd.k12.or.us‌, your examples are helpful! I will take this back to the team to discuss. We see the value of this visualization.

Instructure
Instructure

Hi schmi071@umn.edu, thank you for sharing details of your workflow! The old graphic isn't connected to the new data source so it's not as straightforward as putting the old graphic back in. I will share your feedback with the team as we continue our exploration on improving this chart. 

Learner II

I thought of one more example of something that was easy to spot with this visualization, but hard to spot in most other cases: The person who does the "burst work" thing, but well ahead of the deadlines. Example here:

analytics screenshot with many green tails all on the same day with green dots interspersed

This isn't always a problem, but can be if the quality of work or retention of ideas suffers. (The ones without tails for this particular student were my "check-in" assignments with very narrow open to/from dates - all actual math tended to be done in bursts and then the student would have a few weeks off while they waited for me to open up the next batch of work.)

In this particular student's case, this worked out just fine, but if a student comes in with lower grades on exams as a result of a strategy like this on online assignments then it's something I'd bring up in a "study habits" conversation. (My classes are not self-paced so the exams happen on a schedule to do with the term.) I wouldn't be able to see this pattern at all if I only looked at late versus not late assignments, so the tails were very useful here.

Community Advocate
Community Advocate

What about outcomes? Any chance we can view this inside of the new analytics? 

Instructure
Instructure

Thanks thompsli@hsd.k12.or.us‌, will add to your examples!

Explorer III

Hi there, has there been any requests so far to add 'type of resource' to the 'resources' csv download? I can see resource name which is good, but would also like to see type as well in the report. I think it could be a really valuable report for learning designers. If I had a course that included loads of content pages for example and a few ungraded discussions, and I could see that those ungraded discussions were high in views and participations, perhaps I would include more discussions in future.

Kirsten

Surveyor II

Hello, our account rep mentioned that page views and participation metrics  include an aggregate across all devices, including Canvas mobile apps.  So, we will be able to get a count of which tools students accessed using specific devices (desktop, laptop, and mobile)?

Thanks,

Nancy Webb

Surveyor II

David Schmidt hit the nail on the head. I found the graph most useful in 'old analytics'. 

I find 'new analytics' utterly useless.

Community Member

Yes, PLEASE take this back to the development team! I agree 100% with Linnea Thompson. This kind of visualization is incredibly useful!  I was shocked to learn it was dropped in the New Analytics.

Community Member

It would be very helpful if we could find in New Analytics:

 

1.date of each submission is exceptionally important; personally I don't need the Due date in New Analytics, I already know it since I posted the Assignment

2.how many hours each student spent in CANVAS each day

3.how many hours each student participated in the online conferences each day

4.last day of activity in CANVAS, either attending online Conferences, Reading documents or posting assignments; our college expels students after a certain period of nonattendance; this date is critical both for students and for teachers since we would like to keep them enrolled

5.the number of words per each submission

If these functions are already in place, please advise where would be the best place to get instructions

 

Thank you for your efforts to improve CANVAS

Like  0Reply
Adventurer

I am concerned with the aggregate period being weekly vs. more frequently so that may stall our adoption.   If we are to use an analytics tool for the purpose of instructors seeing what is happening then a weekly update is simply too long.  This is probably long past observed and debated.  

If this is not a big deal, would someone please explain?  

Surveyor

I do NOT like New Analytics. It does NOT show the info I need to track student activity. Can we have the choice to use "old" analytics?

Surveyor

Hello,

I'm wondering if there has been any consideration for giving instructors the ability to apply analytics to daily activity. In the current academic environment, being able to determine what students have done on daily basis is an essential tool. The weekly activity report is just not very useful at all.