Skip navigation
All Places > User Group: New Analytics > Blog

In 2019 we introduced New Analytics to replace and improve upon the course- and user-analytics offering in Canvas. We did so because we saw an opportunity to make an update to a valuable product to provide expanded and more accurate information. Since that original Analytics offering was first written, Canvas has grown and the difference between Canvas as it was and Canvas as it is now has led to disparities between what was presented in that tool and what we would expect to see.

 

All this to say, we know that there are discrepancies between the two. We tried very hard to understand and reconcile as many of those differences as possible, but in some cases we had to make the conscious choice to provide more accurate and relevant data. The first analytics offering was written five years ago, and in the intervening time Canvas has had a long path of changes - from the big to the small-and-nuanced, leading to subtle differences between Canvas’ internals and analytics’ understanding and presentation. This means subtle changes between the two in terms of the interpretation of grade statuses as well as activity. In the case of activity data, this also means slicing and dicing that data in new and different ways (in particular breaking it down to per-week numbers).

 

Our guiding star with this new work was always to do our best to accurately reflect the reality provided in Canvas.

 

We'll take this opportunity to try to describe our interpretation of activity information in particular in hopes of clarifying what is being presented now.

 

The activity information surfaced in New Analytics is derived from request logs from Canvas itself. How a user's action is presented and counted depends on the route/controller used, the status code, and the HTTP verb on that request. For example, loading the URL for an assignment will count as a Page View under the title of that assignment. Below is a table of the URLs that we count and consider in New Analytics and how they are presented (the P column is whether or not it's considered participation)

 

NamePMethod and Path

<Assignment Title>

GET /courses/:course_id/assignments/:id(.:format)

<Assignment Title>POST /api/v1/courses/:course_id/assignments/:assignment_id/submissions(.:format)
<Assignment Title>POST /courses/:course_id/assignments/:assignment_id/submissions(.:format)
<Assignment Title>POST /courses/:course_id/submissions(.:format)
<Attachment Title>GET /courses/:course_id/files/?preview=:file_id
<Attachment Title>GET /courses/:course_id/files/:file_id/download(.:format)
<Attachment Title>GET /courses/:course_id/files/:file_id/download.:type(.:format)
<Attachment Title>GET /courses/:course_id/files/:file_id/preview(.:format)
<Attachment Title>GET /courses/:course_id/files/:id(.:format)
<Attachment Title>GET /courses/:course_id/files/:file_id/file_preview(.:format)
<Attachment Title>POST /courses/:course_id/files/:file_id/inline_view(.:format)
<Calendar Event Title>GET /courses/:course_id/calendar_events/:id(.:format)
<Calendar Event Title>PATCH /courses/:course_id/calendar_events/:id(.:format)
<Calendar Event Title>PUT /courses/:course_id/calendar_events/:id(.:format)
<Collaboration Title>GET /courses/:course_id/collaborations/:id(.:format)
<Conference Title>GET|POST /courses/:course_id/conferences/:conference_id/join(.:format)
<Outcome Title>GET /courses/:course_id/outcomes/:id(.:format)
<Page Title>GET /courses/:course_id/pages/:id(.:format)
<Page Title>GET /api/v1/courses/:course_id/pages/:url(.:format)
<Page Title>POST /api/v1/courses/:course_id/pages(.:format)
<Page Title>PUT /api/v1/courses/:course_id/pages/:url(.:format)
<Quiz Title>GET /courses/:course_id/quizzes/:id(.:format)
<Quiz Title>GET /courses/:course_id/quizzes/:quiz_id/history(.:format)
<Quiz Title>GET|POST /courses/:course_id/quizzes/:quiz_id/take(.:format)
<Quiz Title>GET /courses/:course_id/quizzes/:quiz_id/take/questions/:question_id(.:format)
<Quiz Title>POST /api/v1/courses/:course_id/quizzes/:quiz_id/submissions(.:format)
<Tool Title>GET /api/v1/courses/:course_id/external_tools/:external_tool_id(.:format)
<Tool Title>GET /courses/:course_id/external_tools/:id(.:format)
<Topic Title>GET /courses/:course_id/discussion_topics/:id(.:format)
<Topic Title>GET /courses/:course_id/discussion_topics/:id/:extras(.:format)
<Topic Title>POST /courses/:course_id/discussion_entries(.:format)
<Topic Title>POST /api/v1/courses/:course_id/discussion_topics(.:format)
<Topic Title>POST /api/v1/courses/:course_id/discussion_topics/:topic_id/entries(.:format)
<Topic Title>POST /api/v1/courses/:course_id/discussion_topics/:topic_id/entries/:entry_id/replies(.:format)
<User Name>GET /courses/:course_id/users/:id(.:format)
Course AnnouncementsGET /courses/:course_id/announcements(.:format)
Course AssignmentsGET /courses/:course_id/assignments(.:format)
Course CollaborationsGET /courses/:course_id/collaborations(.:format)
Course ConferencesGET /api/v1/courses/:course_id/conferences(.:format)
Course ConferencesGET /courses/:course_id/conferences(.:format)
Course DiscussionsGET /api/v1/courses/:course_id/discussion_topics(.:format)
Course DiscussionsGET /courses/:course_id/discussion_topics(.:format)
Course FilesGET /courses/:course_id/files(.:format)
Course GradesGET /courses/:course_id/grades(.:format)
Course GradesGET /courses/:course_id/grades/:id(.:format)
Course HomeGET /courses/:id(.:format)
Course HomeGET /api/v1/courses/:id(.:format)
Course HomeGET /api/v1/accounts/:account_id/courses/:id(.:format)
Course ModulesGET /courses/:course_id/modules(.:format)
Course OutcomesGET /courses/:course_id/outcomes(.:format)
Course PagesGET /courses/:course_id/pages(.:format)
Course PeopleGET /courses/:course_id/users(.:format)
Course QuizzesGET /courses/:course_id/quizzes(.:format)
Course SyllabusGET /courses/:course_id/assignments/syllabus(.:format)
SpeedGraderGET /courses/:course_id/gradebook/speed_grader(.:format)
Katrina Hess

New vs. Old Analytics

Posted by Katrina Hess Employee Dec 16, 2019

Hello Canvas Community!

 

We will continue to iterate on improvements to course-level analytics. In the meantime, we've created a guide of the differences. Please let me know in the comments if we've missed anything and I'll be sure to add it to the list!

 

New AnalyticsOld Analytics
Aggregate PeriodWeekly—per student, per section, per course*Daily—per student and course
Hourly AggregateNo*Yes—per student
Category AggregatesNo*Yes
Account AggregatesNo*Yes—across multiple courses in a sub-account
Data DisplayTable view
In development: Submission Date column in Student View data
Graphic Illustration view
Data ComparisonsCourse, Section, StudentN/A
Data CustomizationFilter by assignments, quizzes, and graded discussions and enable shape representationsN/A
Activity View
  • View specific students who viewed, didn't view, participate, or didn't participate
  • View specific students with missing or late assignments
  • Message students regarding status
  • View number of students with page views or participations
  • View number of students with missing or late assignments
Assignment Grade StatisticsMean score for the class, sections, and students
In development: median, high, and low scores per assignment grade
Median, high, and low scores per assignment grade
Rollup Resource DataView number of resources viewed with list of resourcesView number of resources viewed
Rollup for Student Assignment StatusStatus per assignment
In development: number of missing, late, on time per student
Missing, late, on time
Mobile Page View DataIncludedN/A
Student Messaging in Score RangesYes—via group message sent to students individuallyNo
Student Messaging for Missing/Late AssignmentsYes—via group message sent to students individuallyNo
CSV Data DownloadYesNo

 

*improves overall performance; this functionality will continue to be improved

 

 

Best, Katrina

 

 

Update 1-30-2020: New analytics provides a better representation on user activity than the old analytics. We removed extraneous requests to better align the data with student behavior.

Hello Canvas Community! 

 

There was a glitch in the matrix. We discovered a problem that affected the accuracy of your data. To ensure that data numbers are what they should be, we are updating the data source for both New Analytics and Canvas Data's request table.

 

What is changing?

In about a week or so, you'll notice an increase in the number of HTTP requests received daily, which will increase activity numbers in New Analytics. Reports will be adjusted to include the previously missing data—including historic activity. Canvas Data's request table is also affected but will only show updated activity numbers moving forward. No schema changes are involved.

 

[UPDATE 2019-11-20] We are also looking into the issue of increased request activity as stated in the comments below.

 

Why did this happen?

Canvas Data's request table and New Analytics expose HTTP request data to customers. These products rely on logs generated by Canvas web application servers which are archived to S3 periodically. A flaw was discovered in the archiving process that caused some logs to be discarded when web application servers were shut down.

 

How are we preventing this from happening again?

The Canvas DevOps team has developed a separate archiving process which does not exhibit the same data loss behaviors. Canvas Data's request table and New Analytics will be moving to this new archive source for HTTP request data.

 

We apologize for any concerns that may have been caused by this data inconsistency. This adjustment is one improvement in a longer journey towards data excellence at Instructure. Today represents an improvement in your data. We will continue to ensure we continuously deliver a high standard of data services for you because of how much data excellence matters to us.

 

Best, Katrina

Hello Canvas Community!

 

Our commitment to improving our Canvas analytics offering starts with the New Analytics release. This release focuses on interactive views for analytics within the course, as well as, the student view for a given course.

 

We've received insightful feedback since we launched to the beta environment back in Sep, and in production last month. We take your feedback seriously and are considering implementation and priority of each request.

 

As some of you may know, our approach to the product roadmap is to commit work to the closest upcoming quarters. This allows us to innovate with you as needed.

 

Instructure: How We Roadmap

 

That said, here are some of the features you can expect in the coming months:

  • bug affecting SIS ID column in the CSV download
  • interaction improvements to the detail card in the Course Grade chart
  • improvements to the data table layouts in the Course Grade and Weekly Online Activity charts
  • improvements to the support and performance for large course sizes
  • documentation for the updated API
  • enabling the back browser button
  • adding gestures and mouse scrolling for all charts
  • end of life for legacy course analytics
  • showing email subjects in the Communications chart
  • improvements to the grade distribution data (to include mean, median, and mode)

 

There are improvements slated throughout the year so please stay tuned and keep the feedback coming!

 

Best, Katrina

Hi everyone! Thanks so much for your feedback and patience as we roll out our latest version of course analytics. We hear you and want to make this transition as smooth as possible for your institution. 

 

Here are the changes we are currently making: 

  1. In place of an account-level feature option, we are creating New Analytics to be a course-level feature option. This change will let admins allow New Analytics on a course-by-course basis. 
  2. When the feature option is enabled, the New Analytics course navigation link will be visible by default. If the feature option is hidden, the navigation link will also be hidden.

 

Please watch the existing Analytics release notes document for updates on when these changes are available in your production environment. We anticipate we can get these updates to you this week.

 

Best, Katrina

Hello Analytics Beta Community,

Thanks for the continued questions and feedback. The team has been heads down this winter working on the second of three views for the new Course Analytics, and we are just about ready for you to try it out and let us know what you think.

 

Next Monday afternoon (MDT) we'll be turning on the Weekly Online Activity view for teachers in all existing Analytics Beta LTI instances. This view will be accessible via a dropdown option from the Course Grades view, and will include weekly trend data on page views and participations, the ability to compare by section or student, and drill down to a specific week, and immediately outreach to students who may need a nudge.

 

Along  with the release we'll be adding user guides, FAQ details, and a list of known issues to the Analytics Beta Community home page. We'll also include some definitions to help answer a few common questions that we get around this data. 

 

In the meantime, thanks again for all your feedback and help in making sure we work out the kinks before going to GA.

 

Best,

Kevin 

Hello Analytics Beta Community,

Thank you for trying out Analytics Beta, joining the community, and providing feedback. Open Beta is the place where we can kick the tires of new features, and make adjustments before general availability. To that end, we'd love some help with with a specific issue we are currently seeing with Activity and Interaction data.

 

In some cases the data in the table of Course Grades doesn't match what we are seeing with the actual student page view or submissions data. We're having a hard time pinning down the root cause and we'd love to collect more examples of inconsistent data so that we can identify any patterns if they exist.

 

If you are up for it, and see an data inconsistencies or other issues with Activity and Interaction data, please log a ticket with support either by sending an email here: support@instructure.com or via the help icon in the global nav of your Canvas Instance.

 

Please include "Analytics Beta-Activity Data" in the subject, and include details of the specific example. The course URL, student name, and screenshots  would be excellent if you can provide them.

 

As soon as we get to the source of the issue I'll reach back out to this group and provide an update.

 

Thanks for the help!

Kevin

Kevin Turco

Welcome to Analytics Beta

Posted by Kevin Turco Administrator Oct 29, 2018

Welcome to Analytics Beta!

My name is Kevin Turco, the new Product Manager of Canvas Analytics, and will be updating the community as we continue work on the new Canvas Analytics.

 

Today we are excited to announce the launch of Analytics Beta to all production environments.  This is the first of several releases leading up to General Availability.

 

Below are some helpful links that include important release information, user guides, FAQs, and ways for providing feedback.

 

What is New Analytics? 

New Analytics: Known Issues 

FAQ

Submit Feedback

 

These links along with other useful information can be found in our Analytics Beta Community page

User Group: Analytics Beta

 

Additionally, If you'd like to report an issue or bug, please use the HELP menu in Canvas to "Report a Problem" with our Support Team.

 

Thanks for trying out Canvas Analytics Beta. We look forward to hearing from you!

 

Thanks,

Kevin

Canvas beta resets every Monday. Until we launch to production later this month our engineering team must re-enable Analytics Beta for all beta accounts. If you have already dragged Analytics Beta to be visible in Course Navigation, you won’t have to do it again. Thanks for your patience!

 

Best, KH