Live Events Services - Table of Contents

Document created by Oxana Jurosevic Administrator on Oct 17, 2018Last modified by Oxana Jurosevic Administrator on Oct 18, 2018
Version 16Show Document
  • View in full screen mode

You are here :  Canvas Data Services  >  Live Events > Live Events Services - Table of Contents

 

 

Data sharing options: Event Data

 

Event data is a record of a single event at a particular moment in time. An event consists of core attributes such as a event_name and event_type, and can be annotated with many additional attributes to provide more context. For example, Canvas captures detailed events when a user traverses a Canvas account or course.

 

Canvas collects numerous data points and emits some of them via its Live Events Data Service. The Live Events user interface displays a list of available event types that the customer could  choose from depending on their needs. There are two formats of event data available for Canvas customers: Caliper v1p1 Basic Implementation, and Canvas Live Events Raw (Alpha). The event type and format matters when you want to answer a specific business need.

 

This document explains which tasks each data format is most suitable for, how to configure Canvas customer account to receive event data, and how other Canvas customers use such data.




Overview: Event Data

 

Event data is useful when you need to filter data based on arbitrary characteristics—such as data queries that are unique to customer external data warehouse application, learning analytics, histograms, etc. Canvas emits numerous events with a list of attributes attached to each event. For example, individual user logged_in and logged_out sessions, assignment submission details, and grade_changed transactions.

 

  • Event data is not aggregated over time; therefore, it should not be used as a single source for your data warehouse. Use it in conjunction with Canvas Data Extracts and Canvas APIs to ensure integrity of your LMS analytics dataset.

 

  • Event data is not sequenced, and can only be arranged in time based order via the "event_time" attribute expressed in ISO-8601 where data and time values are formatted with the addition of millisecond precision. The format is yyyy-MM-ddTHH:mm:ss.SSSZ where ‘T’ separates the date from the time while ‘Z’ indicates that the time is set to UTC.

 

  • Events are delivered in a "best-effort" fashion. In order to not slow down web requests, events are sent asynchronously. That means that there's a window where an event may happen, but the process responsible for sending the request to the queue is not able to queue it.

 

  • At this time Event data can only be sent to AWS SQS queue.



Event Data Formats

 

There are two types of Canvas event format available to our customers:

 

 

Each of the listed formats can help you to meet your institutions unique analytics needs.

 

 

Configuration requirements

 

Due to a new technology concept Live Events is currently an invite-only feature. All major required configurations are completed by the Instructure engineering team based on requests submitted by a client CSM.

 

As soon as configuration setup is complete, the navigation menu in the clients Canvas account will include a new option that reads: “Canvas Data Portal.”

The Canvas Data Portal will provide access to Canvas Data with data extracts, user management, and API credentials as well as Live Events Subscription - Caliper ONLY: where you can add a new subscription destination (SQS URL) and select the type of events you would like to receive.

 

Things to consider:

 

  • Not all event types are listed in the UI of the subscription. The format you are receiving events in will change the amount of event types you receive.
    • Please refer to the Event Type by Format table for the list of available event types for each format.
  • eventTime for Canvas based events does not provide millisecond precision and is expressed in the following format, e.g. "eventTime": "2018-10-01T21:22:11.000Z".
    • eventTime for Quizzes.Next does provide millisecond precision and expressed in the following format: "event_time": "2018-10-02T17:10:32.881Z"


Roadmap considerations

 

The following features are currently considered for Live Events services :

 

  • User Interface for an Administrator to subscribe to specific events for Live Events and Caliper formats
  • Ability to subscribe to specific events based on the producer ( User generated vs System generated )
  • Ability to receive data via https endpoint
  • Enable mobile asset_accessed events for the asset_accessed events available for web app
  • Additional request table data added to body of the events : course_account_id, URL of the accessed object, IP address of the event producer
  • Event sequencing : Include milliseconds in the Canvas "event_time" timestamp
  • Add support for additional  client side quizzes.next events :  student - item interactions
  • Add support for  Arc media events : student - media interaction

 

 

Event Type by Format

 

This table describes which type of Event is supported by a specific Format. Note that any event that starts with “quizzes” describes an event that occurs in Instructure’s Quizzes.Next solution. See the table with event types and available formats here. 

 

asset_accessed event : type, subtype and triggers

 

See the TABLE: Asset Types, Asset Subtypes and Triggers for details

 

How the Events are Used

 

Business Needs:

 

To determine ways in which you can make use of different types of events requires that you ask a specific business question and seek its answer by collecting and visualizing data. Here are some of the questions our clients ask when they are considering using our Live Events service:

 

Assessment (Quiz) Submissions

 

Tracking patterns using the quizzes.next and old quizzes events will allow instructors to understand more about how students are interacting with their assessments

 

  • How much time is required to submit an assessment?
  • If test-taking times are flexible, when do students submit their assessments?
  • How much time does it take to grade an assessment after a students submits it?
  • How often are assessments being revised by instructors?
  • How often are instructors relying on external assessment content or re-using assessment content (imported via QTI) when building their assessments?

 

   User Sessions

 

A session is a group of user interactions in the Canvas Learning Platform that take place within a given timeframe, e.g. a single session can contain multiple page views, submission events, discussion entries, and assignment content interaction. We can think of a session as a container for the actions a user takes in Canvas. A single user can open multiple sessions. Those sessions can occur on the same day, or over several days, weeks, or months. Sessions are limited based on time limit.

 

  • What is my average user session duration?
  • Number of sessions by type of canvas user.
  • Are there any common patterns in different user sessions that attribute to the distribution and setup of the course content? I.e. Are your course structures/content convoluted and students get lost a lot?
  • What is the typical time of the day your users visit Canvas by user type?
  • When do users typically leave Canvas by user type?
  • What does a user Canvas session look like and are there any similar patterns across the same types of users—e.g. users participating in the same group or course?
  • How often is the content viewed?
  • What paths are taken to reach the specific content?

 

Course assignments and submissions

 

This event data could help instructors to gather insight about the relationship between students and their assignments.

 

  • What type of assignments are typically gradable?
  • How long does it take to submit something that has been assigned?
  • What assignments present the biggest challenge, e.g. which need the most retake attempts?

 

    Course content  

 

This event data could help administration to assess the account course content changes on periodic basis.

 

  • When does content typically get revised?
  • How often is course content being copied over?
  • How complex is the typical course content structure?
  • How often does the course syllabus get revised?  

 

 

How Customers and Partners use Events Data Service

 

Examples of how partners and customers are using our data services where events services is one of them.

 

            Blackboard Ally

 

  • Leverages Event Data Service as a notification service for a number of objects in Canvas course, events serve as triggers for this product to access Canvas data via API calls to assemble full dataset around specific course content to produce analytics around the content accessibility level. See type of events used by Ally here. 

 

            Instructure Catalog

 

  • Uses a full payload of one of  our events to track student progress thru the course module progression  and displays key payload data in the catalog course progress section



           Clark County School District, Las Vegas

  • Uses events as a notification service and Canvas Data as a main source for LMS data to provide support for multiple homegrown applications such as :
    • nudge - student notification services
    • current user activity reports
    • education tools:  student assistance services - live chat with instructors ( track when students are available  to assist with their course work)

 

 

Examples of Use Cases from Customer Interviews

 

 

Use Case

Event Type

Notes

Count of logins over time by institution or account

logged_in

Current payload for logged_in event type provides "user_account_id"

First login by instructors for new institutions (for adoption support)

logged_in

Current payload for logged_in event type provides "user_account_id"

Number of LTI launches by tool

asset_accessed : context_external_tool

Reading activity vs. assessment activity

asset_accessed, submissions events

this requires some classification to determine what constitutes "reading activity"

Measure the time between when the instructor first publishes the assignment to when a student first views it

asset_accessed ( by user_id and context_type with asset_type = assignment) assignment_created

currently the only way to determine when an assignment is published is to look in an extension value for the workflow state = 'published'


Also, determining when a student first "opens" an assignment requires parsing the object_id in the Navigation event for the specific assignment url

Which pages are most frequently accessed

asset_accessed

Avg session duration across all students in a course

logged_in, logged_out

This use case is often asked about but can be tricky to calculate based on existing events (primarily due to the fact that there is no event reflecting the "end" of a session

Click-path analysis

asset_accessed

Attempt timestamp usage trends

submission_created, submission_updated

Avg time spent by students in any given assessment

asset_accessed, submission_created, submission_updated

Generate alerts when student has failed attempt more than X times

submission_created, submission_updated

How many question types used in assessments

quizzes.item_created

This use case is better to approach by querying Canvas Data dimensions

When instructors create assignments

assignment_created

This use case is better to approach by querying Canvas Data dimensions

When instructors set due dates for assignments

assignment_created, assignment_updated, discussion_topic_created

This use case is better to approach by querying Canvas Data dimensions

 

Community Resources

 

The Problem with the Requests Table

Caliper Focus Group

Caliper v1.1 Guide

 

 



You are here :  Canvas Data Services  >  Live Events > Live Events Services - Table of Contents

3 people found this helpful

Attachments

    Outcomes