Decrease latency in Canvas Data - other options?

[nate_mcclennen]

Hi, We are exporting data from Canvas Data and notice a 1-2 day delay in what has been entered into Canvas by teachers. Does anyone know how/if there is anything we can do to get more real-time data out of the Canvas platform? Obviously the reports that come out of Canvas in settings are much closer to real-time data, so that is an access point, but we need data that is not found in reports.

[robotcars]

Hi  @nate_mcclennen 

The delay in Canvas Data is do to the batch nature, the current product simply can't produce the volume of data with less latency. Canvas is currently working on a new version which would offer less latency.

More here Canvas Data Access Offering: Share Your Thoughts about Datasets 

Canvas Live Events on the other hand is a real-time message system the delivers events in two formats. The Alpha or (Canvas Raw) format, which closely mirrors what you'd expect from the API or CD, and the Caliper format, which uses IMS Global standards, so the the data, fields, and structure look different.

Here is a discussion about some differences between the two formats, and an example of how I combined the two data sets for real-time reporting. Live Events: Suffering with IDs 

Also see  @millerjm ‌'s presentation Combining canvas data and live events to have timely data 

Depending on your budget, skills, team etc, you can build Lambda architectures easily with AWS and other products, or you can look into some legacy hail mary options on the Awesome list linked above.

Happy to answer any questions.