Stay Ahead with CD2 / DAP: Your Feedback, New Features, and the 4th Upcoming Live Meetup

sgergely
Instructure
Instructure
7
1762

Canvas.png

We are conducting the Instructure Data Questionnaire to inform our 2025 roadmap and understand the cloud adoption trends among our users. Your input will help us tailor features, optimize data services, and plan future enhancements. The questionnaire is anonymous, but there is an option to share your email for follow-up. Whether you’re using a cloud provider, data warehouse, or BI tool, your feedback is invaluable. Complete the survey to shape the future of Instructure’s data capabilities. Link to Questionnaire

Customer Reported an Edge Case for DAP Query API

When using the DAP Query API, understanding edge cases is crucial for ensuring smooth operations, especially when working with rarely updated tables like canvadocs_annotation_contexts. This blog post explores a unique edge case around query expiration and how to handle it.

The Edge Case Explained

Imagine a customer querying a rarely updated table. The query result is cached, and the expires_at field is set 24 hours from the original query creation time. However, if the same query is issued just before expiration and reused after it expires, the downloadable links may fail despite the data still existing in our system for debugging purposes. Here’s a UML diagram to illustrate the situation:

edge-case-uml-diagram.png

In this scenario, if a customer tries to retrieve downloadable links after the query expires (e.g., September 2, 00:01), the system will return a 404 error, even though the data still exists in our system for debugging purposes. The downloadable links are no longer valid because the query has technically expired. To handle this edge case gracefully, your application needs to be aware of the expires_at field and take action accordingly.

Importance of Using Snapshot Only Once

One of the core principles of using the DAP Query API effectively is utilizing snapshots only once and then relying on incremental updates to keep your data up-to-date. Snapshots are resource-intensive, both on the API and client side, and should only be used to initialize your data set. Once the snapshot is established, incremental queries should be used to fetch only the latest data changes. This approach minimizes resource consumption and ensures your data stays fresh.

Upcoming DAP CLI 1.3.0 Release

In the upcoming DAP CLI 1.3.0 release, we’re adding edge case handling for query expiration, as described above, directly into our CLI and library.

Along with this, the release will also introduce support for Microsoft SQL Server.

Upcoming Canvas Data 2 Online Meetup

The 4th meetup is already scheduled for Fri, Nov 21, 2024 05:30 PM CET / 11:30am EST / 8:30am PST, don't miss it, subscribe here!

The previous meetup slides and recorded video can be checked once the survey is filled.

Question for the comment section:

What would be your feedback if we would launch New Quizzes data through DAP Query API in an additional, separate namespace?

7 Comments
robotcars
Community Champion

@sgergely 

Friday November 29th, is wrapped into a holiday in America and most education staff won't be working that day.

NathanielMille4
Community Member

Friday November 29th, is wrapped into a holiday in America and most education staff won't be working that day.

sgergely
Instructure
Instructure
Author

@NathanielMille4 and @robotcars thanks for the heads up! I will reschedule ASAP.

sgergely
Instructure
Instructure
Author

I have modified the event to Nov 21, same time. I hope you can make it!

KELANESPY
Community Member

Friday November 29th, is wrapped into a holiday in America and most education staff won't be working that day.

sgergely
Instructure
Instructure
Author

Hey @KELANESPY I have modified the date to 21 on the event page and in the blog post as well. Are there any additional place where it is not yet modified besides the comments here?

robotcars
Community Champion

Does it seem like other replies are quoting mine?