Canvas Data 2/DAP Client Library Production Readiness
My institution has historically used Canvas Data to access and extract data that in turn is loaded and used in our data warehouse. In response to the announcement of Canvas Data 2 (CD2), we started an implementation of new data flows which access CD2 through the DAP Client Library (CLI) provided by Instructure. However, we haven't been unable to reliably extract the data we need through Canvas Data 2 using the provided CLI.
The issues we've seen are primarily generic HTTP 400 errors (with equally generic error messaging), which are raised by the CLI when attempting to download snapshots of data tables requested through CD2. However, this also creates a problem in which we have not been able to fully validate the integrity and accuracy of the data returned from CD2 against "Canvas Data 1" (CD1) data we are currently using in Production (as is recommended by Instructure when integrating CD2 into ETL). We're told the download issues are a result of the amount of data our institution has and that the engineering team was already working on some fix to ensure larger datasets can be downloaded successfully, but were provided no timeline for when this fix may be available. We understand there are other institutions experiencing this issue as well. As an additional factor, the DAP Client Library is still listed as being in the beta stage of development despite CD2 reaching GA eight months ago.
With the deprecation date for CD1 being next month, there is growing concern at my institution from both technical and business stakeholders that Canvas Data 2/the DAP Client Library is not ready for Production use. Is Instructure still committed to the December 31st deprecation date? If so, what options do institutions have to ensure uninterrupted access to reliable, complete, and accurate Canvas data? Finally, will the DAP Client Library be promoted to a Production-ready state in time for the deprecation date?