Found this content helpful? Log in or sign up to leave a like!

Are web_logs records in Canvas Data 2 ever updated

SpencerWilliams
Community Participant

I am currently assuming that a web_logs record is never updated inside of Canvas Data 2. Is this assumption true?

Background:

I am currently importing Canvas Data 2 web_logs records via the dap sync command regularly. I have a process that regularly consumes the web_logs records and deletes them from our database after processing them (this saves on required storage space). I currently have a problem where there seems to be a discrepancy between our copy of Canvas Data 2 (at the time of being processed before being deleted) and the data from the Data Access Platform after fetching a fresh snapshot at a later time while troubleshooting the problem.

Is there ever a situation where a web_logs record has a column later updated? For example, in the Data Access Platform, a web_logs record is created with a missing course_id column but then a few seconds later that same record's course_id is populated? If this is the case, then for my pipeline it would be possible that I sync that record, delete it, but then later when syncing there is an update for that same record but that record is deleted so that update is never reflected in my copy of CD2.

0 Likes