Register for InstructureCon25 • Passes include access to all sessions, the expo hall, entertainment and networking events, meals, and extraterrestrial encounters.
Found this content helpful? Log in or sign up to leave a like!
I am currently assuming that a web_logs record is never updated inside of Canvas Data 2. Is this assumption true?
Background:
I am currently importing Canvas Data 2 web_logs records via the dap sync command regularly. I have a process that regularly consumes the web_logs records and deletes them from our database after processing them (this saves on required storage space). I currently have a problem where there seems to be a discrepancy between our copy of Canvas Data 2 (at the time of being processed before being deleted) and the data from the Data Access Platform after fetching a fresh snapshot at a later time while troubleshooting the problem.
Is there ever a situation where a web_logs record has a column later updated? For example, in the Data Access Platform, a web_logs record is created with a missing course_id column but then a few seconds later that same record's course_id is populated? If this is the case, then for my pipeline it would be possible that I sync that record, delete it, but then later when syncing there is an update for that same record but that record is deleted so that update is never reflected in my copy of CD2.
Hello @LeventeHunyadi, hope you are well! Would you be the right person to answer this question?
To interact with Panda Bot in the Instructure Community, you need to sign up or log in:
Sign In