Canvas Data 2 schema version errors

[mclark19]

@sgergely It seems that schema versions for a number of tables in the the canvas namespace have reverted. For example, "rubrics" had been at version 2 in mid February, but shows version 1 when downloaded today from DAP. The following tables seem to have issues:

access_tokens
assignment_overrides
assignments
attachments
conversation_messages
developer_key_account_bindings
discussion_topics
grading_standards
learning_outcomes
master_courses_master_content_tags
rubrics
submission_comments
submissions

We aren't using the DAP client to load the tables, but we do check versions numbers as part of our process.

[phanley]

I'm also seeing errors ('cannot drop values in an enumeration: pending' ) with the same tables this morning

Sample error log:

Running command: ['/usr/bin/bash', '-c', 'set -e; dap syncdb  --namespace canvas --table access_tokens || dap initdb  --namespace canvas --table access_tokens ']
Output:
Python version: 3.11.11 (main, Feb 25 2025, 02:38:55) [GCC 12.2.0]
Platform: uname_result(system='Linux', node='canvas-data-sync-dap-init-or-sync-db-i6kk27gj', release='5.15.0-131-generic', version='#141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025', machine='x86_64')
Package versions: {'pysqlsync': '0.8.2', 'json_strong_typing': '0.3.7', 'instructure-dap-client': '1.3.1', 'aiohttp-retry': '2.9.1', 'types-aiofiles': '24.1.0.20241221', 'tsv2py': '0.7.1', 'aiofiles': '23.2.1', 'PyJWT': '2.10.1', 'aiohttp': '3.11.11'}
pysqlsync.postgres:connecting to cd@dbserver:5432/canvas
pysqlsync.postgres:PostgreSQL version 16.0.6 final
Database version: PostgreSQL 16.6 (Debian 16.6-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
INFO: pysqlsync.postgres:connecting to cd@dbserver:5432/canvas
ERROR - operation not permitted; cannot drop values in an enumeration: pending
INFO - Python version: 3.11.11 (main, Feb 25 2025, 02:38:55) [GCC 12.2.0]
INFO - Platform: uname_result(system='Linux', node='canvas-data-sync-dap-init-or-sync-db-i6kk27gj', release='5.15.0-131-generic', version='#141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025', machine='x86_64')
INFO - Package versions: {'pysqlsync': '0.8.2', 'json_strong_typing': '0.3.7', 'instructure-dap-client': '1.3.1', 'aiohttp-retry': '2.9.1', 'types-aiofiles': '24.1.0.20241221', 'tsv2py': '0.7.1', 'aiofiles': '23.2.1', 'PyJWT': '2.10.1', 'aiohttp': '3.11.11'}
INFO:pysqlsync.postgres:connecting to airflow_user@canvas-data-rw:5432/canvas
ERROR - operation not permitted; cannot drop values in an enumeration: pending
INFO - Command exited with return code 5
ERROR - Task failed with exception

[StevenWalls]

We are getting those errors this morning as well on multiple table syncs.

[reynlds]

I'm seeing something very similar on these tables:

access_tokens
assignment_overrides
assignments
attachments
conversation_messages
custom_gradebook_columns
developer_key_account_bindings
discussion_topics
grading_standards
learning_outcomes
lti_resource_links
master_courses_child_subscriptions
master_courses_master_content_tags
rubrics
submission_comments
submissions
submission_comments
submissions
web_conference_participants

Here are the errors I'm seeing...everyt weekend things go to heck...:

2025-03-03 06:05:27,773 - ERROR - operation not permitted; cannot drop values in an enumeration: pending

2025-03-03 06:08:21,369 - ERROR - operation not permitted; cannot drop values in an enumeration: Course

2025-03-03 06:08:38,755 - ERROR - operation not permitted; cannot drop values in an enumeration: outcome_alignment_cloningfailed_to_clone_outcome_alignment

2025-03-03 06:09:26,514 - ERROR - operation not permitted; cannot drop values in an enumeration: public

2025-03-03 06:16:03,345 - ERROR - operation not permitted; cannot drop values in an enumeration: audio/aacaudio/amraudio/flacaudio/mp4audio/mpegaudio/oggaudio/vnd.dlna.adtsaudio/wavaudio/webmaudio/x-m4aaudio/x-ms-wmaaudio/x-wavvideo/3gppvideo/avivideo/mp4video/mpegvideo/oggvideo/quicktimevideo/webmvideo/x-m4vvideo/x-matroskavideo/x-ms-asfvideo/x-ms-wmvvideo/x-msvideo

2025-03-03 06:19:40,419 - ERROR - operation not permitted; cannot drop values in an enumeration: deleted

2025-03-03 06:21:46,117 - ERROR - operation not permitted; cannot drop values in an enumeration: not_threaded

2025-03-03 06:25:41,398 - ERROR - operation not permitted; cannot drop values in an enumeration: archived

2025-03-03 06:28:34,728 - ERROR - operation not permitted; cannot drop values in an enumeration: weighted_averagestandard_decaying_average

2025-03-03 06:29:29,780 - ERROR - malformed HTTP response:

<TITLE>ERROR: The request could not be satisfied</TITLE>

<H1>504 ERROR</H1>

2025-03-03 06:29:29,799 - ERROR - malformed HTTP response

2025-03-03 06:30:46,728 - ERROR - malformed HTTP response:

2025-03-03 06:30:46,747 - ERROR - malformed HTTP response

2025-03-03 06:31:02,721 - ERROR - operation not permitted; cannot drop values in an enumeration: MediaTrack

2025-03-03 06:37:35,275 - ERROR - operation not permitted; cannot drop values in an enumeration: archiveddraft

2025-03-03 06:38:51,185 - ERROR - operation not permitted; cannot drop values in an enumeration: audio/aacaudio/amraudio/flacaudio/mp4audio/mpegaudio/oggaudio/vnd.dlna.adtsaudio/wavaudio/webmaudio/x-m4aaudio/x-ms-wmaaudio/x-wavvideo/3gppvideo/avivideo/mp4video/mpegvideo/oggvideo/quicktimevideo/webmvideo/x-m4vvideo/x-matroskavideo/x-ms-asfvideo/x-ms-wmvvideo/x-msvideo

2025-03-03 06:39:32,565 - ERROR - operation not permitted; cannot drop values in an enumeration: audio/aacaudio/amraudio/flacaudio/mp4audio/mpegaudio/oggaudio/vnd.dlna.adtsaudio/wavaudio/webmaudio/x-m4aaudio/x-ms-wmaaudio/x-wavvideo/3gppvideo/avivideo/mp4video/mpegvideo/oggvideo/quicktimevideo/webmvideo/x-m4vvideo/x-matroskavideo/x-ms-asfvideo/x-ms-wmvvideo/x-msvideo