CD2 Quiz submissions missing / losing data

KeithSmith_au
Community Contributor

I am having some issues with this table, and I suspect that there is an underlying issues, probably to do with the statement in the documentation that:

Stores attributes for the last submitted quiz.

This tables persists one record per user_id and quiz_id

My assumption of fetching any data for any table in CD2 is that executing a snapshot, then subsequent delta queries using the "from" of the most recent fetched data, and an unbounded "until", and applying the subsequent upsert/deletes should bring the dataset to the same place as taking a snapshot.  i.e. a complete incremental query from the first time in the data should be the same as a snapshot.

For this table, this is not the case.  I fetched the incremental delta, then fetched a snapshot - which both return data to the same time.  I compared my built up data to the snapshot, and I have MORE records than the snapshot - things have disappeared, but no D incremental deletions have been processed.

I have also fetched an incremental delta specifying FROM the 01 January this year.  I know that a LOT of the quizzes in these courses have multiple submissions. As the ID of the quiz_submission table changes for each submission, and as it is supposed to keep only the most recent submission for each user and quiz_id, I would be expecting to have a lot of "D" updates as the old records are replaced.  I only found 66 out of nearly 700,000 records.

This leads me to believe that the incremental update query for this table is not returning appropriate deletions.

However, there is a bigger problem here as well.  Some of the records that have disappeared (no longer in the latest snapshot) have NO records for that user_id and quiz_id in the table - but the user is still enrolled and active in the course.  So not only are quiz_submissions disappearing without the incremental update being recorded, but we are losing data.

Can anyone shed any light on what is going on here - or do I just raise a support ticket?

During the beta release I requested that this table not hold only the most recent quiz submission, but that it expose all of them - which may correct this problem.

Labels (1)
0 Likes