Data and Analytics Group Discussions

Data and Analytics Group questions and discussion forum

Hi Folks!After running into the issue of no data being inserted after syncdb, I reinstalled everything, including python, DAP client, PostgreSQL and upgraded to tsv2py 0.6.2. That seemed to solve the issue of *no* data being inserted into my PostgreS...

  • 0 Likes
  • 2 Replies

Has anybody that was using the Harvard Data 1 extract successfully converted it to use Data 2 and was it just changing the API data?

Community Novice
Data and Analytics Group
  • 0 Likes
  • 0 Replies

We are looking to download certain table data, transform it, and then upload to our Azure DataLake storage.The solution I have been working on involves a Python Azure Function app. Since the DAP parquet files have a "key", "value", and "meta" columns...

  • 0 Likes
  • 6 Replies

In a course's People list, each user has their total activity (hh:mm:ss) listed. Is there any way a teacher in the course could export this data as a report? Or as an admin, for the entire account activity? It would be ideal if admins could report on...

  • 0 Likes
  • 1 Replies

Good morning folks!After updating to the new stable release of the DAP CLI tool, I was able to smooth out some existing issues which some of you have already experienced:TCI_KEEPIDLE issue was an OS/Python issue "Please use initdb" when trying to use...

  • 0 Likes
  • 6 Replies

I'm trying to extract all clicks to a specific URL from web logs, with user ID-level detail. However, when filtering the logs to that URL, I noticed some users have multiple records (5-10) with timestamps only a few seconds apart. This suggests repea...

  • 1 Likes
  • 3 Replies

Hi - I'm using DAPClient 1,1 and postgres 16 With an incremental query I've discovered we seem to have started losing pseudonym "id"'s. e.g. users.id 2720 has pseudonym.id 2689, but in the pseudonymn table 2689 has been skipped, so has 2693. Confirme...

Community Participant
Data and Analytics Group
  • 0 Likes
  • 7 Replies

Hi,    Does anyone have any idea how to track the usage of SCORM packages across the entire instance. I need the number of packages deployed and the number of courses that have at least one package deployed

  • 0 Likes
  • 1 Replies

To follow standard best practices in online courses, we require our facilitators to send welcome messages (sometimes more than one) to participants before the course is published.  The roster in grades and people shows student logins or SIS ID's (whi...

Community Explorer
Data and Analytics Group
  • 2 Likes
  • 0 Replies

On several table downloads using initdb I'm getting dozens of the error below. This has only been on a few tables so far. I have tried dropping the table and re-running, that worked once but not on the conversation_messages table.  We're using Python...

  • 1 Likes
  • 4 Replies

It would be immensely helpful if the Canvas Sys Admin (or course teacher if that was possible) was able to see the progress New Analytics was making.  If it goes over 24 hours, usually teachers will contact support thinking something is wrong, and th...

Community Explorer
Data and Analytics Group
  • 1 Likes
  • 0 Replies

Hello all, our team is new to developing solutions to pull data from Canvas.  I have been lurking in the forums for a bit, trying to gather as much information I can to set up the correct tools that we would need to be successful.  However the one pa...

  • 0 Likes
  • 2 Replies

Hi All,Our group recently rebuilt our entire Canvas database with the release of DAP 1.1.0 (we were behind and using a deprecated version until that point). After dropping and initdb'ing all our tables again, everything was working again, with the ex...

  • 0 Likes
  • 3 Replies

Hey Community,I’m currently trying to migrate my application to use Canvas Data 2 and I’ve been running against a problem and was wondering if you could help.Example:RootAccountSubAccount1Account1Account2SubAccount2Account1Account2I want to retrieve ...

  • 1 Likes
  • 4 Replies

Confession: I am not a developer or an SQL expert, but I can view certain kinds of code, sort of learn it, and make adjustments as needed. Otherwise, I rely heavily on ChatGPT, particularly when it comes to SQL.  That said, I was wondering if someon...

  • 2 Likes
  • 15 Replies

When we are running our DAP synchronizer python script, we are running into issues going back to Feb 27. This was the last time it ran successfully and we noticed it 2 days ago. When diving into the issue we are finding that several tables now are tr...

  • 0 Likes
  • 1 Replies

Hello Community,I'm currently working with the DAP CLI tool to import Canvas Data 2 into a database, but I'm running into an issue where the tables and column headers are created successfully but the table rows remain empty. This is after running the...

  • 0 Likes
  • 3 Replies

Good afternoon, PostGres SQL queries using Canvas Data 2 for the public table, submissions column, submitted_at field returns inaccurate dates. Below is an example of the results of a submitted_at field for a student in a class.   The top is null bec...

  • 0 Likes
  • 6 Replies

In v1.1 of the newly released dap client, there is now a decompress parameter in the API.  Is there a plan to include this parameter in the CLI version of the dap client?  I love this feature and have a workaround in place to change the default to Tr...

  • 0 Likes
  • 1 Replies

I am getting a 403 error on the job endpoint specifically. Please see the message below ********************** Got response: HTTP/1.1 403 Forbidden [Server: CloudFront, Date: Fri, 22 Mar 2024 13:53:34 GMT, Content-Type: text/html, Content-Length: 915...

  • 0 Likes
  • 4 Replies

I believe that I have followed the install directions for placing the Canvas Data 2 Command Line tool on my machine. I've created my Canvas Data 2 API key.I thought I would try out the simple command below, which is supposed to return a list of table...

  • 0 Likes
  • 1 Replies

We devised a simple workaround to utilize dap 0.3.18 to load our production MySQL 5.7.15 tables. All of our CD2 SQL queries have been tested and deployed into production using the non-prefixed table names, e.g. assignments rather than canvas__assignm...

  • 0 Likes
  • 2 Replies

Hi , In the CD1 to CD2 schema mapping document below,[PUBLIC]: CD1 to CD2 schema mapping - Google Sheetsenrollment_dim&fact sheet user_id field is obfuscated using the command "obfuscate_global(cast(enr.user_id AS bigint), g_shard_id)".  But it's com...

Community Member
Data and Analytics Group
  • 0 Likes
  • 0 Replies

Can somebody please answer this question for me.  I am experiencing very confusing anomalies between data obtained via API calls and corresponding data obtained via the New Analytics UI.Specifically, I ran the following API call to extract details of...

  • 0 Likes
  • 2 Replies

I was working on switching over a process I had previously been using CD1 Requests for to show the break down of the types of Operating Systems and Browsers are used by our faculty and students. When I was comparing the data I noticed there was subst...

  • 1 Likes
  • 5 Replies

Howdy, I am trying to gather information on the number of assessment submissions are being marked by each of our marking staff. I use CD2 CLI to download CD2 tables. I then import into a database and then run SQL queries on the data. In the submissio...

  • 1 Likes
  • 1 Replies

I am able to call the schema endpoint, but I am getting a 404 error on the data endpoint 'https://api-gateway.instructure.com/dap/query/canvas/table/' + $splitValue + '/data' The error is: Failure: Failed to execute the request, or the endpoint respo...

  • 0 Likes
  • 4 Replies

Hello,   I am trying to use DAPSession.stream_resource to stream resources/files directly into an S3 bucket. The idea is to pipe the stream directly into S3 without downloading and storing the entire file locally first, and then reuploading it into S...

  • 0 Likes
  • 7 Replies

Hi all,We'd like to sync Canvas Data2 to our data warehouse, which is on Snowflake. Snowflake uses its own SQL dialect.The DAP Client documentation states that it only supports Postgres and MySQL. Prior to the release of v1.0.0, there were instructio...

  • 0 Likes
  • 7 Replies

In the CD2 data I'm working with I have noticed that a course in the Courses table has a NULL value for the Grading_Standard_ID column.  However, in the Grading_Standards table there is a row where the Context_ID column has the ID of the course. Is t...

  • 0 Likes
  • 2 Replies