AnsweredAssumed Answered

How do we relate Canvas Data Portal to caliper events?

Question asked by Pooja Kosala on Sep 21, 2018
Latest reply on Sep 24, 2018 by Robert Carroll

Like • Show 0 Likes0
Comment • 7

I am exploring canvas OLAP database similar to this (Canvas Data Portal ) with caliper events captured for canvas. I struggling to relate the content in canvas with caliper.

For example, I am trying to find the number times a user viewed a course announcement. Since the ID for each table(dimension) is not global, i am not able to relate data correctly.

For example, I am looking up id(announcement id) field of discussion_forum_topic in caliper data (object_extensions_enitityid) this field sometimes has id(announcement id) and sometimes has course id (from cvs_course_dim) how do i correctly join the course id in caliper with course_id in canvas, and similarly for other components like announcements, modules etc.

Do you have a documentation or mapping to relate caliper with canvas? If yes, could you share that?

Appreciate your help !

No one else has this question

Outcomes

7 Replies

Kona Jones Sep 23, 2018 7:30 AM
Mark Correct
Correct Answer
Pooja, greetings! Due to the technical nature of this question I’m going to share this with the Canvas Developers and Canvas Data Services groups in the Community. My recommendation is to also join or follow these groups so you’ll have access to their information and resources!

Kona
Like • Show 0 Likes0
Actions
- Robert Carroll @ Kona Jones on Sep 24, 2018 2:10 PM
  Mark Correct
  Correct Answer
  Kona, thanks for sharing!
  
  Hi Pooja Kosala,
  
  I had about 80% of a very detailed response written up here, and as I kept working on it and rereading your question I found a few issues that I decided needed further clarification before I could answer your question.
  
  Note that discussion_forum_topic doesn't seem to be an event type yet,
  but these two exist: discussion_topic_created, discussion_entry_created
  
  I had already played with discussion_entry_created, but not discussion_topic_created, so I wrote it into my code to see what was happening. This event happens when a new discussion is created, which can be an announcement, but has nothing to do with the user reading it. Discussion Entry Created, is fired when a student replies to a discussion, this one is fun to watch in the terminal, out of context from viewing the thread.
  
  For announcement views, if the specific data exists, I believe the event you are looking for is Asset Accessed, but I have to confirm with some testing. I've been playing around with this for a bit, but not confident enough to fully describe what's happening and the metadata for this event becomes very unpredictable. I currently have this data polling into the database so I can collect the messages and look at them from a different perspective.
  
  While that's collecting, I thought I'd share this query.
  -- mssql, course announcements viewed by user SELECT course_id, user_id, enrollment_type, discussion_id, title, count(timestamp) views FROM ( SELECT requests.course_id, requests.user_id, ed.type AS enrollment_type, discussion_id, url, dtd.title, requests.timestamp FROM dbo.requests JOIN dbo.discussion_topic_dim dtd ON (dtd.id = requests.discussion_id) JOIN dbo.enrollment_dim ed ON (ed.course_id = requests.course_id AND ed.user_id = requests.user_id) WHERE requests.course_id = 123456 AND PATINDEX('/courses/%/discussion_topics/%', url) >= 1 AND dtd.type = 'Announcement' ) x GROUP BY course_id, user_id, enrollment_type, discussion_id, title
  This would answer the 'number of times a user in a course viewed an announcement' using Canvas Data. You would have to import those tables, and all the requests table files for the current term. I download CD and requests every day allowing me to pull queries like this daily or in our LTI.
  
  I would still like to answer the question 'how to relate canvas data to caliper events',
  ...after some clarification I can explain how I join Live Events Data to Canvas Data records.
  
  I can also help if you need assistance mapping the event message and caliper docs to their respective Canvas relationships.
  
  Over 5,000 events stored in my asset_accessed table in 15 minutes, I'll see what I can find.
  2 people found this helpful
  Like • Show 0 Likes0
  Actions
  - Pooja Kosala @ Robert Carroll on Sep 24, 2018 2:26 PM
    Mark Correct
    Correct Answer
    Hi Robert,
    
    Thank you for your response. My Organization is logging Canvas events using Capiler and dumping the canvas data (OLAP) data. I am specially interested in getting assistance with mapping the Capiler events with Canvas as you mentioned:
    I can also help if you need assistance mapping the event message and caliper docs to their respective Canvas relationships.
    
    Here is the snapshot of caliper table, if that helps:
    Like • Show 0 Likes0
    Actions
    - Robert Carroll @ Pooja Kosala on Sep 24, 2018 3:06 PM
      Mark Correct
      Correct Answer
      Since Canvas Data and Live Events are two different Canvas features, can we be specific about what data we're looking at? Canvas Data or Live Events, but no Caliper (OLAP) Canvas Data.
      
      Back to the questions:
      How are you getting all those fields flat, what tool are you using? My own curiousity*
      Are you looking for discussion topics/entries or?
      Are you still trying to find announcement views in a course?
         Looking at all those flat fields/columns, have you identified one that contains announcement_id?
      Did you look through Asset Accessed?
      Which event are you trying to map?
      Are you trying to append/join Live Event records to static Canvas Data?
         Or are you just working with Live Events/Caliper?
      
      Or... are you simply trying to map any/all events and push them to SQL?
         Or are you processing them somewhere else?
      
      Since my previous reply I have collected over 13,000 asset_accessed Live Events, with 150 being announcements. In 0 of those messages can I find an announcement id to answer you question. But I am manually mapping JSON/Ruby Hash's to arrays and importing them to SQL.
      
      * I'd love to know what the JSON object for every message looks like as a schema if you can share.
      
      As you can see there aren't very many community resources for this topic. I've been working with Live Events for about 6 months now, and it's frustrating to say the least. With almost every event I end up emailing our CSM who forwards my questions to Engineers and I await replies. For instance, in looking at the discussion_topic_created event today, I am not getting a body field. I have been planning to release a small sample of code for the events I've mapped to hopefully get other's on board to help document these issues.
      1 person found this helpful
      Like • Show 0 Likes0
      Actions
      - Pooja Kosala @ Robert Carroll on Sep 24, 2018 3:04 PM
        Mark Correct
        Correct Answer
        I am afraid I can't share the data. But I am referring to canvas data and caliper data (Captured in the way described here). I am not referring to Live Events' data.
        Since canvas is closely associated with caliper, i was under the impression that Canvas folks have a mapping between above two datasets which can be used to relate data from canvas and caliper.
        We can discuss further if you are aware of caliper event logging for canvas and have worked on that data before.
        
        Thank you for your prompt responses!
        Like • Show 0 Likes0
        Actions
        Robert Carroll @ Pooja Kosala on Sep 24, 2018 3:33 PM
        Mark Correct
        Correct Answer
        Sorry for the confusion, I wasn't asking for any data, just the schema/fields for each event. Someone else mentioned to me before they pushed Live Events through Splunk and were able to get a footprint of every field across the variations of all the messages, but you're not mapping Live Events.
        
        How are you getting the OLAP data if you are not using Live Events?
        Are you self hosting?
        
        I don't think there are equal mappings between fields in Canvas Data and Caliper Events.
        
        Unfortunately, I am unaware of how Canvas processes data internally. I've attempted to use the source code to map certain Live Events before I had some of the provided documentation, but it's not easy to navigate.
        
        I will ask around and see if I can find anymore details.
        1 person found this helpful
        Like • Show 0 Likes0
        Actions
Pooja Kosala Sep 24, 2018 12:42 PM
Mark Correct
Correct Answer
Can someone please answer this question?
Like • Show 0 Likes0
Actions

How do we relate Canvas Data Portal to caliper events?

Outcomes

Related Content

Recommended Content