url is only "https://" in the csv

[xxie34]

Hi all,

I am seeing some data with url being https://  in many students' pageviews (the csv that you can download from Canvas). Does anyone have an explanation of what that is? They seem to be happening at various time. There is always an interaction_seconds for those rows of data.

Thanks!

Canoe

[robotcars]

Hi  @xxie34 ‌,

Did that URL get truncated in your post, or is that what you're seeing?

You've piqued my curiosity, because I can't find any information in Canvas Data Portal that show interaction_seconds is in Canvas Data. I know that page_view requests are in the Canvas Data Requests table, and that I've long hoped for the interaction_seconds parameter to be in that url string or row. Which Canvas Data file/table are you seeing it in, or was this from a generated report from within Canvas?

From my understanding, in most cases interaction_seconds refers to a tiny piece of JavaScript Canvas uses to count the number of seconds a user was on the page, then submits those seconds on the next page for collection and logging user activity within Canvas/Courses. These are in requests.web_application_controller = 'page_views', but there doesn't seem to be any useful information for us.

If you're seeing this in Canvas Data before any one else, that's cool. I think Canvas deploys pieces of updates early sometimes, or pushes an early commit in a bug fix. Maybe it will be out for the rest of us soon!

Some source references

canvas-lms/page_views.js at master · instructure/canvas-lms · GitHub 

canvas-lms/page_view_spec.rb at master · instructure/canvas-lms · GitHub 

canvas-lms/page_view.rb at master · instructure/canvas-lms · GitHub 

[xxie34]

Hi Robert,

Thanks for taking the time to read my question.

No the url "https://" is ALL i am seeing. That's why i am confused. 
I am getting the data by going to "People", locating the student I want, and click "user account details" on the right side menu. That takes me to the student's user page and it shows me a table of their most recent 300 pageviews. I can download that table by pressing the csv button. The reason why I mentioned interaction_seconds is because all the other "normal" looking data don't have that. 

Hopefully this makes sense. 

[robotcars]

Not Canvas Data, but definitely Page Views.

I followed your steps and I see the same pattern.

For 300 rows, my randomly chosen student has 12 rows with interaction_seconds, 3 of which have mostly empty cells, while the other 9 seem normal with URLs and column data.

I'm not sure if this is normal or new.

Beyond that I'm speculating as I don't have any regular experience with this data.

We need to ask someone who uses this data regularly. I don't typically pull the course reports because it's too cumbersome for the number of users we have. I think that  @James  and cesbrandt have both written scripts that used API endpoints for similar purposes, maybe they've seen this before.

[cesbrandt]

I don't personally use this report "often," but comparing the table on the user page to the table downloaded, every entry that comes up for "https://" as the URL registers the browser as "Unrecognized Browser." On the spreadsheet, the field is left empty, but no other information is given.

I'm afraid I can't offer anything more than that. I don't see anything surrounding these entries that indicates a pattern to it occurring.

If you really want to know what those entries are about, it would probably be best to submit a ticket to Instructure support.

My best guess? They're "ghost" entries caused by the complex dynamic loading of the various Canvas pages. It could happen with any page, as they basically all make multiple pull requests to build a single page, but it doesn't consistently happen.

[James]

When I first started looking into this, it seemed that it happened after the student had done something like submitting a quiz.

I pulled up on request and matched it to the corresponding entry in Canvas Data.

The one that got blanked out for that particular call was to the users user_dashboard.

However, more interesting was that about 4 seconds after that, there was a POST (includes PUT) call to /page_views/XXX where XXX is the page view ID of the page that got blanked. 

id	timestamp	url	method	micros	controller	action
9db46a00-0b66-4938-b8a1-bc949f3e4333	2018-09-18 13:32:50.850	/	GET	421121	users	user_dashboard
3ce68e5c-be06-4dd1-b10e-c61bfcb84ffd	2018-09-18 13:36:48.585	/page_views/9db46a00-0b66-4938-b8a1-bc949f3e4333	POST	23936	page_views	update

Following that the update happened from the page_views controller and the update action, I went there and found the update function, that has a comment "page view update happens in log_page_view after_action".

I didn't take time to track it down fully. The page_view_token that was supplied is mostly in the application_controller.rb file and most of it there deals with adding interaction seconds to something.

After searching for some more key parts and following some code, I ended up in app/models/page_view/pv4_client.rb. There is a section that tries to map the internal structure of a page view into the one needed for consumption. I think, without digging too much, it returns a value and then deletes it, so it maps things like sessionid into the CSV's session_id. There is a line (48) that says 

pv['url'] = "#{HostUrl.protocol}://#{pv.delete('vhost')}#{pv.delete('http_request')}"‍

Now, the https:// is going to show up even if there is no vhost or http_request in there. The only other time vhost shows up in the code is in the spec file to go along with that and that doesn't shed light.

It appears that somewhere, something is trying to strip out the page views, but not completely getting rid of it. The protocol https:// is still there, but the host and request are removed.

I'm leaning toward Canvas updating how long someone spent doing something, but it's not getting scrubbed completely from the system in the page views through the WebUI. In Canvas Data, it's showing up.

This is with a sample size of 1. It's probably more complex than that.

[xxie34]

Thanks James and Christopher. The reason why I am so "obsessed" with this issue is because one of the students  "lost" 17 minutes or so on a timed quiz. At 10:02 he viewed the quiz. At 10:19 he started the quiz (we know that by looking at Google Analytics. That's when the "....../quizzes/xxxxx/take" took place), and the row with 10:19  in Canvas page view report (downloaded from Canvas) is https://. But canvas seems to think that he started the quiz at 10:02, and in the quiz log, the first 16 minutes of his quiz is blank. That's why we want to know if some kind of glitch happened, which resulted in  1) Canvas gives this https:// url, and 2) thinks that he started the quiz at 10:02.

We have submitted a ticket to Canvas. I will let you all know if I hear back.