Request hostname changing during LTI 1.3 launch sequence

[marcspringshare]

Hi all,

We are developing a 1.3 version of our existing LTI tool for Canvas and other LMS, but implementation is proving difficult.

We are running the 2020-09-09 release on a Bitnami provided AWS EC2 instance, and we access Canvas via the EC2 hostname provided by AWS.  When I launch the tool in an attempt to add an new link to the MOdules section of Canvas I am taken through the following sequence:

GET http://ec2-1-2-3-4.compute-1.amazonaws.com/courses/1/external_tools/19/resource_selection 
placement=resource_selection
secure_params=undefined
context_module_id=1

POST https://my.tool.domain/lti1p3/login/981 
iss=https://canvas.instructure.com
login_hint=b582bed226a0c6e204b6b811cf8a37e7b2f8b0ce
client_id=10000000000008
target_link_uri=https://my.tool.domain/lti1p3/launch/981
lti_message_hint=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ2ZXJpZmllciI6ImRkZDExM2ViZjUzNDMxNWI3YjJiNWNiZDFjM2NlMDg2ODkyNTZjYzgxZTZlMGQwNjVkNzYwMzhlNjMwMThjZGIxNmY5MDFiNmI3NzY3OGQ0YmFhOGM0YmZlYWU2NzNlYTNjNjk1NjAyYmJiZmQ4Y2RkNjAyNjJhMGJlMTMyYzVmIiwiY2FudmFzX2RvbWFpbiI6IjU0LjgzLjIzMi4zOCIsImNvbnRleHRfdHlwZSI6IkNvdXJzZSIsImNvbnRleHRfaWQiOjEwMDAwMDAwMDAwMDAxLCJleHAiOjE2MDUzMDcwNTZ9.Rc-81tY4AquzAHvSnZhw1eiz28M2GhAyl-FkMRGDvtg
canvas_region=not_configured

GET http://ec2-1-2-3-4.compute-1.amazonaws.com/api/lti/authorize_redirect 
scope=openid
response_type=id_token
response_mode=form_post
prompt=none
client_id=10000000000008
redirect_uri=https%3A%2F%2Fmy.tool.domain%2Flti1p3%2Flaunch%2F981
state=state-5faf09853db303_54805647
nonce=nonce-5faf09853db4e6.56249148
login_hint=b582bed226a0c6e204b6b811cf8a37e7b2f8b0ce
lti_message_hint=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ2ZXJpZmllciI6ImRkZDExM2ViZjUzNDMxNWI3YjJiNWNiZDFjM2NlMDg2ODkyNTZjYzgxZTZlMGQwNjVkNzYwMzhlNjMwMThjZGIxNmY5MDFiNmI3NzY3OGQ0YmFhOGM0YmZlYWU2NzNlYTNjNjk1NjAyYmJiZmQ4Y2RkNjAyNjJhMGJlMTMyYzVmIiwiY2FudmFzX2RvbWFpbiI6IjU0LjgzLjIzMi4zOCIsImNvbnRleHRfdHlwZSI6IkNvdXJzZSIsImNvbnRleHRfaWQiOjEwMDAwMDAwMDAwMDAxLCJleHAiOjE2MDUzMDcwNTZ9.Rc-81tY4AquzAHvSnZhw1eiz28M2GhAyl-FkMRGDvtg

GET http://1.2.3.4/api/lti/authorize 
client_id=10000000000008
login_hint=b582bed226a0c6e204b6b811cf8a37e7b2f8b0ce
lti_message_hint=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ2ZXJpZmllciI6ImRkZDExM2ViZjUzNDMxNWI3YjJiNWNiZDFjM2NlMDg2ODkyNTZjYzgxZTZlMGQwNjVkNzYwMzhlNjMwMThjZGIxNmY5MDFiNmI3NzY3OGQ0YmFhOGM0YmZlYWU2NzNlYTNjNjk1NjAyYmJiZmQ4Y2RkNjAyNjJhMGJlMTMyYzVmIiwiY2FudmFzX2RvbWFpbiI6IjU0LjgzLjIzMi4zOCIsImNvbnRleHRfdHlwZSI6IkNvdXJzZSIsImNvbnRleHRfaWQiOjEwMDAwMDAwMDAwMDAxLCJleHAiOjE2MDUzMDcwNTZ9.Rc-81tY4AquzAHvSnZhw1eiz28M2GhAyl-FkMRGDvtg
nonce=nonce-5faf09853db4e6.56249148
prompt=none
redirect_uri=https%3A%2F%2Fmy.tool.domain%2Flti1p3%2Flaunch%2F981
response_mode=form_post
response_type=id_token
scope=openid
state=state-5faf09853db303_54805647

At this point I am shown the following error message in Firefox.  Clicking the "Open site..." button generates this final request:

POST https://my.tool.domain/lti1p3/launch/981 
utf8=✓
authenticity_token=mlnx+Ut8PiFLeFLmcREqCvw1C5c/p6jwo186h/XTfbzQO7+JeShVaS45FpFFcG9NynE821jk3rj6CHzTgaAO3w==
error=login_required
error_description=Must have an active user session
state=state-5faf09853db303_54805647

So...a lot going on there.  The first thing that jumps out at me is that when going from /api/lti/authorize_redirect to /api/lti/authorize on the Canvas end, the domain changes from the machine hostname to its IP address.  I don't know if that is problematic, but it is a question for sure.

[svickers2]

One suggestion: check that your domain is properly set in the config/domain.yml file.

[marcspringshare]

That was helpful, thanks!

I used the LetsEncrypt cert creation tool to generate a certificate for dev-canvas-2020-09-09.xxx.com and verified that same domain is in the YML file you referenced.  That resolved the issue of Canvas switching over to an IP address during the LTI launch sequence.  And it also - along with the cert configuration I'm assuming - got rid of the Firefox "open in a new window" error.  But of course, that only gets me to the next error!

GET https://dev-canvas-2020-09-09.xxx.com/courses/1/external_tools/19/resource_selection 
placement=resource_selection
secure_params=undefined
context_module_id=1

POST https://my.tool.domain/lti1p3/login/981 
iss=https://canvas.instructure.com
login_hint=b582bed226a0c6e204b6b811cf8a37e7b2f8b0ce
client_id=10000000000008
target_link_uri=https://my.tool.domain/lti1p3/launch/981
lti_message_hint=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ2ZXJpZmllciI6IjY2MDU2YjZhMzhjYzdkM2M3ZmU2MGExZjllMGM1ZWUwNjQ4YWUwODJjNTMwYzFkZmJlNTg5M2U4YWNlOWYzMjYyMGNlOWJkYTBiNTg2ZmMwNDg1YmI5ZGZkZTljMzVkMzRiODcxZWRjYTZjZGI5YTc0MGEyYjg1ZTM4ZmFmYThjIiwiY2FudmFzX2RvbWFpbiI6ImRldi1jYW52YXMtMjAyMC0wOS0wOS5zcHJpbmd5YXdzLmNvbSIsImNvbnRleHRfdHlwZSI6IkNvdXJzZSIsImNvbnRleHRfaWQiOjEwMDAwMDAwMDAwMDAxLCJleHAiOjE2MDU1NDQxNDl9.Uqb1fM2RXw7g3qG4u8qR1YJQN5GZpMLGc86aYM-1nS8
canvas_region=not_configured

GET https://dev-canvas-2020-09-09.xxx.com/api/lti/authorize_redirect 
https://dev-canvas-2020-09-09.xxx.com/api/lti/authorize_redirect?scope=openid&response_type=id_token&response_mode=form_post&prompt=none&client_id=10000000000008&redirect_uri=https%3A%2F%2Fmy.tool.domain%2Flti1p3%2Flaunch%2F981&state=state-5fb2a7ab35d323_32533045&nonce=nonce-5fb2a7ab35d633.50088451&login_hint=b582bed226a0c6e204b6b811cf8a37e7b2f8b0ce&lti_message_hint=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ2ZXJpZmllciI6IjY2MDU2YjZhMzhjYzdkM2M3ZmU2MGExZjllMGM1ZWUwNjQ4YWUwODJjNTMwYzFkZmJlNTg5M2U4YWNlOWYzMjYyMGNlOWJkYTBiNTg2ZmMwNDg1YmI5ZGZkZTljMzVkMzRiODcxZWRjYTZjZGI5YTc0MGEyYjg1ZTM4ZmFmYThjIiwiY2FudmFzX2RvbWFpbiI6ImRldi1jYW52YXMtMjAyMC0wOS0wOS5zcHJpbmd5YXdzLmNvbSIsImNvbnRleHRfdHlwZSI6IkNvdXJzZSIsImNvbnRleHRfaWQiOjEwMDAwMDAwMDAwMDAxLCJleHAiOjE2MDU1NDQxNDl9.Uqb1fM2RXw7g3qG4u8qR1YJQN5GZpMLGc86aYM-1nS8

GET https://dev-canvas-2020-09-09.xxx.com/api/lti/authorize 
client_id=10000000000008
login_hint=b582bed226a0c6e204b6b811cf8a37e7b2f8b0ce
lti_message_hint=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ2ZXJpZmllciI6IjY2MDU2YjZhMzhjYzdkM2M3ZmU2MGExZjllMGM1ZWUwNjQ4YWUwODJjNTMwYzFkZmJlNTg5M2U4YWNlOWYzMjYyMGNlOWJkYTBiNTg2ZmMwNDg1YmI5ZGZkZTljMzVkMzRiODcxZWRjYTZjZGI5YTc0MGEyYjg1ZTM4ZmFmYThjIiwiY2FudmFzX2RvbWFpbiI6ImRldi1jYW52YXMtMjAyMC0wOS0wOS5zcHJpbmd5YXdzLmNvbSIsImNvbnRleHRfdHlwZSI6IkNvdXJzZSIsImNvbnRleHRfaWQiOjEwMDAwMDAwMDAwMDAxLCJleHAiOjE2MDU1NDQxNDl9.Uqb1fM2RXw7g3qG4u8qR1YJQN5GZpMLGc86aYM-1nS8
nonce=nonce-5fb2a7ab35d633.50088451
prompt=none
redirect_uri=https%3A%2F%2Fmy.tool.domain%2Flti1p3%2Flaunch%2F981
response_mode=form_post
response_type=id_token
scope=openid
state=state-5fb2a7ab35d323_32533045

This last GET to the /api/lti/authorize endpoint returns a 500 with the following response body:
while(1);{"errors":[{"message":"An error occurred.","error_code":"internal_server_error"}],"error_report_id":142}

I thought I had come across another post that explained how to view error reports when logged into Canvas as an admin, but I can't find that any longer.  Things are definitely going in the right direction, but I'm now just running into the next brick wall.

[svickers2]

Check the database for a more detailed log of the error; I think there is a table named error_reports.  My best guess as to the likely cause is that Canvas is not finding a private key defined so it cannot sign the id_token to be sent to the tool.  Check the config/dynamic_settings.yml file; if you're running the production instance of Canvas, make sure the private key is defined for the production section.  But that may not be the cause at all; let us know what error appears in the log.

[svickers2]

PS One quick way to check if my best guess is anywhere near the mark would be to call the .../api/lti/security/jwks endpoint.  If it returns public keys then all is probably fine with your private key.

[marcspringshare]

Looks like you can view the details of an error report by visiting /error_reports/{error_report_id} off your Canvas domain.

I believe I finally find a solution here:
https://community.canvaslms.com/t5/Developers-Group/Canvas-LTI-1-3-Error-Unknown-Key-Type/m-p/390285...

The TLDR is that the Bitnami installation is missing the dynamic_settings.yml file, so I created one by copying the example file:
sudo cp /opt/bitnami/apps/canvaslms/htdocs/config/dynamic_settings.yml.example /opt/bitnami/apps/canvaslms/htdocs/config/dynamic_settings.yml

And then I added a "production" section to dynamic_settings.yml, containing the following:
production:
  store:
    canvas:
      lti-keys:
        jwk-past.json: {copy entire json object from the development section at the top of the file}
        jwk-present.json: {copy entire json object from the development section at the top of the file}
        jwk-future.json: {copy entire json object from the development section at the top of the file}

Restarting the Bitnami services after making that change seems to have resolved things:
sudo /opt/bitnami/ctlscript.sh restart

Hope that saves someone else some time!

[marcspringshare]

Looks like you can view the details of an error report by visiting /error_reports/{error_report_id} off your Canvas domain.

I believe I finally find a solution here:
https://community.canvaslms.com/t5/Developers-Group/Canvas-LTI-1-3-Error-Unknown-Key-Type/m-p/390285...

The TLDR is that the Bitnami installation is missing the dynamic_settings.yml file, so I created one by copying the example file:
sudo cp /opt/bitnami/apps/canvaslms/htdocs/config/dynamic_settings.yml.example /opt/bitnami/apps/canvaslms/htdocs/config/dynamic_settings.yml

And then I added a "production" section to dynamic_settings.yml, containing the following:
production:
  store:
    canvas:
      lti-keys:
        jwk-past.json: {copy entire json object from the development section at the top of the file}
        jwk-present.json: {copy entire json object from the development section at the top of the file}
        jwk-future.json: {copy entire json object from the development section at the top of the file}

Restarting the Bitnami services after making that change seems to have resolved things:
sudo /opt/bitnami/ctlscript.sh restart

Hope that saves someone else some time!