ERROR - Cannot connect to host api-gateway.instructure.com:443 ssl

[SatishAttili]

Hello,

I am trying to configure DAP client on a new Windows server, followed all the instructions as per the documentation.

When trying to fetch the list of tables using "dap list --namespace canvas", I got the below errors. Any help in resolving this error is really appreciated.

Thank you.

C:\Users\attilisatish>dap list --namespace canvas
2024-01-04 12:58:57,814 - ERROR - Cannot connect to host api-gateway.instructure.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)')]
Traceback (most recent call last):
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 992, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 1112, in create_connection
transport, protocol = await self._create_connection_transport(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 1145, in _create_connection_transport
await waiter
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\asyncio\sslproto.py", line 575, in _on_handshake_complete
raise handshake_exc
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\asyncio\sslproto.py", line 557, in _do_handshake
self._sslobj.do_handshake()
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 979, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\__main__.py", line 133, in console_entry
main()
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\__main__.py", line 125, in main
asyncio.run(dapCommand.execute(args))
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\commands\commands.py", line 31, in execute
executed = await super().execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\commands\base.py", line 49, in execute
if await subcommand.execute(args):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\commands\base.py", line 45, in execute
await self._execute_impl(args)
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\commands\commands.py", line 153, in _execute_impl
tables = await session.get_tables(args.namespace)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\api.py", line 398, in get_tables
table_list = await self._get(f"/dap/query/{namespace}/table", TableList)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\api.py", line 213, in _get
await self.authenticate()
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\api.py", line 363, in authenticate
properties = await self._post_auth_request(self._credentials.basic_credentials)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\dap\api.py", line 308, in _post_auth_request
async with self._session.post(
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\client.py", line 1187, in __aenter__
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\client.py", line 574, in _request
conn = await self._connector.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 544, in connect
proto = await self._create_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 911, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 1235, in _create_direct_connection
raise last_exc
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 1204, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\attilisatish\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 994, in _wrap_create_connection
raise ClientConnectorCertificateError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorCertificateError: Cannot connect to host api-gateway.instructure.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)')]

Thank you,

Satish

[SatishAttili]

Installing AWS CLI fixed the issue.

https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

[StevenWalls]

@SatishAttili  do we know how/why this fixed the issue?

[danlasota]

Did you ever get a response to this or find a solution?

[SatishAttili]

No, we found this on our own. 

[lmsstaff]

Just to piggy-back on this thread, rather than start a new one.....

Was there a change made in the past couple of days to the schema endpoint?  I was testing this week with dap client v1.0 with no issues for both schema and data downloads using this command line (for the schema, as an example):

dap --loglevel info --logfile d:/Integrations/automated/logs/EpsITCanvasDataMining/accounts_schema.log schema --table accounts --output-directory d:/Integrations/automated/download/EpsITCanvasDataMining/accounts/schema/

Yesterday, I upgraded to v.1.1 and now 2 issues arise:

1) It requires the namespace value (--namespace) to passed in the command.

2) The endpoint only resolves if the namespace ("Canvas") is passed all lowercase (even though the usage help text gives an example of "Canvas".  So only passing --namespace canvas will work as shown here:

bad requestbad requestgood request

So ya, kinda odd.  I am at a loss as to why this is happening.

[ColinMurtaugh]

The namespace parameter was required in 1.0.0 as well; could you have actually been using an older version of the client? 

The CLI examples that I see in the documentation all show a namespace parameter of "canvas". I'm not seeing where the help text says "Canvas", but it does need to be all lower-case. 

--Colin

[RobertSznyida]

@lmsstaff The behavior you explained was present already in v1.0.0 too, there was no change in v1.1.0 affecting namespaces. And it is the expected behavior; the --namespace argument is not optional anymore, and namespaces are case-sensitive, and currently all namespaces are lowercase: canvas, canvas_logs, catalog.

[RichardCarballo]

i fought insanely hard for like 4 hours on this topic Friday, the whole update AWS thing isn't even relevant as it has nothing to do with local certificates, just lucky for those that worked on.

What worked for me was updating PIP, and then updating the dap tool using the following command

To upgrade to the latest version of DAP, run:Copy

pip install --upgrade instructure-dap-client
pip install --upgrade "instructure-dap-client[postgresql,mysql]"

i used the first upgrade command and it began working again. 

[wjones20]

Sadly that didn't work for me. pip was already the latest version (24.3.1 for Python 3.13.1) but I ran the upgrade anyway. No change. I also tried upgrading AWS CLI, even though folks thought that was unrelated. I'm still getting:

ERROR - Cannot connect to host api-gateway.instructure.com:443 ssl:True [SSLCertVerificationError: (5, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1018)')]

I'm running on Windows 10. Any ideas are welcome. Thanks, Bill

[BrianWynder]

I as well am getting this issue.

It started on my server.  As part of troubleshooting I installed python 3.11.9 on a separate box, and installed dap, and upgraded everything (pip and dap and AWSCLI) - same issue.

I've tried several different python releases with none of them working.

[SajjadRahman]

Here is a solution I got that worked for me in our all 3 environments (DEV, TEST, and PROD).

In our case, our existing running environment was:

Python 3.10.10

Pip 22.x.x

Dap 0.3.10

After getting the error, I have found that dap client was yanked. So I have updated pip, dap client and postgresql:

python -m pip install --upgrade pip

pip install --upgrade instructure-dap-client

pip install --upgrade "instructure-dap-client[postgresql,mysql]"

Even though I was getting the same SSL certificate error.

Then I followed the below steps to update the certificate:

1. pip install --upgrade certify
2. I am in Windows, so I have updated my local certificate as below.
3. Using Git Bash, I got the certificate details of the Canvas API gateway:

   openssl s_client -connect api-gateway.instructure.com:443

4. Copy the certificate from

   -----BEGIN CERTIFICATE-----

   To

   -----END CERTIFICATE-----

   And save the file as .crt

5. Now update the local certificate.

6. Open Certificate Manager:
   1. Press Windows + R
   2. Type certlm.msc
   3. Click OK/Run as Administrator

7. Navigate to Certificates:

• Expand "Trusted Root Certification Authorities"
• Click on "Certificates"

8. To Import New Certificates:

• Right-click on "Certificates"
• Select "All Tasks" → "Import"
• The Certificate Import Wizard will open
• Click "Next"
• Browse to locate your certificate file (.cer, .crt, or .pem)
• Follow the wizard steps

  And the issue got resolved.

[BrianWynder]

Thank you very much.

For those of you who find this post in the future, I was able to download the certificate a bit more easily by going to ttps://api-gateway.instructure.com, clicking on the icon to the left of https:// in the URL, clicking "Connection is secure" and clicking to view the certificate - then i could export it from there directly in the correct format without having to open notepad or git bash.

Then I skipped to the certificate manager and imported in the certificate in trusted root certification authorities and it worked well...on my test installation using python 3.11

My server itself has an older version of python 3.10 and when I do the same steps it now has switched to saying "invalid client", so I suspect that I will also have to update the python version ... however that will potentially impact some of the other scripts running on that box so I'm going to save that process for the new year.

In the meantime, importing the certificate into trusted root certification authorities did indeed seem to be the magic sauce I was missing (after doing all the other updating by itself didn't fix the issue). 

Thank you again !