Trouble with S3

[shorowitz]

Running Canvas on our own infrastructure with local file storage. I have spun up a new test server and I am trying to get things to work with S3-based storage.

After modifying the bucket's CORS policy, I am able to upload and download fine. However, the background worker cannot generate thumbnails. Every time I upload, I get:

category: Aws::S3::Errors::SignatureDoesNotMatch
created at: 2020-02-17 14:28:42 -0700

attempts: 0
handler: --- !ruby/struct:Delayed::PerformableMethod object: !ruby/ActiveRecord:Attachment 17 method: :create_thumbnail_size args: - :thumb fail_cb: permanent_fail_cb:
hostname: canvas-s3-test
id: 1774
job_tag: Attachment#create_thumbnail_size
max_attempts: 1
pid: 1262
priority: 10
process_type: BackgroundJob
run_at: 2020-02-17 21:28:40 UTC
shard_id: 1
source:
strand: attachment_thumbnail_10000000000017_thumb
worker_name: canvas-s3-test:1262

I have tried this with an S3-compatible service at Linode and with Amazon's actual S3 service, so it's not an issue with a nonstandard S3 implementation. Obviously my key and secret are fine because the main photo goes up to the cloud and is visible in my course files, and can be viewed/downloaded from within canvas.

Any suggestions for how I can troubleshoot this further?

[robotcars]

Hi  @shorowitz 

I see that you also posted this question to the Canvas Dev Google Group, and that you already have some replies. You will probably get a better response there than here as more engineers and developers frequent that channel than the community. There is also the IRC channel if you want to chat. See more here, Canvas Dev & Friends - Instructure Tech Blog 

I'll tag Canvas Developers‌, just in case anyone has experience with this who might be hanging around.

[shorowitz]

I have one reply from another user there, which doesn't address the thumbnail issue. The other posts on that thread are me with additional data.  

I will try to pop into the IRC channel.