Playing nice when calling APIs concurrently

Community Participant

I wanted to ask a question about best ways to play nice when running concurrent API calls. I am aware of the Canvas API's Rate Throttling Model and am looking at the most effective way to scale API calls to avoid exceeding quotas. I am using a resource pool of open HTTP connections (currently only using GET operations). However I make no attempt to introduce a delay between successive HTTP calls. When my resource pool is fully loaded, all of the HTTP calls will be  made in very rapid succession (then, as the process runs there is a natural delay between calls, introduced by different return times, controlling when the pool is refreshed). The resource pool is keeping me from exceeding the quota, but I still need to look at smart ways of dynamically scaling the size of this pool to account for variability on a production system.  As I prepare to move from testing on a non-production instance of Canvas to production, considering if my approach is production ready (just because I can make a bunch of http calls rapidly,  does not mean I should..) . Also wondering if I should avoid running a concurrent process during peak use hours - is any process that comes close to hitting the rate throttle limit going to noticeably degrade system performance for other users. Would be interested in others input on these considerations.