I am currently testing Alpaca’s historical stock quote/trade API (via the alpaca-py client).
Downloading a single day of AAPL quotes (or trades) takes roughly 2 mins with an average download speed of ~50 kB/s. My internet connection is certainly not the bottleneck here (200MBit/s). My guess is that Alpaca’s market data API is throttled.
I am wondering whether the download rate is increased as soon as one switches to an Alpaca Pro account? If so, what is the max download speed for historical stock quote/trade data for the Pro account?
@Rob Generally, the maximum transfer rate for a large number of trades or quotes is about 13,000/sec. Alpaca doesn’t limit this rate for either the Basic or the Alpaca Pro data plans.This is based upon some basic tests I’ve run with a cloud instance located in the same datacenter as the Alpaca servers (Google us-east4 data center in North Virginia). I’m speculating the connection rate is about 3G/sec but in any case doesn’t hit the public internet. I’m able to download a days worth of AAPL quotes (1,476,537 from 2023-08-08) in an average of 114 seconds over 10 trials. Not sure how many quotes or on what day you were testing, but fetching a days worth of quotes in about 2 minutes seems close to the max you would expect.
A big issue/bottleneck is the max trades/quotes returned by the API in a single call is 10,000. If the total number is more than this, then multiple calls need to be made. The overhead to create a connection and issue an API call (any call) is about 100ms independent of any data transfer. If one is fetching 1,476,537 quotes that will take 148 API calls or about 14 seconds just for the API calls. Another issue/bottleneck is the JSON HTTP protocol. That takes about 10x longer than WSS ‘streamed’ data and sometimes 100x longer than the raw bit rate (because of TCP and HTTP overhead and typical network retries etc). There is a bit on that here
The bottom line is the raw transfer rate isn’t typically the limiting factor for HTTP JSON transfers. I’d like to hear about other user experiences but from my tests about 13,000 quotes or trades / sec would be a typical download speed and doesn’t depend upon the choice of data plans.
Thanks a lot for your detailed answer! I understand now where the limitation of the download speed comes from. Unfortunately, this makes it impossible to download larger amounts of tick data via the API. AFAIR, I have seen someone else on these forums asking for a feature to download flat files containing historical data. I am gonna add a “+1” to that request.
