Setup failing when run as multiple processes

Alpaca Trading
1

Hi,

All goes well when I run my Python trading program as one process. However, things go awry, when I run the same setup as multiple processes internally (that is parallelise it to ensure more efficient execution). This is what exactly happens:

1.This is the error I get when I run the program as 2 processes (I am running with only 2 processes for testing):

ge
client.get_option_latest_quote(
File “/root/miniconda3/lib/python3.12/site-packages/alpaca/data/historical/option.py”, line 126, in get_option_latest_quote
raw_latest_quotes = self._get_marketdata(
^^^^^^^^^^^^^^^^^^^^^
File “/root/miniconda3/lib/python3.12/site-packages/alpaca/common/rest.py”, line 393, in _get_marketdata
response = self.get(path=path, data=params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/root/miniconda3/lib/python3.12/site-packages/alpaca/common/rest.py”, line 225, in get
return self._request(“GET”, path, data, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/root/miniconda3/lib/python3.12/site-packages/alpaca/common/rest.py”, line 131, in _request
return self._one_request(method, url, opts, retry)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/root/miniconda3/lib/python3.12/site-packages/alpaca/common/rest.py”, line 195, in _one_request
response = self._session.request(method, url, **opts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/root/miniconda3/lib/python3.12/site-packages/requests/sessions.py”, line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/root/miniconda3/lib/python3.12/site-packages/requests/sessions.py”, line 746, in send
r.content
File “/root/miniconda3/lib/python3.12/site-packages/requests/models.py”, line 902, in content
self._content = b"“.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b”"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/root/miniconda3/lib/python3.12/site-packages/requests/models.py”, line 828, in generate
raise RequestsSSLError(e)
requests.exceptions.SSLError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2580)

NOTE: As one can see, this happens when we call client.get_option_latest_quote. So is this an issue, when it used with multiple processes?

Observation: While re running repeatedly, the script continues to run even after throwing this exception. So can this be ignored? Feel free to let me know if you need more information.

Thanks

2

An update from my end:

This looks to be happening when the trade client and OptionHistoricalDataClient was shared across multiple processes. I have not seen this error after they were created separately for each process.

Thanks

3

@sarahvignale You figured it out!

The client objects (eg the OptionHistoricalDataClient or the StockHistoricalDataClient) store state information about the TCP connection and a few other things. The client object can only store one state at a time, which works fine in a sequential synchronous process flow. However, if that object is used asynchronously (ie trying to manage two or more concurrent connections) it gets confused and data gets dropped etc.

The fix? If you have asynchronous functions, instantiate any clients locally inside those functions. Do not use a global client which could potentially be used simultaneously by another function.

Don’t do this

data_client = StockHistoricalDataClient(api_key=ALPACA_API_KEY_ID, secret_key=ALPACA_API_SECRET_KEY)

async def run_asynch_1:
  data_client.get_stock_bars(bars_request_params)

async def run_asynch_2:
  data_client.get_stock_bars(bars_request_params)

Do this instead

async def run_asynch_1:
  data_client = StockHistoricalDataClient(api_key=ALPACA_API_KEY_ID, secret_key=ALPACA_API_SECRET_KEY)
  data_client.get_stock_bars(bars_request_params)


async def run_asynch_2:
  data_client = StockHistoricalDataClient(api_key=ALPACA_API_KEY_ID, secret_key=ALPACA_API_SECRET_KEY)
  data_client.get_stock_bars(bars_request_params)

That way each client maintains state for each separate process.

1 Like