I’m running multiple strategies which need to connect to the Market Data websocket. Is there a subscription plan that allows to open multiple connections using the same account?
In the python repository (GitHub - alpacahq/alpaca-trade-api-python: Python client for Alpaca's trade API) they suggest to use this other project GitHub - camelpac/alpaca-proxy-agent, but it’s outdated and it doesn’t work with the Market Data API v2, is there other option?
Hey @gomezdanielg - The unlimited plan will allow for multiple calls, but a single websocket connection.
There’s some more details here: Real-time data - Documentation | Alpaca
Is this really correct, @Mallory ? I have the unlimited plan, but when I connect to a second stream, my handlers for trades/quotes/bars are never called (I’m using the Python API).
Can any other Alpaca user with the unlimited plan let us know if you can use multiple connections?
Hi @alpalpalpa - Yes, its correct. Are you getting any particular errors when you try to make that second connection? What is returned?
I have the unlimited plan and I’m experiencing the same issue as @alpalpalpa. I’m not getting any particular error or log message that allow me to address the issue.
Just as @gomezdanielg has said, there is no error or log message. The handlers are simply never called.
To verify this issue, you can run the following example script from Alpaca’s GitHub page in two different terminal windows and observe that it will only properly work from the first window: alpaca-trade-api-python/avoid_server_connection_errors.py at master · alpacahq/alpaca-trade-api-python · GitHub
And to further clarify, it looks like you can have a ‘sip’ connection properly working at the same time as an ‘iex’ connection, but you can’t have multiple of either type working at the same time.
I’m having the same issue. It would be great to run multiple alphas simultaneously on different servers rather than attempting to parse trades from different symbols on a single connection.
Same happens here with golang. Upgraded my account, but second connection still doesn’t work.
INFO datav2stream: connecting to wss://stream.data.alpaca.markets/v2/sip, attempt 1/20 …
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 500ms, attempt 2/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 1s, attempt 3/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 1.5s, attempt 4/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 2s, attempt 5/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 2.5s, attempt 6/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 3s, attempt 7/16
INFO datav2stream: connecting to wss://stream.data.alpaca.markets/v2/sip, attempt 2/20 …
INFO datav2stream: established connection
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 500ms, attempt 2/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 1s, attempt 3/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 1.5s, attempt 4/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 2s, attempt 5/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 2.5s, attempt 6/16
INFO datav2stream: auth error: connection limit exceeded
INFO datav2stream: retrying auth in 3s, attempt 7/16
INFO datav2stream: connecting to wss://stream.data.alpaca.markets/v2/sip, attempt 3/20 …
Seeing the same issue as others reported.
There seems to be a limit of ~3500 stocks per websocket client. If you try to open a second connection and stream batches of tickers, the second connection returns an error like @alpalpalpa reported.
It does not seem possible to stream all nasdaq tickers using a single “unlimited” account. If this is intentional, perhaps add a feature to pay another $x dollars for more websocket connections in the same account?
This is using an SIP unlimited account. Tried on both paper-api and api endpoints.
Examples:
...
tickers = [el.symbol for el in api.list_assets(status='active')]
...
# Try to subscribe to 4k tickers
@stream.on_bar(*tickers[0:4000])
async def _(bar):
...
Here’s the error you get:
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=<Opcode.CLOSE: 8>, data=b'\x03\xf1read limited at 16385 bytes', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - event = connection_lost(None)
DEBUG:websockets.protocol:client - state = CLOSED
DEBUG:websockets.protocol:client x code = 1009, reason = read limited at 16385 bytes
WARNING:alpaca_trade_api.stream:websocket error, restarting connection: code = 1009 (message too big), reason = read limited at 16385 bytes
DEBUG:websockets.protocol:client x closing TCP connection
If you try to open a second connection and stream batches of tickers in two processes, the second process never initiates. Here’s an output from debug mode using python sdk:
DEBUG:asyncio:Using selector: KqueueSelector
INFO:alpaca_trade_api.stream:started data stream
DEBUG:websockets.protocol:client - state = CONNECTING
DEBUG:websockets.protocol:client - event = connection_made(<asyncio.sslproto._SSLProtocolTransport object at 0x1239abbe0>)
DEBUG:websockets.server:client > GET /v2/sip HTTP/1.1
DEBUG:websockets.server:client > Headers([('Host', 'stream.data.alpaca.markets'), ('Upgrade', 'websocket'), ('Connection', 'Upgrade'), ('Sec-WebSocket-Key', '************'), ('Sec-WebSocket-Version', '13'), ('Sec-WebSocket-Extensions', 'permessage-deflate; client_max_window_bits'), ('Content-Type', 'application/msgpack'), ('User-Agent', 'Python/3.9 websockets/9.1')])
DEBUG:websockets.protocol:client - event = data_received(<276 bytes>)
DEBUG:websockets.server:client < HTTP/1.1 101 Switching Protocols
DEBUG:websockets.server:client < Headers([('Date', 'Thu, 23 Sep 2021 18:25:47 GMT'), ('Connection', 'upgrade'), ('Sec-Websocket-Accept', '******'), ('Sec-Websocket-Extensions', 'permessage-deflate'), ('Upgrade', 'websocket'), ('Strict-Transport-Security', 'max-age=15724800; includeSubDomains')])
DEBUG:websockets.protocol:client - state = OPEN
DEBUG:websockets.protocol:client - event = data_received(<30 bytes>)
DEBUG:websockets.protocol:client < Frame(fin=False, opcode=<Opcode.BINARY: 2>, data=b'\x91\x82\xa1T\xa7success\xa3msg\xa9connected', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=<Opcode.CONT: 0>, data=b'', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=<Opcode.BINARY: 2>, data=b'\x83\xa6action\xa4auth\xa3key\************\xa6secret\xd9********', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - event = data_received(<54 bytes>)
DEBUG:websockets.protocol:client < Frame(fin=False, opcode=<Opcode.BINARY: 2>, data=b'\x91\x83\xa1T\xa5error\xa4code\xce\x00\x00\x01\x96\xa3msg\xb9connection limit exceeded', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=<Opcode.CONT: 0>, data=b'', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - event = data_received(<3 bytes>)
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=<Opcode.PING: 9>, data=b'1', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - received ping, sending pong: 31
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=<Opcode.PONG: 10>, data=b'1', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - event = data_received(<45 bytes>)
DEBUG:websockets.protocol:client < Frame(fin=False, opcode=<Opcode.BINARY: 2>, data=b'\x91\x83\xa1T\xa5error\xa4code\xce\x00\x00\x01\x94\xa3msg\xacauth timeout', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=<Opcode.CONT: 0>, data=b'', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=<Opcode.CLOSE: 8>, data=b'\x03\xe8', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - state = CLOSING
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=<Opcode.CLOSE: 8>, data=b'\x03\xe8', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - event = connection_lost(None)
DEBUG:websockets.protocol:client - state = CLOSED
DEBUG:websockets.protocol:client x code = 1000, reason = [no reason]
DEBUG:websockets.protocol:client x closing TCP connection
I also looked for a way to create multiple keys as a workaround, but it seems like there is just a single key per account.
Maybe I missed something?
Is there a way to stream more stocks or establish a second connection with the ‘unlimited’ account?
Thanks
FYI for others, bars accept an Asterisk. Doesn’t help for trades/quotes limits, but it looks like you can stream all the bars.
From the docs:
You can subscribe to trades, quotes and bars of a particular symbol (or * for every symbol in the case of bars).
Glad to see Alpaca raised some a healthy round…
From a user perspective – It’d be nice to see someone from Alpaca moderate/answer these forums, or just get rid of them.
Did you get this to work? I’m not sure this is still a feature
I compared the Alpaca data stream on a per-ticker level to TDA and Polygon…
It wasn’t close. Volume and pricing is consistently off in Alpaca for the same ticker, periods, time… Haven’t considered using Alpaca since.
Reading up on alpaca – Looks like they lost polygon, but raised 50m. I’m guessing they are tooling up to rebuild from scratch, and just trying to avoid the bad press in the meantime. Seems dangerous.
Big red flag to anyone using Alpaca data to trade real money with… Don’t want to compromise any data agreements, but compare alpaca data to another source.
Maybe this is resolved, but as of the end of September 2021, this was the case.
Maybe someone from Alpaca could clarify what’s going on with the data feed and why it would not match other vendors if it’s direct SIP? Is it usable currently? Are new versions pending a rollout?
Good luck.