Updating front-end client using Alpaca's streamConn websocket

[widk]

I have a browser dashboard using Python & Flask. I wanted to update the data without manually refreshing the browser nor have javascript inefficiently make a request every second or minute. (Would Alpaca ban my ISP if made a request every second? ) If I can get the data from the streamConn websocket to push/pass the data to my browser dashboard that would be awesome. Flask would not work because it needs a request in order is issue a response. Flask-SocketIO or FastAPI maybe are solutions?

Is it possible to pass/push the data from streamConn to my dashboard browser? What libraries would you use? How would you do it?

Can you nest websockets in the following way?

# Alpaca websocket connection: data inflow
@conn.on(r'^T.*$')
async def on_data(conn, channel, data):
    data_dict = data._raw
    print('data processed:', data_dict)

    # FastAPI websocket creation: pass-through/ data outflow
    @app.websocket("/ws")
        async def websocket_endpoint(websocket: WebSocket):
        await websocket.accept()

        while True:
            payload = data_dict 
            await websocket.send_json(payload)


conn.run(['T.AAPL']])
uvicorn.run(app:main)

FastAPI RT-streaming

[Shlomik]

Hi,
you have built a server-client solution.
you have 2 options:

1. connect to the api from your server - python sdk
2. connect to the api from your client - js sdk (GitHub - alpacahq/alpaca-trade-api-js)

let’s focus and option 1 and leave option 2 to your architecture decisions.

if you connect to the Streaming api from your server then your server is now both a client and a server.

it’s a client to the alpaca servers
it’s a server to your js client

that means you open one websocket to the alpaca servers (StreamConn)
and you open a websocket server for your js client

now what you can do is when a message is received from StreamConn you need to pass it to your client.
what you did in your example is not good. you don’t want to wait for a connection from your browser every time you get a message from alpaca.
you want to hold 2 live connections that are never closed (1 to alapca with StreamConn 1 to your client) and streamline the messages you receive from one to the other.

[widk]

Hi Thanks for the reply,

I’m thinking maybe Python and FastAPI would be the server-client solution, maybe. I haven’t broken through the barrier of how to pass the data. What do you think about the following:

websocket_params = dict(key_id=aplaca_auth['key_id_paper'],
                    secret_key=aplaca_auth['secret_key_paper'],
                    base_url=base_url['alpaca_paper'],
                    data_url=data_url['polygon_stocks'],
                    data_stream=data_stream['polygon'])

conn = alpaca.stream2.StreamConn(**websocket_params)

js client connecting to server’s websocket?

<script>
    const ws = new Websocket("ws://localhost:8000/ws");
    ws.onmessage = function(event){
        const payload = JSON.parse(event.data);
        console.log(payload);
    }
</script>

server-side code:

# open server websocket for js client?
@app.websocket("/ws")
    # server for js client
    async def websocket_endpoint(websocket: WebSocket):
        await websocket.accept()

# listening for alpaca subscription messages
@conn.on(r'^T.*$')
async def on_data(conn, channel, data):
    # alpaca client
    data_dict = data._raw
    data_dict['foo']='grok'

#     while True:

    # "now what you can do is when a message is received
     # from StreamConn you need to pass it to your client."
    # "streamline the messages you receive 
     # from one to the other."

    # is this how you tap into StreamConn websocket feed
     # and pass the data to the js client?

    payload = data_dict
    # if this is not nested like in the previous wrong example,
     # then wouldn't the following line be out of this function's scope?
   await websocket.send_json(payload)


# "if you connect to the Streaming api from your server then 
 # your server is now both a client and a server."

# how does one connect to StreamConn from server?
# opening one websocket to the alpaca server?
# one live connection?
conn.run(['T.AAPL']) 

# running ASGI server?
# second live connection?
uvicorn.run("main:app", host="",port=8000,reload=True, debug=True) 

If you can provide any insight as to how to pass the data from alpaca websocket to the front-end websocket that would be amazing.

[Shlomik]

Just look at what I did here and copy the logic to your solution: GitHub - shlomikushchi/alpaca-proxy-agent

[widk]

@Shlomik Thanks for sharing your hardwork. However, it was over my head. It’s like level 10 and my knowledge is at like level 1.5 lol. Thanks for writing a doc string for send_response_to_client()!

So basically, to get data to my js client websocket connection from alpaca’s websocket message feed , I need to capture the data by sending it to a queue.Queue.put()? and retrieve the data from within my @app.websocket(’/ws’) endpoint via a queue.Queue.get()? and then send/serve it to my client?

This seems logical… I’ll give it a go.

However, how to match threads… this is still foggy, but I guess that’s just something to look forward too

[Shlomik]

yes, basically 2 threads one for each websocket connection.
then they communicate with each other using a Queue.