Surge in the latency for snapshot data

I’m usually getting Alpaca snapshot data within 50 ms. But at least 10% of my requests are taking more than 3,000 ms with some beyond 5,000 ms. Data is requested for only one stock every 5 seconds so there should be no throttling. Code is running on Google Cloud at their us-east4 data center North Virginia so low network latency is assured. Is this expected behavior?

Would love some feedback from community and Alpaca engineers.

Environment

Language
Node JS

Alpaca SDK Version
v2.16.1

Other Environment Details
Code is running on Google Cloud at their us-east4 data center North Virginia.

Problem

Summary
10% of my Alpaca requests for snapshot data are taking more than 3,000 ms.

Paper or Live Tradng?
Paper trading.

Example Code
const snapshot = await alpaca.getSnapshot(ticker);

Can somebody from Alpaca take a moment to reply?

Hi,
I took a quick look and I am not seeing that perf regression. Can you share details regarding your request and any logging?

Hi Raja,
Thanks for reply. Here is just some random timeframe from recent days.
Most requests are well under 50ms, but you can see there’re decent amount of outliers although all request conditions are the same.

Time for getting Alpaca data: 14 2022-09-20T14:31:49.165170Z
Time for getting Alpaca data: 17 2022-09-20T14:31:44.038588Z
Time for getting Alpaca data: 16 2022-09-20T14:31:38.932208Z
Time for getting Alpaca data: 21 2022-09-20T14:31:33.815300Z
Time for getting Alpaca data: 44 2022-09-20T14:31:28.680798Z
Time for getting Alpaca data: 3684 2022-09-20T14:31:23.562309Z
Time for getting Alpaca data: 19 2022-09-20T14:31:14.742857Z
Time for getting Alpaca data: 16 2022-09-20T14:31:09.624713Z
Time for getting Alpaca data: 2513 2022-09-20T14:31:04.528821Z
Time for getting Alpaca data: 419 2022-09-20T14:30:56.921130Z
Time for getting Alpaca data: 3604 2022-09-20T14:30:51.410161Z
Time for getting Alpaca data: 717 2022-09-20T14:30:42.744188Z
Time for getting Alpaca data: 20 2022-09-20T14:30:36.915065Z
Time for getting Alpaca data: 18 2022-09-20T14:30:31.819796Z
Time for getting Alpaca data: 843 2022-09-20T14:30:26.739984Z
Time for getting Alpaca data: 1852 2022-09-20T14:30:20.767874Z
Time for getting Alpaca data: 18 2022-09-20T14:30:13.821316Z
Time for getting Alpaca data: 2551 2022-09-20T14:30:08.705322Z
Time for getting Alpaca data: 19 2022-09-20T14:30:01.034434Z
Time for getting Alpaca data: 15 2022-09-20T14:29:55.944984Z
Time for getting Alpaca data: 17 2022-09-20T14:29:50.820957Z
Time for getting Alpaca data: 1260 2022-09-20T14:29:45.720865Z
Time for getting Alpaca data: 14 2022-09-20T14:29:39.393268Z
Time for getting Alpaca data: 20 2022-09-20T14:29:34.279049Z
Time for getting Alpaca data: 13 2022-09-20T14:29:29.181193Z
Time for getting Alpaca data: 17 2022-09-20T14:29:24.102413Z
Time for getting Alpaca data: 1318 2022-09-20T14:29:18.961971Z
Time for getting Alpaca data: 15 2022-09-20T14:29:12.577884Z
Time for getting Alpaca data: 14 2022-09-20T14:29:07.403748Z
Time for getting Alpaca data: 23 2022-09-20T14:29:02.305987Z
Time for getting Alpaca data: 16 2022-09-20T14:28:57.148288Z
Time for getting Alpaca data: 19 2022-09-20T14:28:52.075214Z
Time for getting Alpaca data: 13 2022-09-20T14:28:46.909545Z
Time for getting Alpaca data: 15 2022-09-20T14:28:41.812607Z
Time for getting Alpaca data: 24 2022-09-20T14:28:36.718400Z
Time for getting Alpaca data: 27 2022-09-20T14:28:31.634252Z
Time for getting Alpaca data: 21 2022-09-20T14:28:26.520674Z
Time for getting Alpaca data: 2374 2022-09-20T14:28:21.420488Z
Time for getting Alpaca data: 16 2022-09-20T14:28:13.902533Z
Time for getting Alpaca data: 14 2022-09-20T14:28:08.771Z
Time for getting Alpaca data: 1992 2022-09-20T14:28:03.642350Z
Time for getting Alpaca data: 15 2022-09-20T14:27:56.567163Z
Time for getting Alpaca data: 29 2022-09-20T14:27:51.445375Z
Time for getting Alpaca data: 14 2022-09-20T14:27:46.281050Z
Time for getting Alpaca data: 16 2022-09-20T14:27:41.135911Z
Time for getting Alpaca data: 16 2022-09-20T14:27:35.973504Z
Time for getting Alpaca data: 43 2022-09-20T14:27:30.752445Z
Time for getting Alpaca data: 148 2022-09-20T14:27:25.566499Z
Time for getting Alpaca data: 20 2022-09-20T14:27:20.351649Z
Time for getting Alpaca data: 34 2022-09-20T14:27:15.247401Z
Time for getting Alpaca data: 107 2022-09-20T14:27:10.131556Z
Time for getting Alpaca data: 23 2022-09-20T14:27:04.856939Z
Time for getting Alpaca data: 16 2022-09-20T14:26:59.711193Z
Time for getting Alpaca data: 14 2022-09-20T14:26:54.585904Z
Time for getting Alpaca data: 33 2022-09-20T14:26:49.372187Z
Time for getting Alpaca data: 14 2022-09-20T14:19:08.791010Z
Time for getting Alpaca data: 15 2022-09-20T14:19:03.706849Z
Time for getting Alpaca data: 15 2022-09-20T14:18:58.613593Z
Time for getting Alpaca data: 14 2022-09-20T14:18:53.541923Z
Time for getting Alpaca data: 13 2022-09-20T14:18:48.464649Z
Time for getting Alpaca data: 12 2022-09-20T14:18:43.340578Z
Time for getting Alpaca data: 20 2022-09-20T14:18:38.255631Z
Time for getting Alpaca data: 17 2022-09-20T14:18:33.159082Z
Time for getting Alpaca data: 19 2022-09-20T14:18:28.066462Z
Time for getting Alpaca data: 13 2022-09-20T14:18:22.987165Z
Time for getting Alpaca data: 16 2022-09-20T14:18:17.895707Z
Time for getting Alpaca data: 30 2022-09-20T14:18:12.804331Z
Time for getting Alpaca data: 25 2022-09-20T14:18:07.676268Z
Time for getting Alpaca data: 1253 2022-09-20T14:18:02.562324Z
Time for getting Alpaca data: 14 2022-09-20T14:17:56.236341Z
Time for getting Alpaca data: 29 2022-09-20T14:17:51.137160Z
Time for getting Alpaca data: 16 2022-09-20T14:17:46.031830Z
Time for getting Alpaca data: 24 2022-09-20T14:17:40.917162Z
Time for getting Alpaca data: 15 2022-09-20T14:17:35.821639Z
Time for getting Alpaca data: 14 2022-09-20T14:17:30.712565Z
Time for getting Alpaca data: 20 2022-09-20T14:17:25.596808Z
Time for getting Alpaca data: 14 2022-09-20T14:17:20.473866Z
Time for getting Alpaca data: 52 2022-09-20T14:17:15.363084Z
Time for getting Alpaca data: 24 2022-09-20T14:17:10.202536Z
Time for getting Alpaca data: 4175 2022-09-20T14:17:05.087340Z
Time for getting Alpaca data: 16 2022-09-20T14:06:03.941491Z
Time for getting Alpaca data: 23 2022-09-20T14:06:03.898763Z
Time for getting Alpaca data: 20 2022-09-20T14:05:58.789415Z
Time for getting Alpaca data: 25 2022-09-20T14:05:58.730153Z
Time for getting Alpaca data: 14 2022-09-20T14:05:53.626555Z
Time for getting Alpaca data: 23 2022-09-20T14:05:53.591127Z
Time for getting Alpaca data: 318 2022-09-20T14:05:48.500818Z
Time for getting Alpaca data: 17 2022-09-20T14:05:48.154972Z
Time for getting Alpaca data: 17 2022-09-20T14:05:43.074698Z
Time for getting Alpaca data: 17 2022-09-20T14:05:43.009793Z
Time for getting Alpaca data: 2525 2022-09-20T14:05:37.929996Z
Time for getting Alpaca data: 28 2022-09-20T14:05:35.354603Z
Time for getting Alpaca data: 29 2022-09-20T14:05:30.203296Z
Time for getting Alpaca data: 26 2022-09-20T14:05:30.134857Z
Time for getting Alpaca data: 19 2022-09-20T14:05:25.016358Z
Time for getting Alpaca data: 23 2022-09-20T14:05:24.946735Z
Time for getting Alpaca data: 551 2022-09-20T14:05:19.838280Z
Time for getting Alpaca data: 13 2022-09-20T14:05:19.241748Z
Time for getting Alpaca data: 22 2022-09-20T14:05:14.155978Z
Time for getting Alpaca data: 332 2022-09-20T14:05:14.093576Z
Time for getting Alpaca data: 2537 2022-09-20T14:05:08.701322Z
Time for getting Alpaca data: 14 2022-09-20T14:05:06.112629Z
Time for getting Alpaca data: 27 2022-09-20T14:05:00.994647Z
Time for getting Alpaca data: 591 2022-09-20T14:05:00.893931Z
Time for getting Alpaca data: 44 2022-09-20T14:04:55.228871Z
Time for getting Alpaca data: 50 2022-09-20T14:04:55.129256Z
Time for getting Alpaca data: 19 2022-09-20T14:04:49.986745Z
Time for getting Alpaca data: 39 2022-09-20T14:04:49.903975Z
Time for getting Alpaca data: 17 2022-09-20T13:44:42.518718Z
Time for getting Alpaca data: 632 2022-09-20T13:44:42.456463Z
Time for getting Alpaca data: 16 2022-09-20T13:44:36.717816Z
Time for getting Alpaca data: 16 2022-09-20T13:44:36.652020Z
Time for getting Alpaca data: 31 2022-09-20T13:44:31.523401Z
Time for getting Alpaca data: 17 2022-09-20T13:44:31.453089Z
Time for getting Alpaca data: 16 2022-09-20T13:44:26.339587Z
Time for getting Alpaca data: 12 2022-09-20T13:44:26.245549Z
Time for getting Alpaca data: 22 2022-09-20T13:44:21.146577Z
Time for getting Alpaca data: 19 2022-09-20T13:44:21.044216Z
Time for getting Alpaca data: 19 2022-09-20T13:44:15.915243Z
Time for getting Alpaca data: 23 2022-09-20T13:44:15.839049Z
Time for getting Alpaca data: 16 2022-09-20T13:44:10.730011Z
Time for getting Alpaca data: 63 2022-09-20T13:44:10.645645Z
Time for getting Alpaca data: 17 2022-09-20T13:44:05.503294Z
Time for getting Alpaca data: 18 2022-09-20T13:44:05.389331Z
Time for getting Alpaca data: 18 2022-09-20T13:44:00.269175Z
Time for getting Alpaca data: 21 2022-09-20T13:44:00.147742Z
Time for getting Alpaca data: 19 2022-09-20T13:43:55.014215Z
Time for getting Alpaca data: 17 2022-09-20T13:43:54.891242Z
Time for getting Alpaca data: 24 2022-09-20T13:43:49.747320Z
Time for getting Alpaca data: 26 2022-09-20T13:43:49.621485Z
Time for getting Alpaca data: 136 2022-09-20T13:43:44.454161Z
Time for getting Alpaca data: 44 2022-09-20T13:43:44.232399Z
Time for getting Alpaca data: 30 2022-09-20T13:43:38.999929Z
Time for getting Alpaca data: 18 2022-09-20T13:37:12.203729Z
Time for getting Alpaca data: 27 2022-09-20T13:37:07.041315Z
Time for getting Alpaca data: 24 2022-09-20T13:37:01.912091Z
Time for getting Alpaca data: 88 2022-09-20T13:36:56.797486Z
Time for getting Alpaca data: 23 2022-09-20T13:36:51.611981Z
Time for getting Alpaca data: 21 2022-09-20T13:36:46.490802Z
Time for getting Alpaca data: 83 2022-09-20T13:36:41.352074Z
Time for getting Alpaca data: 189 2022-09-20T13:36:36.177279Z
Time for getting Alpaca data: 605 2022-09-20T13:36:30.875237Z
Time for getting Alpaca data: 17 2022-09-20T13:36:25.153221Z
Time for getting Alpaca data: 551 2022-09-20T13:36:20.013251Z
Time for getting Alpaca data: 14 2022-09-20T13:36:14.298618Z
Time for getting Alpaca data: 18 2022-09-20T13:36:09.156926Z
Time for getting Alpaca data: 1120 2022-09-20T13:36:04.018646Z
Time for getting Alpaca data: 17 2022-09-20T13:35:57.784621Z
Time for getting Alpaca data: 13 2022-09-20T13:35:52.633834Z
Time for getting Alpaca data: 21 2022-09-20T13:35:47.499225Z
Time for getting Alpaca data: 27 2022-09-20T13:35:42.351504Z
Time for getting Alpaca data: 1463 2022-09-20T13:35:37.167040Z
Time for getting Alpaca data: 43 2022-09-20T13:35:30.558207Z
Time for getting Alpaca data: 85 2022-09-20T13:31:18.590060Z
Time for getting Alpaca data: 41 2022-09-20T13:31:13.366353Z
Time for getting Alpaca data: 17 2022-09-20T13:31:08.171063Z
Time for getting Alpaca data: 30 2022-09-20T13:31:02.985524Z
Time for getting Alpaca data: 97 2022-09-20T13:30:57.786118Z