Hi all,
Today I used the minute bar real time data and missed 3 ticks of bars data for GOOGL stock.
The missing data was on “2022-03-30” between “17:19:00” until “17:22:00”.
I found the missing data in the historical API few hours later, any suggestions ?
Maybe if there is no change in those minutes there is no minute bar tick ?
Really appreciate your answers,
Thanks !
Hi, I’m missing minute bars for several symbols also. If I sum volume data it is way under reported volume for almost every symbol.
John
Thanks, but it’s not related to the missing bars.
Are you experienced missing bars on real time data also ?
yes, I sum volume from minute bars throughout the day. Missing bars mean the sum of volume becomes inaccurate as the day progresses.
John
So how do you handle missing bars ?
There is no work around. Most of the symbols or ETFs I’m using are very liquid. They should not miss minute bars. That means missing data.
But what does it means when a minute bar data is missing ?
Does it mean that no trades where done in those minutes ?
Does it mean that the stock / ETF price didn’t change during the missing minute ?
Thanks again for your contribution
I don’t know why the minute bars are missing. I was hoping someone from Alpaca could explain. I can follow minute bars online almost anywhere. If you use an online broker, mine is Schwab, there are no minute bars missing. Run AAPL…there are many trades every second. Why would you miss minute bars?
@Tekton @Omer1 The first and foremost cause of ‘missing’ bars is simply that bars aren’t generated if there aren’t any eligible trades for the minute. No trades - no bar. That was the case for GOOGL between 17:19:00 - 17:22:00 on 2022-03-30.
Below is a list of all trades between 17:19:00 - 17:23:00. Notice all the trades have an “I” condition which means an odd lot trade under 100 shares. Those trades aren’t included in bar prices and do not trigger the creation of a bar.
Here is the full list of conditions which would exclude a trade from being included in a bar.
| Feed | Tape | Condition | Description |
|---|---|---|---|
| CTS | AB | B | Average Price Trade |
| UTDF | C | W | Average Price Trade |
| CTS/UTDF | ABC | 4 | Derivatively Priced |
| CTS/UTDF | ABC | 7 | Qualified Contingent Trade (“QCT”) |
| CTS/UTDF | ABC | 9 | Corrected Consolidated Close (per listing market) |
| CTS/UTDF | ABC | C | Cash Sale |
| CTS/UTDF | ABC | G | Bunched Sold Trade |
| CTS/UTDF | ABC | H | Price Variation Trade |
| CTS/UTDF | ABC | I | Odd Lot Trade |
| CTS/UTDF | ABC | M | Market Center Official Close |
| CTS/UTDF | ABC | N | Next Day |
| CTS/UTDF | ABC | P | Prior Reference Price |
| CTS/UTDF | ABC | Q | Market Center Official Open |
| CTS/UTDF | ABC | R | Seller |
| CTS/UTDF | ABC | T | Form T |
| CTS/UTDF | ABC | U | Extended Trading Hours (Sold Out of Sequence) |
| CTS/UTDF | ABC | V | Contingent Trade |
| CTS/UTDF | ABC | Z | Sold (out of Sequence) |
Volume is a dilemma. Most trades, including most of those excluded above, are included in volume calculations. If there is a bar, then the trades are included in volume. However, if there isn’t a bar then the volume for any ‘excluded’ trades doesn’t get counted (ie there isn’t a bar to attach it to). Summing the bars for the day will therefore generally sum to less than the volume for the day. The ‘best practice’ to getting the current volume for the day is to fetch a snapshot and use the daily_bar.volume which is returned. That will be the actual daily volume up to the current time.
Hope that all makes sense and answers the questions.
Dan, thanks, this helps me a lot. I will switch to using a snapshot daily bar.volume. The volume difference at the end of the day using minute bars is significant.
John
Omar, here is a code snippet from info Dan shared. I refresh every minute. Works perfectly and my first tests of data is accurate.
api = REST(config.ALPACA_API_KEY_ID, config.ALPACA_API_SECRET_KEY, base_url=‘https://api.alpaca.markets’)
snapshot = api.get_snapshot(“QQQ”)
print(snapshot.daily_bar.volume)
print(snapshot.daily_bar.close)
print(snapshot.daily_bar.low)
print(snapshot.daily_bar.high)
print(snapshot.daily_bar.open)
print(snapshot.daily_bar.timestamp)
Thanks for all the information man !
Really appreciate it
Wow Dan, thanks a lot !
I think that your detailed answer will help many users on this forum.
10x, Kudos
The explanation of why the sum of intraday candle volumes do not equal daily candle volume was very helpful, thank you @Dan_Whitnable_Alpaca!
For backtesting purposes, how would I go about calculating total volume on the day so far, if I did not want to always underestimate the true value? I’d hate to fetch a snapshot for each minute, for each ticker.
