I am noticing a lot of missing bars in the minute bars API, for example, AAPL at 2023-05-18 20:30:00+00:00. Is this intended?
@Jark It’s very common, especially after market hours, to have missing bars. Bars are only created when there are eligible trades during the bar. No eligible trades - no bar. During the time 2023-05-18 20:30:00+00:00 through 20:31:00+00:00 there simply weren’t any ‘eligible trades’. Below is a list of all the trades during that time. Note they all have a trade condition “I” (odd lot order) which makes them ineligible for calculating a bar. Take a look at this article for insight into how bars are calculated and the number of conditions which can make a trade ineligible.
| exchange | price | size | conditions | id | tape | symbol | |
|---|---|---|---|---|---|---|---|
| timestamp | |||||||
| 2023-05-18 16:30:00.078592860-04:00 | P | 175.1500 | 1 | [@, F, T, I] | 53793 | C | AAPL |
| 2023-05-18 16:30:00.078593008-04:00 | P | 175.1500 | 1 | [@, F, T, I] | 53794 | C | AAPL |
| 2023-05-18 16:30:00.078802560-04:00 | P | 175.1500 | 1 | [@, F, T, I] | 53795 | C | AAPL |
| 2023-05-18 16:30:00.079001820-04:00 | P | 175.1500 | 1 | [@, F, T, I] | 53796 | C | AAPL |
| 2023-05-18 16:30:00.079049227-04:00 | P | 175.1500 | 3 | [@, F, T, I] | 53797 | C | AAPL |
| 2023-05-18 16:30:00.079592044-04:00 | P | 175.1500 | 15 | [@, F, T, I] | 53798 | C | AAPL |
| 2023-05-18 16:30:00.079864395-04:00 | P | 175.1500 | 2 | [@, F, T, I] | 53799 | C | AAPL |
| 2023-05-18 16:30:00.080104636-04:00 | P | 175.1500 | 1 | [@, F, T, I] | 53800 | C | AAPL |
| 2023-05-18 16:30:00.080976393-04:00 | P | 175.1500 | 10 | [@, F, T, I] | 53801 | C | AAPL |
| 2023-05-18 16:30:00.083445924-04:00 | P | 175.1500 | 5 | [@, F, T, I] | 53802 | C | AAPL |
| 2023-05-18 16:30:00.084906840-04:00 | P | 175.1500 | 1 | [@, F, T, I] | 53803 | C | AAPL |
| 2023-05-18 16:30:00.086786519-04:00 | P | 175.1500 | 10 | [@, F, T, I] | 53804 | C | AAPL |
| 2023-05-18 16:30:00.086786680-04:00 | P | 175.1500 | 50 | [@, F, T, I] | 53805 | C | AAPL |
| 2023-05-18 16:30:00.086786810-04:00 | P | 175.1500 | 11 | [@, F, T, I] | 53806 | C | AAPL |
| 2023-05-18 16:30:04.826644148-04:00 | D | 175.1550 | 1 | [@, T, I] | 230925 | C | AAPL |
| 2023-05-18 16:30:09.587912309-04:00 | D | 175.1600 | 1 | [@, T, I] | 230926 | C | AAPL |
| 2023-05-18 16:30:09.664603810-04:00 | D | 175.1600 | 1 | [@, T, I] | 230927 | C | AAPL |
| 2023-05-18 16:30:10.580849570-04:00 | D | 175.1500 | 1 | [@, T, I] | 230928 | C | AAPL |
| 2023-05-18 16:30:11.585812369-04:00 | D | 175.1500 | 1 | [@, T, I] | 230929 | C | AAPL |
| 2023-05-18 16:30:14.390359147-04:00 | D | 175.1700 | 3 | [@, T, I] | 230930 | C | AAPL |
| 2023-05-18 16:30:15.559124234-04:00 | D | 175.1500 | 1 | [@, T, I] | 230931 | C | AAPL |
| 2023-05-18 16:30:16.335629870-04:00 | D | 175.1500 | 2 | [@, T, I] | 33636 | C | AAPL |
| 2023-05-18 16:30:20.068575919-04:00 | K | 175.1500 | 5 | [@, F, T, I] | 38959 | C | AAPL |
| 2023-05-18 16:30:27.778146871-04:00 | D | 175.1700 | 2 | [@, T, I] | 230932 | C | AAPL |
| 2023-05-18 16:30:30.369077829-04:00 | D | 175.1700 | 10 | [@, T, I] | 33637 | C | AAPL |
| 2023-05-18 16:30:36.527248319-04:00 | D | 175.1500 | 1 | [@, T, I] | 230933 | C | AAPL |
| 2023-05-18 16:30:36.721254525-04:00 | D | 175.1600 | 1 | [@, T, I] | 230934 | C | AAPL |
| 2023-05-18 16:30:39.318724328-04:00 | D | 175.1600 | 1 | [@, T, I] | 230935 | C | AAPL |
| 2023-05-18 16:30:40.125693543-04:00 | D | 175.1500 | 1 | [@, T, I] | 230936 | C | AAPL |
| 2023-05-18 16:30:43.432822666-04:00 | P | 175.1500 | 11 | [@, T, I] | 53807 | C | AAPL |
| 2023-05-18 16:30:52.372191143-04:00 | D | 175.1600 | 1 | [@, T, I] | 230937 | C | AAPL |
| 2023-05-18 16:30:58.121211194-04:00 | D | 175.1515 | 1 | [@, T, I] | 230938 | C | AAPL |
| 2023-05-18 16:30:59.625767545-04:00 | D | 175.1500 | 11 | [@, T, I] | 230939 | C | AAPL |
Thanks for replying, is this method mostly standard for data collection? I was wondering how yahoo finance is able to provide no missing data if it is done this way.
@Jark All data providers will have ‘missing’ bars it’s just how they handle them which is different. The guidelines for which trades to include, or exclude, from bar calculations is standardized by the Securities Information Processors (SIPs) where all data providers get their data. Here is an example from the UTP spec. Some providers take liberties with the guidelines but basically if there are no eligible trades then there is no bar.
Not exactly sure how Yahoo does their bars but a very common ‘fix’ is to simply forward fill any missing bars. If you can find a ‘missing’ bar in Alpaca data, which isn’t missing in Yahoo data, my guess is the Yahoo bar is identical to the previous bar and was simply forward filled.