Missing minute data for several stocks

Hello, while running some backtesting I noticed some of the minute bars are missing. Specifically at the market open. I can only assume there will be missing minute bars during the full trading day aswell. I am using the SIP feed when I’m fetching the historical bars.

2023-12-01 16:48:12,558 WARNING:First bar is not 14:30 for symbol: FI Close High Low n Open Volume vw
2023-11-10 14:31:00+00:00 119.62 119.95 119.62 33 119.95 18653 119.919696

2023-11-14 14:30:00+00:00 379.99 380.00 379.520 66 379.520 3715 379.661762
2023-11-14 14:33:00+00:00 380.94 380.96 380.905 48 380.915 1359 380.904312
2023-11-14 14:35:00+00:00 381.85 381.94 381.850 36 381.940 743 381.515579
2023-12-01 16:50:13,245 INFO:First three bars are not 14:30, 14:31, 14:32 for: DPZ

The same here. My minute bar historical data has 3-5 minute gaps. Why is that?

Has there been any updates on this topic - I am seeing missing minutes when querying QQQ data from the IEX feed. SIP feed has no missing data. I assume that the IEX just did not have any activity in that one minute interval. This is creating some problems with properly generating plots with multiple tickers, the number of samples per ticker is varied and not 390 per day.

@torobot There will always be ‘missing bars’. Fundamentally, bars are only created if there are ‘valid’ trades during the bar. No valid trades then no bar. Even with highly traded symbols such as SPY there can be a ‘missing’ bar every once in awhile. You can read a bit more on how bars are created and what constitutes a ‘valid trade’ in this article.

Using IEX data there will be more missing bars than when using SIP data. Why? There simply are fewer trades executed on the IEX exchange. SIP data includes all trades from all execution venues. The trades executed on the IEX exchange are only a small subset of these (about 2%). As long as one sets an end datetime no more current than the previous 15 minutes, one can specify feed=sip and get full market bars even with a free data subscription.

For applications which don’t respond well to ‘missing’ bars, a typical approach is to forward fill the data. In python, pandas has several useful methods for this. Check out ffill and resample.