Historical Data limited to (now - 9h) instead of (now - 15m)

Alpaca Market Data
1

I’ve been playing around with the Alpaca Python API (paper trading with the free subscription) but have been troubled by the problem above.

I use the StockHistoricalDataClient to fetch the historical data, but no matter what I do, it returns up to 9 hours from the current time only. I’ve tried various things like adding start and end time, removing end time, removing start time, and tweaking the limit, but all of them fall under the same limitation. Is anyone having the same problem?

2

this is working for me–it retries with available bars in case the specific period you are looking for is not available. have you tried toggling between sip and iex?

     timeframes = ['1Min', '15Min', '30Min', '1Hour', '2Hour']  # List of timeframes to try
    prices = {}
    attempted_symbols = set()  # Track symbols that have been retried
    symbols_to_remove = set()  # Track symbols to remove after all timeframes are tried

    for timeframe in timeframes:
        logging.info(f"Trying with timeframe: {timeframe}")
        symbols_str = ','.join([urllib.parse.quote(symbol.strip()) for symbol in valid_symbols if symbol not in symbols_to_remove])
        if not symbols_str:
            break  # Exit loop if there are no valid symbols left

        url = f"https://data.alpaca.markets/v2/stocks/bars?symbols={symbols_str}&timeframe={timeframe}&start={start_time}&end={end_time}&limit=1000&adjustment=raw&feed=sip&sort=asc"
        headers = {
            "accept": "application/json",
            "APCA-API-KEY-ID": API_KEY,
            "APCA-API-SECRET-KEY": API_SECRET
        }
        logging.debug(f"Requesting URL: {url}")  # Debug logging
        response = requests.get(url, headers=headers)
        logging.debug(f"Response status code: {response.status_code}")

        if response.status_code == 200:
            data = response.json()
            logging.debug(f"Response data: {data}")

            for symbol in valid_symbols:
                symbol_str = str(symbol).strip()
                if symbol_str in data['bars']:
                    bars = data['bars'][symbol_str]
                    if bars:
                        closest_bar = min(bars, key=lambda x: abs(datetime.fromisoformat(x['t'].replace('Z', '+00:00')) - one_hour_ago))
                        prices[symbol_str] = closest_bar['c']
                        logging.info(f"Data found for {symbol_str} in timeframe {timeframe}. Price: {prices[symbol_str]}")
                        symbols_to_remove.add(symbol_str)  # Remove the symbol from future retries
                    else:
                        logging.warning(f"No bars found for {symbol_str} in the given time range for timeframe {timeframe}.")
                else:
                    logging.info(f"No data found for {symbol_str} in timeframe {timeframe}.")

        elif response.status_code == 400 and 'invalid symbol' in response.json().get('message', '').lower():
            invalid_symbol = response.json()['message'].split(': ')[1].strip()
            cleaned_symbol = invalid_symbol.replace('%20', '').strip()  # Clean up %20 and strip extra spaces
            logging.error(f"Invalid symbol detected: {invalid_symbol}. Attempting retry without spaces.")

            # Retry the cleaned symbol if it hasn't been retried yet
            if cleaned_symbol != invalid_symbol and cleaned_symbol not in attempted_symbols:
                valid_symbols = [cleaned_symbol if symbol == invalid_symbol else symbol for symbol in valid_symbols]
                attempted_symbols.add(cleaned_symbol)  # Mark as retried
            else:
                logging.error(f"Symbol {invalid_symbol} is still invalid after retry. Removing from the list.")
                symbols_to_remove.add(invalid_symbol)  # Mark symbol to remove after retry

        else:
            logging.error(f"Failed to fetch data: {response.status_code}. Response: {response.text}")

        if not valid_symbols or len(symbols_to_remove) == len(valid_symbols):
            break  # Exit when all symbols have been processed

    # Final log for any symbols that couldn't retrieve data
    for symbol in valid_symbols:
        if symbol not in prices:
            logging.warning(f"No data found for symbol {symbol} after all retries.")
1 Like
3

I haven’t tried using requests directly since I was trying out the API methods first. I just tested it out, and the result was the same. The only difference was that for SIP the exact time delta was 9 hours and 15 minutes, while for the IEX is 9 hours flat. I even tested changing timezones, but still has the same blank.

4 
from alpaca.data.historical import StockHistoricalDataClient
from alpaca.data.requests import StockBarsRequest
from alpaca.data.timeframe import TimeFrame
import datetime

# Initialize Alpaca's StockHistoricalDataClient
client = StockHistoricalDataClient(API_KEY, API_SECRET)

# Define the start and end dates for the data you want to fetch
start_date = datetime.datetime.now() - datetime.timedelta(days=365)  # 1 year ago
end_date = datetime.datetime.now()  # Current time

# Create the StockBarsRequest for daily data
request_params = StockBarsRequest(
    symbol_or_symbols=["AAPL"],  # You can replace this with your desired stock symbol
    timeframe=TimeFrame.Hour,  # Timeframe for daily data
    start=start_date,
    end=end_date
)

# Fetch the historical data
bars = client.get_stock_bars(request_params)

# Print the fetched bars
for bar in bars:
    print(bar)

Using the above code I am able to pull data from 16/10/2023. Not sure what your issue is. You need to paste your code if you want help.

1 Like
5

This is the exact method I used first, and thus the said delay. I couldn’t find any solution to it, so instead, I focused on creating my own dataset instead. I switched over to the live data stream using the alpaca.data.live.StockDataStream which, of course, returned real-time data instead. It’s a bit of a roundabout way, but with the minute bar data, I can make hours or day windows myself by aggregation. Thanks for the answer though! :grin: