Accurate OPG and CLS prices for Paper trading

[DavidC]

It would be very helpful if the paper trade order prices accurately reflected the market for OPG (Market On Open, MOO) and CLS (Market On Close, MOC) orders. Right now it looks as if the OPG and CLS orders are grabbing some price near the open or close rather than the actual open or close price and this can often vary significantly. For example, one of my paper trades this morning was to short VIAC at the open. The market open was $36.81 but the paper trade had the trade open price with a short at $36.70 or 0.3% lower than it would have been in the real world.

I would think that it would be very easy from a programming perspective to use the actual market opening and closing prices for OPG and CLS orders with paper trading. Unless an OPG or CLS trade was large enough in size to affect the initial price auction (say, more than ~2% of the average daily volume), you can pretty much guarantee that, in the real world, an OPG or CLS order would be filled at a stock’s Open or Closing price.

Now, I’m not saying that the paper trade should look at the trade’s volume compared to the average volume; let the programmer understand that. But, anyone with a reasonable number of shares for a trade should get the Open and Closing price with OPG or CLS orders and this would be very easy to code into the paper trading engine.

Thanks!

[sean]

I cannot say for certain if the Alpaca API is returning the correct opening auction price. However you should note that there is a difference between the first print on the NMS tape and the opening auction print at the primary exchange.

For example, looking at VIAC on Yahoo Finance for Dec 29th, 2020, it shows the first NMS price at $36.81. However there’s no guarantee that the market maker on the Nasdaq opened at exactly that price in their opening auction. In fact, looking at the opening 1-minute high/low bar, it seems like there was a lot of selling pressure. It’s likely that there was an order imbalance during the Nasdaq opening auction which meant the market maker would have printed the opening lower to allow for all the buying they had to do.

Short answer: Never assume the opening price from NMS will be the price you get on an MOO/LOO order. You’ll get close on Nasdaq. But if it’s a NYSE/AMEX listed name, you could easily see a price deviation of 2%.

[DavidC]

We may have to agree to disagree here. Talking with my trader who has 20+ years of experience on the street with large and institutional investors, he says that if you put in a MOO order and you don’t get the opening price, then something is wrong. (Note that Alpaca uses Market → OPG instead of MOO.)

Prior to the open, all MOO, LOO, or GTC limit orders are tabulated and based upon the order prices, an opening price is generated. All MOO orders should receive that price. Depending on that opening price, GTC Limit or LOO orders may or may not be filled. But Market On Open orders should always be filled and at that opening price. My trader said that if he didn’t get the market price for a MOO order, he’d be on the phone to get that price changed.

After watching the Alpaca paper trading engine work for awhile, I can say that it doesn’t work this way…essentially ever. The Alpaca engine is looking at MOO orders as if they’re normal market orders during the trading day. If you have a market sell order, it should go off at the highest bid providing that there are enough shares in the bid to fill your order. If not, the price should go down to the next highest bid…and so on until your sell order is filled.

If your market order is to buy some shares, then your buy order price will be the lowest asking price, again providing that there are enough shares on the ask to fill your buy.

But that’s not the way a MOO / Market-OPG order is supposed to work as I’ve outlined above.

It’s become pretty apparent after watching weeks worth of paper Market-OPG orders, that Alpaca never put any effort into the paper trading engine code to have Market-OPG orders function as they would in the real world. I mean, it does make sense from their perspective; paper trades don’t directly equate into real order flow selling or other revenue generating opportunities. So they skimped on the coding a little bit…though the coding for this capability would be extremely minor. Just have a Market-OPG order price automatically equal the market’s actual opening price.

This would cost a little programmer time (a very little time) but it would then allow more than functional testing of a trading algorithm; it would allow anyone using MOO or LOO orders to actually test the logic behind their algo’s trades.

Their current method for Market-OPG orders creates really bizarre artifacts. For example, about 10 to 15% of the time, my test Market-OPG orders receive partial fills…and I’m talking partial fills on $4k paper money orders. Also, since bid / ask volume can be pretty slow immediately after the open, you can get some strange results. For example, one of my algos had a MOO buy on for KLAC and it paper filled at the lowest ask after the open…at $305.71. The real market open was $301.60…which also was the HOD. So, with this flawed / lazy way of doing Market-OPG orders, my algorithm’s opening buy came in at 1.36% above the HOD.

Let’s just say that my trader is very unimpressed with how the paper trading engine works and it makes him question why we’re using Alpaca for our programs. He wonders if we’re going to see the same ridiculous differences with live money trades. And while I try to explain to him how this is a non sequitur when it comes to programming, I can understand how someone whose career has been focused within the street rather than directly in technology would lose some confidence when seeing paper trades go off so inaccurately. If our initial investors saw these trade prices, I suspect they also would be far less confident in trusting us and Alpaca with their money. And because of this, I’m pretty surprised that Alpaca wouldn’t put a very small amount of programmer time into fixing how their Market-OPG orders work. And, I suspect, Market-CLS orders have this issue but none of my algorithms use MOC / LOC orders currently.

[Vin_Z]

This is a huge problem. Alpaca should really look into it. There was a 0.76% price difference between my OPG order ($69.01) and the real market open price ($69.54). 0.76% return loss per day can result into 85% yoy loss.

[pablo.mitchell]

To test the paper treatment of CLS and OPG prices I conducted a simple experiment where I randomly selected relatively liquid stocks and bought a small random number of shares at the MOC auction with time in force CLS. I then sold the same positions the next morning at the MOO auction with time in force OPG.

For each auction I compared the fill prices to adjusted open or close prices provided by Tiingo. I computed the average discrepancy in basis points for each auction. To try and get a sense for how extreme the discrepancy in each case was I performed a bootstrap analysis with the data I collected. This entailed drawing 20,000 random samples with replacement and then comparing the resampled distribution of averages to each sample average.

The 99% confidence interval (bootstrap histograms at the bottom) for each side are as follows:

Discrepancy

• MOC Buys: 12 to 22 bps
• MOO Sells: -43 to -31 bps

The data were gathered for 183 different names from 2021-04-06 to 2021-05-12 and comprising 1000 buys/sells.

It seems there is a systematic problem with the way these auctions are being simulated. I suspect these distributions should (perhaps not perfectly) be centered closer to zero. These tests are statistically VERY different from that.

However, I am open to the idea that the experiment and/or analysis is flawed. I would welcome feedback and comments.

[Dan_Whitnable_Alpaca]

Currently, paper trading doesn’t simulate MOO and MOC order prices. It treats those orders as regular market orders which simulate prices using the bid and ask (ie buy orders fill at the ask and sell orders fill at the bid). Improved price simulations is definitely an enhancement which is on the features list.

Paper trading has a number of limitations. The first and foremost use case of paper trading is to test and debug ones algo. While simulating live trading results and validating the efficacy of ones strategy is doable in paper trading, it is not perfect and will be off in many instances. MOO and MOC orders are one case. Some other issues with paper trading are it does not consider volume in filling orders. One could place a buy order for 100000 shares of some stock and it will fill at the current ask price. In live trading, that large order would more than likely impact price. Paper trading also doesn’t currently account for margin fees, short borrowing fees, FINRA and SEC fees. For some types of trading this can have a sizable impact. Fill times in paper trading are ‘instantaneous’ however in live trading there is generally a 50-100ms delay and, at times, much longer than that. Paper trading doesn’t account for corporate actions and adjust limit prices for outstanding orders accordingly. Paper trading also doesn’t consider halts in stock trading which can throw off fill prices. Those are just some of the ways which paper trading will not entirely mimic live trading. Alpaca is working to add more fidelity to paper trading simulations, but do be aware of it’s limitations.

[pablo.mitchell]

Good to know that’s the case. Thanks for clarifying. Any sense for when these enhancements will happen? For paper orders that fill, why not simply give portfolios the CLS or OPG price?

Also, MOO and MOC do NOT have bid/ask spreads in the sense you refer to – you either get those prices or you don’t. That is in large part why people use them. The other points you make are good ones but for the most part not relevant to this discussion.

I believe leaving things the way they are is doing Alpaca and users a disservice. Yes, paper portfolios should be used to remove bugs from code. But that’s not the only reason to use a paper portfolio. In some instances you will be giving users a false sense of confidence/pessimism if they are basing whether or not to use a strategy on prices they receive at CLS or OPG. These prices are not noise around CLS and OPG, they are systematically and directionally biased.

The more successful strategies run on Alpaca the better for everyone wouldn’t you agree? Please make this a priority.

[person]

Hey, this is causing me grief as well.
It would be nice to fix it, especially since it can be fixed by simply giving MOO orders the opening price for the relevant exchange, which seems like it would be like 2 hours of developer work (and analogously for MOC).

[DavidC]

Dan,
Can you provide any kind of guidance as to when Alpaca will make the requested improvements to paper trading with respect to OPG and CLS (MOO / LOO / MOC / LOC ) orders. Right now the behavior is abysmal. For this morning’s OPG, I had 45 OPG orders. Of those, 7 never filled and were “canceled” about a minute after the open. An additional 5 were partial fills which are totally ridiculous. All of the stocks that I’m paper trading are S&P500 stocks which means a lot of depth and my algo’s trading size is only $12k per position. (For example, one of the positions was ABBV and the paper trading engine only filled 70 out of 107 shares with today’s volume of 12.7M shares traded.)

So, 12 out of 45 paper OPG trades (both buys and sells) or 26.7% had problems with “execution” on what you admit isn’t really an OPG order but rather a regular market order. Very problematic…

Now, how does this fit into prioritizing new feature ads vs fixing / correcting OPG order behavior? I.e. where’s the bottom line profitability for Alpaca to prioritize this fix / enhancement? (I’ve been in customer support for 30+ years and money is always the bottom line.)

I’m developing and have developed multiple trading algorithms and have a limited amount of investor capital to use for the initial testing. I have more algorithms to test than I have investor capital. As my algorithms prove themselves in the real world, this problem will resolve itself. But my investors want to see 6+ months of performance before committing additional capital on the same algorithm let alone on a new algorithm. So, for me, being able to paper trade an algorithm and show semi-real performance is a way to get these investors to put capital in on my new algorithms now, rather than 6 months to a year from now.

The faster I grow, the faster my accounts grow, and the faster Alpaca’s volume grows. (More volume = more order flow sales = more money for Alpaca.)

Paper trading is understandably a feature that Alpaca intended as a method for proving out trading programs and bots rather than trading algorithm / logic. Totally aware of that. However, with a relatively simple “fix”, for me as I often use OPG orders in my algorithms, the Alpaca paper trading engine moves closer to being a proof of algorithm performance tool. Something that would help me get to higher volumes much more quickly. A win-win situation for Alpaca and myself.

Hopefully you can see the value in this especially considering how relatively simple the paper trading logic enhancement should be.

Thanks,
Dave

[DavidC]

Ok, here’s a funny / sad albeit anecdotal example of how ineffective / inaccurate the current paper trading logic is especially for higher priced / somewhat lower volume S&P500 stocks.

My algo had a market buy of ULTA at todays open. Open auction came in at $377.66. Paper trading produced a paper OPG buy price of $381.77 or 1.1% higher. At the current time, there have been 37k shares traded with the high of the day at $381.75 and, if purchased at the open auction price, my algorithm trade would already have exited with the profit target having been hit. The last trade at the time of this post was $381.01 which is still above my profit target for the algorithm trade.

But the really funny / sad part is that, while my algo would have already successfully completed the trade, the position is showing as underwater in the paper account and my entry price is actually above the current day’s high, an hour+ after the market open and with the stock having moved up nicely from the open.

[Dan_Whitnable_Alpaca]

The primary use case for paper trading is to debug one’s algo and not necessarily to test one’s strategy. While the intention is to have paper trading simulate live trading, there are limitations. The two here are 1) paper orders always fill at the current bid/ask (ask for buy orders / bid for sell orders). This does a pretty good job (actually worst case) of simulating regular market orders, it doesn’t simulate MOO and MOC orders which fill independent of the bid/ask prices. 2) there is a built in randomizer which, every once in awhile, creates a partial fill. This too does a pretty good job of simulating regular market orders, but doesn’t do so well with MOO and MOC orders which generally always (though not 100%) fill complete.

Alpaca is moving towards aligning paper simulations with live execution. But again, the current focus is on the features and not foremost on simulating the fill prices. That said, better simulation of MOO and MOC orders is on our roadmap.