Something to help people understand algos and stocks

So I know there’s a lot of code going around but very limited basic info of what % 's to look for when buying and selling, what time divergence does in a strategy and such so i’m going to try to cover what I believe can help get someone started or explain a lot better what’s going on in algorithmic day trades. don’t confuse this information with HFT (high frequency trades) since it may have some similarities but at its core is completely different.

My clause im only providing information that I believe should help people based on my experiences, I’m not saying im in any way qualified to explain this (even though i am) IE if you lose money because you started programming something im in no way responsible. think of this like algo trading for dummies.

Spoiler alert! if you’re not a programmer you probably will get bored very fast, also if you are and into fast artificial neural networks this might make you want to throw up. and leave you feeling hungry don’t worry i have very detailed documentation on what has proven to be successful for me so far

I’ll break this down into sections that should put a good picture into the pipeline of what’s going on.

First on the list is data without this and receiving it fast can mean the difference in getting a slice of the pie or seeing that there’s just crumbs or even worse, someones knocking on your door because your brother jimmy stole the pie… and now you need to give them money.

Second is a good strategy, if your a programmer after reading what I have learned in the past year & 1/2 it should help you see past what I was learning when I first started.

Third a broker account, this is where trades are executed. your strategy will send orders to a broker

So let’s get started!


DATA
Feeding your application is among one of the most important things to cover since there are many data feeds that are delayed, there are differences in upstream providers and downstream providers. as there are also different reasons to use one over the other depending on your strategy.

When I was first looking for a top gaining data feed i had tested the latency of my location (NY)
to a few known local servers. one of the first of a few was investing.com using a parsing method.
It had worked for awhile until I was needing to consider microseconds and historical data for other people who know why you’d need that,
I’d suggest looking into finviz.com for DS and alphavantage.co for US The paid subscription work fine for what I was doing and you’ll read later why. Also I was able to get my request down to use the free version of alpha vantage.
Alright so that covered the downstream but now this is where calling data to your strategy can get tricky.

So why upstream, when your sending data out or should i say massive amounts of data out the likelihood of finding someone who has a server that can handle 120 request per min becomes more dunghteing to find i’ve made a list of response times, pros & cons and the servers i had tested for this. i’ll just save you the pain since alphavantage.co ended up being the fastest and free.

UPSTREAM TEST RESULTS
polygon.io
02.262004137039 : response time in microseconds
PROS: time
CONS: historical data is not what i need
alphavantage.co
01.927835941314 : response time in microseconds
PROS: data is free for my use case , with good historical data
CONS: time
finviz.com
05.428810119628 : response time in microseconds
PROS: good downstream
CONS: time
investing.com
02.870043992992 : response time in microseconds
PROS: time
CONS: not the data im looking for

The strategy
Go grab some coffee this is gonna be a long one, fundamentally this is your brain. let’s start here have you ever watched the stock market open up at 9:30AM in the morning and tried to buy let me stop you there. Since January 1, 1983 the birth of ARPANET computers have revolutionised the way we communicate, leveraging the removal of daunting tasks making efficiency and productivity, advance and excel ( see what I did there ), on a serious note i don’t think we can compete with the speed of computers anymore. but like my family says if you can’t beat them join them.

The idea
as a programmer we use a lot of open source technology heck i think most of my developments if not all of them are on top of open source, and ive always liked it that way. With sql and memcached my queries & software loop time always seem to improve, as programmers, open source helps us improve our code.

The application
when I had first started I was dumping html data from https://investing.com/equities/top-stock-gainers something about watching the list load at 9:30 gave some trust to the data I was seeing, this data went into a mysql database table.

The loops
in a desktop software application written in php, i had some main processes that use a terminal to display each running loop

  • a loop for an api to input the data to the db table,
    originally from investing then from finviz

  • a loop to look at each stock that was added and generate a number from 1-20 based on the stocks historical open and close prices

  • a loop that repeatedly checks the price of the stock using the upstream provider and compares it to the one in the data set (Note: it doesn’t necessarily need to remember if the price is increasing rather it needs to remember if it drops)

What it’s looking for
Depending on who you talk to you’ll see everyones strategy may be slightly different. but all route from the same idea, when you base your strategy on change the stock market will have a few regulations that you’ll need to account for in your code. in my case i only want to day trade in a manner of low equity, meaning i don’t want to use over $25k, so this is where day trader protection comes in, it doesn’t necessarily mean you can’t day trade rather you need to limit yourself to only 3 trades a week. also since I will not be trading after hours i needed to account for that by letting the code start looking for stocks at 9:30 and only let it buy 3 a week.
ok with all that being said your here now the stock market just opened up what should your code be doing.

What to watch out for
In my software I like to only account for change after the stock market opens up. i do this by measuring the price after that time, since accounting for the change % on other systems might include that. I like to use a number in my dataset and its proven helpful.
Make sure to adjust your collar, some companies advertise 4% max on top of the current price to ensure execution. What I see is you will need to add an additional % defined on how fast your program can execute the trade for example in total I put 6% on top of the price and that seems to work fine for me. more on the collar later.

What its about to do
So any algo looks at a stock and usually just watch the price for a positive change while taking a drop in price into consideration, what I look for is a little complex to explain so ill show you some code

//a better function for php to map vals i use it for the freq to time correlation
function map($value, $fromLow, $fromHigh, $toLow, $toHigh) {
    $fromRange = $fromHigh - $fromLow;
    $toRange = $toHigh - $toLow;
    $scaleFactor = $toRange / $fromRange;

    // Re-zero the value within the from range
    $tmpValue = $value - $fromLow;
    // Rescale the value to the to range
    $tmpValue *= $scaleFactor;
    // Re-zero back to the to range
    return $tmpValue + $toLow;
}


//this is what I use to generate a gain 
//note the $the_min var this is used to map time to make a gain output
$GAIN = (round(map($the_min, 1, 59, 1 ,  20),0,PHP_ROUND_HALF_UP));

//if i was to map gain as an output you see it change something like this 
//9:35AM = 16 % increase in price to look for 
//9:40AM = 19 % increase in price to look for 
//9:45AM = 20 % increase in price to look for 
//9:55AM = 25 % increase in price to look for 

//this is how i map the gains needed to make a buy	
if ($trade[11] ==0 || !isset($trade[11]) || $trade[11] < 0.1 )
	
	{
	$low_threshold   = ($trade[12] * $GAIN_Percent)+ $trade[12];}
		
else{   $low_threshold   =  $trade[12] + ($trade[11]   * $GAIN_Percent);}
							
    	$high_threshold  = ($trade[12] * $GAIN_Percent)+ $trade[12];
			//price found at  the price now
$proxy_Degradation = round(((1 - $trade[12] / $trade[11]) * 100), 1, PHP_ROUND_HALF_DOWN );					
$price_to_buy_at = round(map($trade[12], $MIN_STOCK_PRICE, $MAX_STOCK_PRICE, $low_threshold , $high_threshold),4,PHP_ROUND_HALF_UP);

How i calculate losses is a bit complex also so ill show you a test script

$the_hour = 10;
$the_min  = 45;
$the_ap   = "am";
$quantitative_trade[4]=2.40;
$quantitative_trade[5]=2.70;
$time_divergence = -17;




$loss_val = round(((1 - $quantitative_trade[5] / $quantitative_trade[4]) * 100), 1, PHP_ROUND_HALF_DOWN );
//sell around 10 am 
if ($the_hour == 10 && $the_min >14 && $the_min<59 && $the_ap == "am" ){
	$time_divergence =  "-". round( map($the_min,    "1", "59","1.8",		  abs($time_divergence)),1,PHP_ROUND_HALF_DOWN);
}
//skip 12am lunch to sell after


//sell around 1pm
if ($the_hour == 1 && $the_min >10 && $the_min<59 && $the_ap == "pm"){
	$time_divergence =   "-".round( map($the_min,    "1", "59","1.8",         abs($time_divergence)),1,PHP_ROUND_HALF_DOWN);
}
//sell around 2pm
if ($the_hour == 2 && $the_min >10 && $the_min<59 && $the_ap == "pm"){
	$time_divergence =  "-". round( map($the_min,    "1", "59","1.8",         abs($time_divergence)),1,PHP_ROUND_HALF_DOWN);
}
//sell around 3pm
if ($the_hour == 3 && $the_min >10 && $the_min<59 && $the_ap == "pm"){
	$time_divergence =  "-". round( map($the_min,    "1", "59","1.8",         abs( $time_divergence)),1,PHP_ROUND_HALF_DOWN);
}
//sell around 4pm
if ($the_hour == 3 && $the_min >45 && $the_min<59 && $the_ap == "pm"){
	$time_divergence =  "-". round( map($the_min,    "45", "59","1.8",        abs( $time_divergence)),1,PHP_ROUND_HALF_DOWN);
}


 if ($DOWN_TREND == TRUE){ 
		$ECH_LOSS_VAL = $loss_val * 2.0; //2.0-2.5
 }else{ $ECH_LOSS_VAL = $loss_val * 1.0; //1.0-1.7
 }
 
 //the output sent to user 
if ($ECH_LOSS_VAL	  <=  $time_divergence	){
	
	$what_i_do = "Ill sell it";
	
}else {	$what_i_do = "Ill keep it";}
echo "\n";
echo "Loss is $loss_val \n";
echo "Time Divergence At $the_hour:$the_min:$the_ap is ".$time_divergence. " ". $what_i_do;
echo "\n";

NOTE: This list of code is not here to give you a functional algo day trading system it just shows how to program a bit of one, or more so how I programmed mine

What’s going on
Somethings that change depending how fast the code can process are affected by the processing power, but mostly the read and write speed of your hardware, having a solid state hard drive over a spinning disk platter will greatly affect performance. and if the collar and the % that you have added to it does not account for this change, you will see your orders being rejected. so your loop just saw a stock jump over the gain, could be (16-20% usually) that you were just measuring for that stock, now this is where the broker and collar comes into play.

The broker
Executing trades can be scary but building a back testing script felt even more, I actually was able to build it faster than I thought. Point being definitely back test before you go live

In conclusion
After a year and a half I was able to look at my past with high values for myself in the future, and building an algo was not so bad lol, I’d advise the next person that as long as you feel comfortable programming, you can do it in your spare time and if you need help you’re not alone just don’t try to take it on as your first but next big project.

-Kris

3 Likes

Thanks a ton, Kris! That’s very helpful for everyone here.

Could you elaborate more on how you managed to go down to the free version of alphavantage? It seems quite limited with 5 API requests per minute; 500 API requests per day (https://www.alphavantage.co/premium/). E.g. you could only (almost) make 1 request / minute for 2 stocks for the whole day.

I’d assume that you use finviz to create a shortlist of ca. 2-5 stocks that you want to consider trading on a given day. Then, you start getting alphavantage data for these until you exclude tickers from your shortlist that disqualify (quickly, e.g. in the 1st market hour or so). Then, you keep getting data for the 1-2 stocks that qualify best and could/will be traded.

Is that how you do it?

No not necessarily the system can see 1K rows in the database, per day but will only buy one. I’ll give a in detail explanation of why the 5 request is not a issue by tomorrow if necessary, but understand as long as you write your code to only propagate necessary request you’ll be just fine i assure you.

That would be great! I hope you don’t mind my follow up questions. In any case, I think it’s good we are doing this in the forum, so everyone can read about it directly.

In my understanding, you use daily historical data (question: all time? from where?) and other data (e.g. fundamental data from Finviz) for selecting the 1 ticker that you want to trade on a given day. And then, you are using the alphavantage stock price OHLC data (question: in 1 minute resolution?) for actually trading that 1 ticker, e.g. to spot when you should close your position etc. Question: Are you also using the alphavantage data to actually make the final choice to enter the trade for that 1 ticker or not?

One thing that’s unclear to me: If you consider 1k tickers initially (at a given morning) and if you also want to use the daily data from the last day for selecting that 1 top ticker, you’ll also need an additional 1k requests from e.g. alphavantage (or some other service), right?

Hey just a quick replay that should help clear things up,

historical data ( question: all time? from where? )
alpha vantage the call would look something like this
/query?function=TIME_SERIES_DAILY&symbol=".$symb."&apikey=".$api_key."&outputsize=full"

( question: in 1 minute resolution? )
Yup

Question: Are you also using the alphavantage data to actually make the final choice to enter the trade for that 1 ticker or not?
After a stock price and change after 9 : 30 am is verified i use finviz data for that since im already calling it, when it decides it wants to buy something then it will use alpha. so yes

One thing that’s unclear to me: ‘1k requests’
Just keep in mind this number changes each day, it could 500 rows on a bad day for the stock market or over 1k i haven’t seen go much past that.