In Python, connecting to alpaca_trade_api with the import statement takes an exceptionally long time (about 3 to 5 seconds each run):
import alpaca_trade_api as tradeapi
This is quite a hassle as I frequently rerun the code for debugging.
Did anyone else notice it or is it just on my computer?
Any workarounds or solutions for this problem?
With thanks,
Gil
Hi
ran this:
got this:
don’t see the issue you’re referring
Thanks Shlomi!
Typed the same code on my computer and here’s the result:
(utils) C:\Dropbox\Documents and Writings\Research\Digital Humanities\Python>python test_alpaca.py
it took: 2.5780353
Weird! I wrote an email to the Alpaca team and they referred me to this forum. Any suggestions how to find out what is wrong with my setup? BTW, this is running from a pretty robust Windows 10 system.
With thanks! (and Toda!)
Gil
try installing everything in a fresh virtual env. maybe you have something from other packages that causes that.
I actually just started a new environment (for the first time …). I am now thinking it might be something related to an anti-virus check or something like that … this is so weird!
I will try playing around with a new venv as well. Thanks for your help.
Gil
Is there a way of profiling the import process so it would be possible to track the specific code that takes a longer time? Or perhaps you might think of the specific function that might take long within the alpaca import procedure?
Thanks again
Gil
I don’t think this is the direction you want to go with… don’t think it will give you what you need (profiling the imports)
if you want to go with that, in each module you have an __init__ file which imports other modules.
import each of them and see which takes more time.
but again, I don’t think this is the right way to go
I do recommend using python 3.6.X
I think you are right Shlomi. Since my last email I discovered that there are other imports that take unusually long time:
from pathlib import Path: 0.07232890000000003
import os: 1.7000000000488846e-06
from RepeatedTimer import RepeatedTimer: 0.018011299999999952
import matplotlib.pyplot as plt: 1.6333111999999999
import decolog as lg: 0.003073700000000068
import alpaca_trade_api as tradeapi: 1.7710194
import PySimpleGUI as sg: 0.05677429999999983
Yea, probably what’s slowing this down the most is that it imports pandas. Python imports have side effects (code that runs once behind the scenes at import time; faster CPU will help this), as well as just that some libraries are large and require reading and interpreting a lot of files (faster SSD will help this). Once a library has been imported the Python interpreter caches it so the next time something else requests to import it, those imports will be near immediate (within the same script/process, at least).
Thanks Brian.
What you describe here makes sense on usual terms, but in this case I do have a decent processor and a Samsung SSD … I worked before with pandas and this did not happen. I think the issue of slow loading of pandas started with virtualenv but I can’t place my finger on it (now, even when I run the script without activate, it is still slow …).
Also, there seems to be no caching: everytime it again takes a long time … I am running python SCRIPT_NAME from the command line every time – does the cache even work in such a way?
Thanks again for your help
Gil
Hmm well, when you run python script.py, each time that runs it starts a new process. So it will need to repeat the import work every time. Whereas for example inside of IPython or jupyter-notebook, those are long-running processes where everything remains imported.
Depending on which calls you need to make, most of alpaca_trade_api is just a relatively thin wrapper around the requests and/or websockets libraries, so you may not need to use it? But I doubt if the devs will want to drop pandas support altogether; IMO it’s the right tool for the job, even if it may be slow to import 
Very helpful Brian.
First thing: you inspired me to explore IPython a bit, and now with %run, my code runs in a fraction of a second (with the caching of the imports). So already this is a huge breakthrough for me. (Still a problem: running a debugger outside of IPython … I am working with Vim and vdebug, and pydbgp is very slow …).
– I already tried to minimize the alpaca import to REST only, but it doesn’t really change things. Same long time to load.
– Totally agree on your bigger point: pandas and in this case – alpaca – are great tools and worthwhile these minor irks.
Thanks again!
Gil