Questions about pymarketstore

Alpaca Account Troubleshooting
1

Linux Ubuntu 18.04.6
Language: python 3.11.5

I just started using marketstore and I’m trying to figure out how to populate my database. So far I started the docker container using:
docker run -v "/home/esf0/marketstoredata/data:/data" -i -p 5993:5993 alpacamarkets/marketstore:latest

where /home/esf0/marketstoredata is the path where I want to have the database.

Following the example on github I did:

symbol, timeframe, attribute_group = "TEST", "1D", "OHLCV"
data_type = [('Epoch', 'i8'), ('Open', 'f4'), ('High', 'f4'), ('Low', 'f4'), ('Close', 'f4'),('Volume','i8')]

tbk = "{}/{}/{}".format(symbol, timeframe, attribute_group)
client = pymkts.Client()

data = np.array(df_test[['date','open','high','low','close','volume']].to_records(index=False)
         ,dtype=[('Epoch', 'i8'), ('Open', 'f4'), ('High', 'f4'), ('Low', 'f4'), ('Close', 'f4'),('Volume','i8')])

And this is how the data looks like:

data
array([(1262563200,  31.39,  31.63,  31.1314,  31.3 , 2729300),
       (1262649600,  31.21,  31.22,  30.76  ,  30.96, 2994300),
       (1262736000,  30.85,  31.  ,  30.76  ,  30.85, 2320300), ...,
       (1718841600, 133.26, 134.65, 132.53  , 132.73, 2887363),
       (1718928000, 132.85, 134.5 , 132.2   , 133.25, 4461292),
       (1719187200, 135.3 , 137.71, 134.34  , 135.08, 3339551)],
      dtype=[('Epoch', '<i8'), ('Open', '<f4'), ('High', '<f4'), ('Low', '<f4'), ('Close', '<f4'), ('Volume', '<i8')])

then I wrote it and checked it by querying:
``
cli.write(data, tbk)
cli.query(pymkts.Params(‘TEST’, ‘1D’, ‘OHLCV’,limit=10)).first().df()

and got:

Open High Low Close Volume
Epoch
2010-01-04 00:00:00+00:00 31.389999 31.629999 31.131399 31.299999 2729300
2010-01-05 00:00:00+00:00 31.209999 31.219999 30.760000 30.959999 2994300
2010-01-06 00:00:00+00:00 30.850000 31.000000 30.760000 30.850000 2320300
2010-01-07 00:00:00+00:00 30.780001 30.820000 30.500000 30.809999 2214000
2010-01-08 00:00:00+00:00 30.639999 30.850000 30.400000 30.799999 2670900
… … … … … …
2024-06-17 00:00:00+00:00 131.110001 132.070007 129.500000 131.830002 2064584
2024-06-18 00:00:00+00:00 132.130005 135.149994 131.559998 134.899994 3453080
2024-06-20 00:00:00+00:00 133.259995 134.649994 132.529999 132.729996 2887363
2024-06-21 00:00:00+00:00 132.850006 134.500000 132.199997 133.250000 4461292
2024-06-24 00:00:00+00:00 135.300003 137.710007 134.339996 135.080002 3339551
3642 rows × 5 columns


So far everything looks good, but when I go check the tree, I'm getting an "error opening dir" message.

(base) esf0@DESKTOP-5CVVTN8:~$ tree ~/marketstoredata/data /home/esf0/marketstoredata/data
├── TEST [error opening dir]
├── WALFile.1719352450647782449.walfile
└── category_name 1 directory, 2 files

So, I’m not able to see the list of time frame files. I would appreciate some help troubleshooting that.

Additionally, I see that the timeframe strings have fixed values, but for the atribute groups, in the example I only see “Tick” and “OHLCV” used. I tried both and I don’t see any difference on what’s written when using either of them. Also, I was able to test writing any string (ie. “xxx”) and it would work too. I’m wondering if that is totally up to the user or if there would be any problem if I create my own custom attribute groups, for example if I want something like “OHLCadjOHLC”, would I run into trouble down the line?

Thanks in advance!
Flor

2

The next day the error in the tree was gone and all the files are shown now. For the second question, I tried different attribute group names and they all work, so I guess it could work for any name.