Data Download

[NoSpark]

Good day! 

Thus far I am really enjoying my SmartPi!  I work as an engineer, and an old and senior one at that, for an electrical utility.  The concept of a smart meter really resonates with my fellow workers.  I have the SmartPi web (internet) facing so I really like to show it off around the office and during meetings!  Other engineers like to look at the data and try to figure out what appliances are operating.

Anyway, the only issue I have is with the display of data.  The Download or Table function works well when displaying a day or two of data, although it is a bit slow.  However, if I try to download two or three days the SmartPi stops responding to the web interface.   When it finally does produce the data I notice that the SmartPi has continued to collect data in the background and there is no gap in data continuity.

Two days ago I attempted to download about a month of data and apparently this locked up both the SmartPi Web interface and hung the data collection process resulting a loss of metering data for about 24 hours, that is when I was power cycled the to restart the SmartPi.

I have a 32GB SD card installed and all Raspbian and enerserve software is up to date.

My device is configured for a North American residential supply with a split phase 240 volt 60Hz system
My connections are:

Voltage:
L1 - L1 (120 v to Neutral [GND])
L2 - L2 (120 v to Neutral [GND])
L3 - Neutral [GND]
N - Neutral [GND]

Current:
L1 - L1
L2 - L2 
L3 - Not Connected (Open)
N - Not Connected (Open)

The data is not important to me yet, so this is just a user experience!

Thanks for a great product and your continuing support!

Cheers,
NoSpark

[jens]

Hello NoSpark,

you are right. The collection over a long time is very slow. It is in my mind to speed up this process.

Best regards
Jens

[SuperQ]

I'm working on implementing Prometheus library support in the readout server.  This will let us use the Prometheus time-series database for storing data.  It can easily pull up a month of raw data in couple of seconds.

[NoSpark]

Thanks for the quick reply!
No worries, I understand that this is a "work in progress".

It is really looking good so far!

Cheers,
Tom Hartman

[jens]

Hello NoSpark,

until now, the SmartPi did not create an index over date when the datatable was created. In the Github repository it is already fixed. This will be improved with version 0.5.9. 
All newly created tables get the index. This means that the table in the next month will be created with index.
For the already created tables you could create the index by yourself:

Code:

CREATE INDEX IF NOT EXISTS `dateindex` ON` smartpi_logdata_201705` (`date` ASC)

smartpi_logdata_201705 is the name of the corresponding table and is the same as the database in /var/smartpi /db/.

Best regards Jens