Hey everyone! I'm diving into a new project where I'm setting up a monitoring system for a photovoltaic plant, which includes connecting 18 GW250K-HT inverters to an EzLogger3000U. I've previously built a system that pulls data from an API using Python and Dash, but I think this new challenge will be much tougher. My goal is to read data straight from the EzLogger using ModbusTCP, but I'm a bit concerned about whether Python will handle the high volume of data being transferred every second. Has anyone tackled something similar? I'd love to hear your experiences and advice!
5 Answers
If you're dealing with high data volume, I collect data from hundreds of channels at a rate of 100Hz—which ends up being around 6GB of data per hour. I haven't had any issues since I use a microservices architecture, which distributes the load.
I've had great success using Python's pyModbusTCP on a Raspberry Pi 2 to handle thousands of tags with multiple PLCs at 1Hz. I haven't really tested its limits, but it works well so far.
Absolutely, you should check out pymodbus! We've had a solid experience with it. While Python isn't the fastest language, in this case, the speed of your network will be much more of a bottleneck than Python's performance itself.
Definitely go for pymodbus! Using the Async-Client can give you better performance, especially if you're reading and sending data at the same time. Just a heads up: the Modbus protocol will likely present more challenges than Python will, so keep that in mind!
I've actually been using Golang with InfluxDB and Grafana for a monitoring solution that tracks over 50 parameters every second. Dash just didn't measure up for me; it felt slow and lacking in features. Grafana is a powerful tool for visualization—definitely worth considering!
Grafana looks cool! Is it similar to Power BI? I'm curious if it's customizable since I typically prefer using MSSQL for data storage.
I'm not a fan of async either; it confuses me. I usually stick with threads for concurrent tasks. Plus, I've faced a lot of communication issues with Modbus—sometimes I miss data during the process.