Real-Time Energy Management with InfluxDB and eSoftLink IoT Platform

By Chris Churilo / Product, Use Cases
Jul 21, 2022

Smart energy IoT platforms are empowering consumers to track energy usage and even control spend based on their next bill’s forecast. Yet eSoftThings, a specialist in the Internet of Things (IoT) and artificial intelligence (AI), set out to push smart energy management even further, for both consumers and utility companies, through its IoT platform eSoftLink.

The solution that the company built deploys InfluxDB and Telegraf within its eSoftLink platform to provide real-time energy management, which allows usage visibility down to the appliance level. The solution is end-to-end — from hardware device, to backend platform, to mobile application. Here’s what drove them to build it and how it works.

Making smart energy meters even smarter

Achieving energy savings has become a major goal for both utility companies and consumers. Under Energy Efficiency Directive 2012/27/EU, the European Union (EU) set a target of 20% energy savings by 2020. Yet to save energy, it’s necessary to assess energy consumption and compare it to energy production.

Smart metering is one of the first steps to address challenges surrounding energy consumption. Smart meters allow consumers to make smarter decisions about their energy usage by providing real-time data on electricity and gas usage.

For example, France implemented the Linky program. Linky (smart meter), connects to a concentrator through Power Line Communication (PLC). This system uploads consumption information once a day (due to a small PLC tunnel and limited data rate) with 30-minute granularity. The concentrator uploads the data it collects from the meters, in GPRS, to the information system.

The utility company offers customers a web portal to track their energy consumption on a yearly, monthly, and daily basis. While this enables customers to make high-level comparisons (such as comparing summer and winter energy consumption), it doesn’t allow them to identify the devices that most contribute to overall consumption. Decreasing power consumption requires the ability to check consumption per device in real-time.

Energy consumption visibility at the appliance level

Major studies, like those conducted by National Science Research Institutes, Oxford University, and the American Council for an Energy-Efficient Economy, suggest that consumer access to real-time energy consumption feedback improves energy savings. Consumers are able to increase control over energy spend when equipped with the availability of in-depth data at the appliance level and a real-time feedback tool. “The more real-time the feedback tool, the highest the energy savings percentage at the end of the day by the consumer,” says Technical Marketing Engineer at eSoftThings Samuel Chevrier. This challenge is what drove eSoftThings to transform the smart meter into an IoT system using their eSoftLink IoT platform for their lead customer Total Direct Energie.

Energy meters generate consumption data every second and sends it over the IoT link to the backend platform. The smart meter data also sends the data to eSoftLink every second so that consumers can easily view and monitor their energy consumption.

In building this solution, eSoftThings realized that they needed a time series database for eSoftLink to store and process the large volume of time-stamped data generated by the smart meters. IoT data, after all, is time series data. Says Chevrier: “We wanted a solution that can support this data volume, which led us to deploying a time series database.”

InfluxDB to support large volumes of IoT data

eSoftThings chose the InfluxDB time series platform because it needed a performant solution with high write speeds and scalability, and something with easy-to-find documentation, community guidance, and long-term support.

To ensure that InfluxDB and Telegraf fit well into the system, eSoftThings performed load tests that recorded server activity as well as Telegraf and InfluxDB KPI’s. The test results confirmed that InfluxDB and Telegraf were the right time series platform for eSoftLink and that they had the right configuration.

With its implementation of InfluxDB (and its metrics collection agent Telegraf), eSoftThings is able to collect a large volume of time series data that provides value-added services while maintaining GDPR compliance. eSoftThings uses InfluxDB to store the time series data that its energy application collects from sensors, and uses this data to help monitor, alert, and predict in real-time.

How eSoftThings’ IoT real-time energy management system works

The eSoftLink IoT solution uses an in-house-built message broker to manage data flow. This includes Apache Zookeeper and Apache Kafka for handling huge data volumes quickly, InfluxDB for time series data, Telegraf for metrics collection, and Chart.js with PHP for dashboarding.

eSoftLink’s main purpose is to collect the data coming from all the IoT nodes connected to the smart meter and to present this data through visualization for:

• Customers on their smartphones to monitor their energy consumption

• The utility company to manage the fleet of devices connected to the platform

Platform functions include data storage, data analytics, Smart Contract (an application where people can view and select offers available from producers who want to sell their energy directly to consumers), and device management.

Technical architecture of eSoftLink IoT platform

eSoftLink architecture’s three main pillars are: IoT node, backend platform (with administration panels to manage data securely), and end user frontend (mobile/web application).

eSoftLink can connect to a wide range of IoT sensors and devices. Coupled with integrated over the air (OTA) and device management capabilities, it provides customers with a highly dynamic, flexible, and reactive tool to support their operations and business activities. The integration of a data analytics system and a messaging server provides users with real-time notifications and alerts on events of interest. eSoftLink also allows connection to external data lakes to enable long-term storage and cold-analysis of acquired data. This allows extracting maximum value from any dataset on the fly and over time.

Configuring InfluxDB to ensure GDPR compliance

To ensure compliance with GDPR (a regulation in EU law regarding data protection and privacy), eSoftThings implemented features that had an impact on the architecture and the design within the database. These features included the right to be forgotten, the right to consult, data portability, data log access, and anonymization.

While ensuring GDPR compliance, eSoftThings had to collect a lot of personal information to provide powerful insights to consumers and providers. They applied several database retention policies given the huge amount of data, taking advantage of InfluxDB’s flexible built-in retention policy functionality.

Smarter energy management decisions in real time

eSoftLink is a versatile and robust end-to-end IoT platform designed to work as a stand-alone on customer infrastructure, on the Edge, or in the Cloud. Its flexibility means it can serve as the integration platform between a provider’s devices, infrastructure, and the cloud service of their choice. eSoftLink provides real-time data collection, management, analytics, and reporting tools that allow enterprises to benefit from the data that they collect quickly and efficiently.

eSoftLink’s implementation for utility providers delivers real-time energy monitoring and reporting, with secure data storage and access. The eSoftLink stack offers a full range of REST APIs, so connecting frontend UIs, such as mobile phone applications and web browsers, is quick and easy.

The eSoftLink consumer application and provider application are very empowering because they provide previously unavailable information and visibility in real time.

The consumer mobile app provides real-time information about consumption down to the device level, thereby allowing consumers to control home energy use. Consumers can instantaneously see the consumption difference they can make, which motivates behavioral change.

The mobile app provides visualization and configuration options, as well as a home comparison tool, which shows the customer’s home consumption relative to similar households. It also shows consumption in real-time, power usage, and bill-to-date.

The provider UI empowers the utility company by allowing them to ensure they can quickly service and support their consumers. It allows easy, centralized configuration of platform and device parameters.

Provider UI visualizations include the number of active sessions, connections per day, total energy accumulated the previous day, instantaneous power, number of user accounts created, unique connections per day, and average time per session.

The provider UI allows the solution owner to consult any user ID in order to access details on whether the IoT node has been paired and connected, whether it is plugged into the smart meter, and the date of latest firmware update. The solution owner can view and visualize details regarding the power curve, energy, and spending for a selected period.

For the next iteration, eSoftThings is working on improving two recently deployed functionalities: adding blockchain with the capability to do Smart Contract and adding NILM capabilities.

To read the full story, download this real-time energy management case study.