In an era where energy costs are on the rise and environmental, social, and governance (ESG) reporting guidelines compel companies to disclose detailed information about their consumption, technology —particularly the Internet of Things (IoT) — plays a pivotal role in driving sustainability efforts. A key step in reducing energy costs is making consumption transparent, which requires a digital representation of your energy meters. This blog post outlines a straightforward approach to connecting energy meters to AWS, leveraging a lightweight data pipeline to enhance transparency and sustainability.Energy Monitoring - How to connect energy meters to AWS
Raphael Heinrich – VP Commercial Management IoT Products
In an era where energy costs are on the rise and environmental, social, and governance (ESG) reporting guidelines compel companies to disclose detailed information about their consumption, technology —particularly the Internet of Things (IoT) — plays a pivotal role in driving sustainability efforts. A key step in reducing energy costs is making consumption transparent, which requires a digital representation of your energy meters. This blog post outlines a straightforward approach to connecting energy meters to AWS, leveraging a lightweight data pipeline to enhance transparency and sustainability.Introduction
The increasing cost of energy and the stringent ESG reporting guidelines are pushing companies to adopt measures that can help manage and reduce their energy consumption. IoT technology offers a promising solution by enabling a more sustainable approach to energy management. According to a report by Transforma Insights, IoT can significantly impact sustainability by reducing energy, fuel, and water consumption across various sectors, including residential, transportation, buildings, and agriculture, among others. An important first step towards achieving this is by making energy consumption transparent through the digitalization of energy meters.
Step One: Make Your Energy Meters Smart
Connect Existing and New Meters to AWS
One effective way to digitalize your energy meters is by using devices like Modbus Cloud Connect by grandcentrix, an AWS-certified device fully compatible with AWS IoT Core. This device allows remote monitoring of machines, sensors and meters via a Modbus interface, utilizing Narrowband-IoT and LTE-M for data transmission to the cloud, independent of local IT infrastructure. The Modbus Cloud Connect package includes all necessary components, from hardware to cloud adapter, facilitating a plug’n’work solution that leverages Vodafone’s Global IoT Connectivity for efficient, reliable and secure data transfer. You can read more about the integration here.
IoT Devices and AWS IoT Core Integration
Once connected, IoT devices can communicate with the AWS IoT Core directly thanks to the underlying MQTT broker. Optionally, the data can be filtered by an IoT Rule so that only so that only relevant messages are processed in the later steps. In our case, we are using an IoT Rule to exclude heartbeat events of devices as we are only interested in telemetry data, i.e. power consumption. For analytics use cases added at a later step of the pipeline, we want to persist the data in an AWS S3 buckets. This is done via the AWS Kinesis Firehose service, which collects events over a 15 minute window before writing them as JSON lines format to S3 (raw_data bucket).
Step Two: Process Data on AWS
The goal is to develop a lightweight data pipeline for easy data receipt, storage, visualization, and analysis, focusing on cost-effectiveness and flexibility. AWS Lambda Functions, with its serverless design and pricing, makes it an ideal solution for implementing this pipeline. When a file is written to the S3 raw_data bucket, an AWS Lambda Function is triggered and parses its JSON content, applies a schema and appends it to a delta table stored in another S3 bucket. From here, we use the medallion architecture naming scheme for our buckets, which means that we start with a bronze table S3 bucket (raw data). The writing process again triggers another AWS Lambda Function which excludes outliers, performs gapfilling, calculating actual consumption as difference from energy meter readings and write the data again into a silver table S3 bucket (conformed and cleansed).
Step Three: Visualize Energy Data on AWS QuickSight
Amazon Quicksight, a cloud-native, serverless, business intelligence (BI) service, offers a wide range of visuals and basic forecasting capabilities. But before we can import the data from the silver table, we must convert it into an AWS Glue Table. Afterwards, we can use Amazon Athena to query the data directly from S3 using SQL. QuickSight allows for the creation of alerts for threshold breaches and the sharing of dashboards with the analytics team. Due to the low amount of data in this project (less than one record per second), it is a cost-effective solution for this use case. Also, QuickSight’s ease of integration with Glue tables allows greater flexibility on data visualization, making it a valuable tool in energy management.
Step Four: Establish an Energy Consumption Forecasting Using Machine Learning
An additional layer of this pipeline could include advances analytics, such as basic forecasting or anomaly detection, running as scheduled tasks on a daily basis. For simpler use-cases, one could as well use AWS Lambda functions for the processing, while for ML and AI tasks services like AWS SageMaker would be a better choice. By comparing predicted and actual consumption, valuable insights can be gained that can lead to a more sustainable use of limited resources.
Hannover Messe Showcase:
If you want to learn more about these approaches, you can visit us at Hannover Messe (22. - 26. Apr. 2024) on the AWS booth (Hall 15, Booth D76) in the Sustainability Area. We will be demonstrating the described technology in live use and there will also be a talk in the AWS Conference area together with our customer Eastron.
If you want to learn more about these approaches, you can visit us at Hannover Messe (22. - 26. Apr. 2024) on the AWS booth (Hall 15, Booth D76) in the Sustainability Area. We will be demonstrating the described technology in live use and there will also be a talk in the AWS Conference area together with our customer Eastron.
Conclusion
In conclusion, by making energy consumption transparent and leveraging AWS’s powerful cloud infrastructure, companies can take a significant step towards sustainability and cost reduction. Whether you choose to build your own solution or opt for our ready-made option, the integration of IoT technology and AWS services offers a promising path to achieving your energy management goals.
About the Authors:
Dr. Timon Schmelzer: While doing his PhD in the field of experimental particle physics, Dr. Timon Schmelzer already gained a lot experience processing big amounts of data and applying machine learning techniques to them. Now, about five years later he is working as a Senior Data Scientist at grandcentrix, co-operating with various medium-sized companies always with the primary goal of driving value creation from data.
Raphael Heinrich: With over 10 years of experience in the IoT industry, Raphael Heinrich is currently responsible for sales engineering, commercial support, sales enablement, and marketing within grandcentrix’s IoT product portfolio. He has a broad spectrum of experience, having served in various roles in digitalization for 14 years, including 10 years in leadership positions. He sees himself as a bridge-builder between technology and business, adept at merging these worlds for genuine transformation.
Marcel Bruns: After the graduation as Master of Mathematics, Marcel Bruns joined Vodafone Germany as Trainee in the field of Data Science. On his journey through the traineeship Marcel joined different teams with data background, gaining skills in data science, data engineering, machine learning and cloud computing. He took the chance to join grandcentrix as subsidiary company of Vodafone Germany for a 4 month station.