Webinar Recap: Build an Edge-to-Cloud Architecture Using MQTT and InfluxDB
By
Jessica Wachtel /
Developer
Oct 27, 2023
Navigate to:
Industrial IoT (IIoT) machines and sensors generate valuable time series data. It’s impossible to derive the insights necessary to inform decisions as a company to produce or operate more efficiently without sending operational technology (OT) data to informational technology (IT) systems. Unstable connections, the volume and velocity of time series ingests, and machine controls with weak security precautions are some of the challenges in need of a solution before data can travel safely from OT to IT. The webinar Build an Edge-to-Cloud Architecture Using MQTT and InfluxDB 3.0 discusses and demos a framework that helps overcome those challenges.
Introduction to MQTT
MQTT is an asynchronous publish/subscribe communication protocol used extensively in IIoT. Rather than two components connecting to one another to exchange information directly, the MQTT broker (a third-party server) sits between the components and facilitates communication. The MQTT protocol helps an IIoT device push data to the MQTT broker. Any application subscribing to that broker can consume the information it needs.
MQTT supports edge-driven development because the MQTT protocol pushes data from devices at the edge. This differs from protocols where the exchange of information begins when the application server requests information.
HiveMQ
The company HiveMQ provides an MQTT platform that aids in the development of MQTT applications. The HiveMQ team recently introduced HiveMQ Edge, an embedded lightweight MQTT broker designed for edge deployment. The edge-to-cloud architecture discussed in detail during the webinar focuses on HiveMQ Edge and HiveMQ Cloud. HiveMQ has a suite of MQTT products with both HiveMQ-managed and self-managed options.
InfluxDB
InfluxDB is purpose-built for time series data and its workloads. Recently launched, InfluxDB 3.0 has 45x better write throughput and faster queries for recent data. InfluxDB 3.0 supports unlimited cardinality and is 100x faster when querying against high cardinality data. InfluxDB 3.0 persists data as Apache Parquet which can lead to 90%+ savings in storage costs. For more performance benchmarks, check out this benchmarking paper. Parquet is part of the Apache open data ecosystem, which enables InfluxDB to connect to some of the best tools for statistical analysis, artificial intelligence (AI), and machine learning (ML).
The edge-to-cloud architecture
The high-level diagram below depicts how HiveMQ Edge and HiveMQ Cloud seamlessly integrate the OT and IT environments.
Image credit: HiveMQ
The gray rectangles in the Edge/OT environment represent different machines. It’s best practice to deploy an MQTT broker or multiple MQTT brokers at the edge rather than connect the machines themselves to the cloud. HiveMQ Edge has several legacy protocol adaptors and can provide MQTT connectivity to machines and equipment that normally wouldn’t integrate with MQTT. This includes Modbus and OPC-UA. HiveMQ Edge allows you to contextualize and normalize data. Once the local data is ready, the MQTT brokers push the data to a centralized server via an MQTT bridge. This action creates a unified namespace. A unified namespace is another term for an event hub because this type of messaging architecture is also event-driven.
Not only is InfluxDB necessary for analysis and AI/ML integration, but it’s also the only permanent data store included in this design. The MQTT broker is not a time series database and therefore, not a replacement. However, data is safe with an MQTT broker in the event of a failed connection. If the connection between MQTT components or between MQTT components and InfluxDB goes down, the MQTT broker stores the data until a working connection is available. Upon restoration the MQTT broker flushes the data to its correct destination.
The demo includes the use of the now depreciated Native Subscription feature. InfluxDB users preferred using the Telegraf MQTT input plugin over Native Subscriptions. Telegraf is InfluxDB’s open source data collection agent and has more features for MQTT ingestion than Native Subscriptions.
Watch the webinar!
This brief blog post scratches the surface of how to send machine data from the job site to InfluxDB using HiveMQ. Edge-to-Cloud Architecture with MQTT & InfluxDB discusses the hierarchy of OT technologies and challenges of using each, includes an in-depth demo that covers how to build with and use InfluxDB and HiveMQ starting at 12:56, and a Q&A beginning at 41:46. Have questions after watching the webinar? Ask ‘em in our community slack!
Ready to start building? Open your free InfluxDB Cloud Serverless account.
