TL;DR InfluxDB Tech Tips: IoT Data from the Edge to Cloud with Flux

By Anais Dotis-Georgiou / Product, Use Cases, Developer
Dec 13, 2021

Navigate to:


When it comes to writing data to InfluxDB, you have a lot of options. You can:

  • Write data to an OSS instance on your edge or fog device
  • Write data from your IoT device directly to your InfluxDB Cloud
  • Write data to an OSS instance of InfluxDB on your edge device and push data to InfluxDB Cloud ad hoc

The last bullet is the most powerful and flexible way of maintaining and managing your fleet of IoT devices. That architecture offers you several advantages including:

  • The ability to perform tasks that prepare and clean your data before sending it to InfluxDB Cloud. This not only helps you create a well-organized data pipeline but also ensure that your data has been standardized across IoT devices
  • The flexibility to only send select data to InfluxDB Cloud, like anomalies that were flagged in the OSS instance
  • The option to isolate task workloads and keep them close to the data source
  • The benefit of an additional layer of security between your edge devices, InfluxDB Cloud instance, and end-customers

Architecture drawing of the last bullet. Sensors write data to an OSS instance of InfluxDB at the edge which in turn write data to InfluxDB Cloud.Write data to InfluxDB at edge to cloud

Using the to() function to consolidate data from OSS to InfluxDB Cloud

One way you can achieve this architecture is by using the to() function. The to() function now supports the option to write data from an OSS v2.1 instance to an InfluxDB Cloud instance.

Using the to() function in Flux - an example

Example of using the to() function to write data from an OSS instance to InfluxDB Cloud

On the right-hand side of the above dashboard, the user is querying an OSS instance. On the right-hand side, an InfluxDB Cloud instance displays the results of the Flux script in the OSS instance after the data is written to InfluxDB Cloud with the to() function.

To write data with the to() function from InfluxDB Cloud:

  1. Query for your data.
from(bucket: "RoomSensors")
|> range(start: -10m)
|> filter(fn: (r) => r["_measurement"] == "temperature")
|> filter(fn: (r) => r["_field"] == "temperature")
|> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
  1. Specify the destination host URL, orgID, token, and bucket ID.
|> to(host: https://us-west-2-1.aws.cloud2.influxdata.com, 
        orgID:   04xxx00,
        token: 0xxx==, 
        bucketID: 43xxx00)

You’ll need to have an InfluxDB Cloud account first to try this out. A free tier account will work just fine for this. You can easily sign up for an account here. After you sign up, you’ll need to:

Limitations of and alternatives to the to() function

The to() function offers an easy way to write data from an edge device to the Cloud. However, it comes with several limitations:

  • Sent over HTTP
  • No built-in functionality for failure handling
  • No built-in functionality for batching, retries, or parallelization

The to() function should only be used to consolidate data from OSS to Cloud if you meet the following conditions:

  • You intend on downsampling your data first before writing it to Cloud to limit the size of the request.
  • You have a small amount of devices or are writing a relatively small amount of data this way. You can't use the to() function for large workloads because the data volume might cause a network failure. There also isn't functionality built in for failure handling.
  • You aren't trying to overcome writing a large amount of data with micro-batching and generating a really high request count.

While the to() function has some limitations, there is another option for writing IoT data from OSS to InfluxDB Cloud. You can use the mqtt.to() function to write data to a MQTT broker first if you want to handle larger workloads.

An acceptable usage of to() in a downsampling task

Now that we understand the limitations of the to() function, let’s highlight an example where it could be successful. For this example, we’ll assume we want to downsample and write data on a schedule with a task.

option task = {
    name: "downsample and consolidate task",
    every: 1d,
    offset: 10m,
}
import "http"
import "influxdata/influxdb/secrets"

myToken = "Token ${secrets.get(key: "cloud-token")}"
myUrl= "https://us-west-2-1.aws.cloud2.influxdata.com"
myOrgID = "Token ${secrets.get(key: "cloud-org")}"
myBucketID = "Token ${secrets.get(key: "cloud-bucket")}"

from(bucket: "sensorData")
|> range(start: -task.every)
|> filter(fn: (r) =>
(r["_measurement"] == "sensors"))
|> mean()
|> to(host: myurl, 
        orgID:   myOrgID,
        token: myToken, 
        bucketID: myBucketID)

This task aggregates all of the data from one measurement into one daily mean before writing it to InfluxDB Cloud.

Further reading

While this post aims to provide a comprehensive overview of the to() function, the following resources might also interest you:

  1. How to Consolidate OSS Data into a Cloud Account: This post describes how to share data from multiple OSS instances to InfluxDB Cloud using the http.to() function instead of the to() function. This blog is basically a precursor to the one you're currently reading.
  2. Getting started with InfluxDB tasks: This documentation describes how to get started with InfluxDB tasks in case you've never created one before.
  3. TL;DR InfluxDB Tech Tips – Using Tasks and Checks for Monitoring with InfluxDB: This post is a tutorial on how to create tasks with the UI as well as checks (a special type of task).
  4. Writing Tasks and Setting Up Alerts for InfluxDB Cloud: This tutorial demonstrates how to create tasks with the Flux plugin for VS code for those who are UI adverse.
  5. Using the Flux VS Code Extension for IoT Application Development. This post highlights some new features of the Flux plugin for VS code that allow you to easily convert a Flux query into a task.
  6. Trending Aggregate Values by Downsampling with InfluxDB: This tutorial highlights how you can use tasks to perform a variety of aggregations.

Final thoughts on the to() function with InfluxDB

I hope this InfluxDB Tech Tips post inspires you to take advantage of the to() function to consolidate your data from edge devices to InfluxDB Cloud. If you are using Flux and need help, please ask for some in our community site or Slack channel. If you’re developing a cool IoT application on top of InfluxDB, we’d love to hear about it, so make sure to share your story! Additionally, please share your thoughts, concerns, or questions in the comments section. We’d love to get your feedback and help you with any problems you run into!