3 ways InfluxDB OSS Complements InfluxDB Cloud

By Al Sargent / Product, Use Cases, Developer
Nov 16, 2020

As you may know, InfluxDB is available in multiple forms, including InfluxDB Open Source and InfluxDB Cloud. Customers will sometimes ask us, “If I’m using InfluxDB Cloud, is there any reason I should also consider using InfluxDB open source?”

It’s a fair question. InfluxDB Cloud has several capabilities not in InfluxDB OSS: InfluxDB Cloud is serverless so there are no servers to manage, it can automatically accommodate traffic surges, and we’re continually updating it to automatically deliver you the latest features.

We continually hear from customers that they prefer InfluxDB Cloud because it frees them from tedious database administration tasks so they can focus on their code. Even so, there are a few use cases where an InfluxDB Cloud customer can use InfluxDB Open Source to help manage their time series data. Let’s explore.

InfluxDB development environment

You can install InfluxDB OSS on your local machine to write code anywhere without having to worry about a network connection. During these days of shelter-in-place, you might have network issues due to Zoom-induced bandwidth saturation. Post-pandemic, when we can return to cafes, trains, and airplanes, it’ll be handy to have a database that easily runs on a laptop.

The other day, I drove my kid to go surfing in a remote spot on the California coast without cell coverage. While I watched him from the car, I was able to easily import sample data from a local CSV file into my local InfluxDB instance, then write some Flux code to query it. Coding from the beach – one of the silver linings of shifting to a remote work environment!

Because the InfluxDB UI, CLI, API, and client libraries between InfluxDB OSS and Cloud are practically identical, you can always shift resources like Flux scripts, InfluxDB Templates, InfluxDB Dashboards, Tasks, Alerts, and Grafana dashboards from OSS to Cloud (and vice-versa) with no modifications, facilitating a smooth DevOps pipeline.

The differences between InfluxDB Open Source and Cloud are rare, and highlighted in our documentation’s left-hand navigation.

We also make it easy to switch your Telegraf agents, client code using the InfluxDB client libraries, and InfluxDB CLI from pointing at a local InfluxDB open source instance, to an InfluxDB Cloud instance. Just change your InfluxDB Open Source URL from localhost:8086 to the appropriate InfluxDB Cloud URL, update your org, token, and bucket, and you’re good to go.

When using the CLI, you can create connection configurations, and easily switch between configurations as you move from Open Source to Cloud and back. And, we just added a -c flag to set your profile on the fly.

InfluxDB edge processing

A common pattern in IoT is where people set up Telegraf to dual-write to both a local InfluxDB OSS instance on an edge device, as well as a central InfluxDB Cloud instance. This edge device might be as small as an Arduino board or as large as a windmill. Because InfluxDB OSS is a single lightweight binary, it can easily be deployed on devices with minimal compute resources.

In these situations, the edge instance of InfluxDB OSS often holds full-resolution telemetry for just the edge device, while the central InfluxDB Cloud instance holds telemetry across all devices. The degree to which InfluxDB Cloud’s telemetry is downsampled is often a function of business requirements, as well as connection reliability and bandwidth from the edge devices.

With this approach, folks responsible for managing the local environment – which would be a ferry, oil rig, farmland, etc. – have full telemetry to troubleshoot issues, without having to worry about connectivity to a central cloud instance. And folks in headquarters can analyze trends and issues across an entire IoT fleet using InfluxDB Cloud.

For example, one of our customers runs offshore oil rigs. All of the various time series metrics that they are gathering are stored in a local InfluxDB OSS instance and the operators of the rig have access to the information through dashboards. This information provides critical near real time visibility into what is occurring on the rig itself.

For some of the equipment being monitored, there is a desire to have access to this data – at a lower resolution than is being captured and stored locally – at the corporate headquarters. The corporate HQ folks can analyze the data from all of the rigs and look at the similarities and differences between the operational characteristics of the equipment in the aggregate. This allows for analyses like predictive maintenance to be performed and sent back to the rig operators.

Because InfluxDB OSS and Cloud share the same UI, API, and CLI, and both use Flux, teammates in the field and central office can easily share results and findings with each other.

To implement this pattern, you’ll need to dual-write to both InfluxDB Cloud and InfluxDB OSS to have a local backup. To do this, you’d put text similar to what’s below into your Telegraf configuration file (typically called telegraf.conf):

# Write to InfluxDB open source
[[outputs.influxdb_v2]] # For InfluxDB 2.0
 urls = ["http://127.0.0.1:8086"]
token = "your-token"
organization = "your-organization-name"
bucket = "your-bucket-name"

# Write to InfluxDB Cloud
[[outputs.influxdb_v2]]
# Write to AWS Oregon instance.
# Other cloud URLs: https://docs.influxdata.com/influxdb/cloud/reference/regions/
urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"]
token = "your-token"
organization = "email-you-use-to-sign-into-influxdb-cloud"
bucket = "your-bucket-name"

With this configuration, every write goes to both your on-prem InfluxDB OSS instance and InfluxDB Cloud instance, at the same time. Because of this, data in both instances is kept in sync, up-to-the-second; there’s no batch process to periodically run backups.

Now that InfluxDB 0SS 2.0 is GA, look for us to make further engineering investments in this area of edge processing to simplify and support edge-to-cloud connection across these editions.

Air-gapped monitoring

But what about those cases where you can’t dual-write, because there’s no Internet connection. These include servers and equipment that are disconnected from the Internet (or “air-gapped”) for security purposes and thus can’t send monitoring telemetry to InfluxDB Cloud. For instance, power plants are often disconnected to prevent a wide range of threat vectors.

If any of your servers or equipment are air-gapped, you can install InfluxDB OSS on them and use it to collect monitoring data in order to provide local observability and control. However, if you need higher levels of scalability or reliability than a single node of InfluxDB OSS can deliver, you should consider InfluxDB Enterprise. InfluxDB Enterprise provides horizontal scale-out and a high availability configuration.

So how does this usage of InfluxDB OSS complement InfluxDB Cloud? Because Cloud and OSS share the same API, CLI, web user interface, and client libraries, you can easily share assets like Dashboards, Tasks, and Alerts between your air-gapped InfluxDB OSS instances, and your InfluxDB Cloud instances. For example, if your company has standardized on Kubernetes, you build one set of k8s monitoring dashboards used at both your air-gapped and Internet-connected locations.

InfluxDB Cloud compared to InfluxDB OSS

Here’s a detailed comparison of how InfluxDB Cloud compares to InfluxDB open source.

Conclusion

At InfluxDB, we provide you with multiple options for deploying our database, including lightweight, self-managed InfluxDB Open Source that you can deploy anywhere, and fully-managed, serverless InfluxDB Cloud that frees you from database infrastructure concerns so you have more time to focus on your code.

We’re also grateful to our open source community of nearly 400 contributors to InfluxDB, who bring innovation to the shared InfluxDB Open Source and Cloud codebase. With them, we’re able to provide all of you with a range of options to suit your needs.

To get started with the patterns above, download InfluxDB Open Source and sign up for a free InfluxDB Cloud account. Then check out our getting started guide and feel free to engage our ever-helpful InfluxDB community, Slack, and GitHub repository for questions. Enjoy!