Announcing the General Availability of Our New High-Performance Time Series Engine in InfluxDB Cloud

By Balaji Palani / Product, Use Cases
Feb 01, 2023

Back in October 2022, our Founder and CTO Paul Dix announced the limited release of InfluxDB IOx, our new database engine. After several months of beta testing, we’re excited to announce the next phase of our database engine: general availability. As of today, InfluxDB IOx releases to the rest of the world as the new and improved InfluxDB Cloud.

All time series workloads and all the performance

There are several improvements and updates that we’re thrilled to bring to a global user base. While InfluxDB has always excelled at handling metrics, the new engine can handle metrics, events, and traces with dramatic improvements in performance and efficiency.

What exactly does this mean? Let us go a little deeper.

• Metrics are essentially time series data polled at a regular interval from the same source. They are mostly quantitative and useful to understand the availability or performance of the specific service or device being monitored.

• Events are state changes triggered by some observed condition(s), e.g., deployed code, container crashes, HTTP 5XX errors, etc. Events can be semi-structured and typically contain more context as tags so they can be correlated with metrics to gain additional insights.

• Traces capture information to show the request propagation in a distributed system. Depending on the complexity of the application or service, traces can contain many spans which could be conceptually similar to a wide table.

Currently, there are no other time series datastores available that are optimized to index, store, and analyze metric, event, and trace data together in one place.

InfluxDB Cloud, with the new high-performance time series engine, can ingest, store, and analyze high volume and high velocity data in real-time, and derive metrics on the fly from high granularity events. This expands the number and variety of use cases that InfluxDB can handle, such as observability and distributed tracing, which rely on high cardinality data. Thanks to InfluxDB Cloud’s new columnar structure and compression abilities, the database can handle tag values of unbounded data with no drop in performance. Taken together, these improvements lead to real-time, in-depth insights. This is a huge development and we’re thrilled about all the benefits users will get from it.

Lower cost of ownership

Speaking of data compression, InfluxDB Cloud now uses Apache Parquet as its persistence format. Not only does this contribute to the drastic improvements in compression, but it also enables users to extend the value of their time series data by transmitting it to other systems and ecosystems that utilize Parquet. Perhaps most significant though is that Parquet’s compression means that users can save more data using less space on cloud object storage, like Amazon S3, which is very inexpensive compared to SSD or Elastic Storage Block, reducing storage costs.

Optimized for low latency queries

Ingesting and storing data in InfluxDB is only half the story. Being able to query and use all that data effectively is just as important. We optimized the new engine for performance thanks to several under-the-hood updates. While concepts like parallelism, pushdowns, and in-memory caching are critical aspects driving low latency query performance, what really matters for users is that fast, reliable queries improve user experiences.

Improving developer efficiency

At InfluxData, we talk a lot about how we meet developers where they are. That same sentiment continues with the new InfluxDB Cloud. The most significant development in this regard is the addition of native SQL queries, thanks to Apache DataFusion. Users can still use InfluxQL and Flux language for more advanced data processing, too. But the bottom line is that you can choose the query language that’s right for your needs and maximizes your productivity.

The fine print

With so much great stuff becoming available today, the big question is how to get it. The rollout of the new database engine will occur in stages. Today marks the first stage and it’s available on two AWS regions – Virginia (us-east-1) and Frankfurt (eu-central-1). As of today’s GA launch, everyone who signs up for InfluxDB Cloud in either of these regions will automatically have all the functionality of the new database engine.

When thinking about interoperability with other tools, we have native integrations ready to go at launch. These include Apache SuperSet, Grafana for data visualization, and Flight SQL for interoperability with compatible tools.

More coming soon

Again, this is the first phase of a much larger rollout. Keep an eye out in the coming weeks and months for expanded offerings with the new database engine. We plan to expand the number of regions and cloud providers running it and to update the new engine across our product suite, including InfluxDB Enterprise. We will announce additional regions and cloud provider support as they occur. This includes more AWS regions, as well as Microsoft Azure and Google Cloud.

Our current plan is to provide support for business intelligence tools such as Tableau and PowerBI. We also plan to provide migration tools for those who want to move from the TSM engine to the new database engine.

As of today, you can sign up for the new InfluxDB Cloud powered by IOx here.