Simplifying Multi-Node Setups with InfluxDB 3 Enterprise Modes

By Peter Barnett / Developer
Apr 02, 2025

Navigate to:

As your time series data grows, managing increasing workloads can quickly become a headache. High data ingestion rates, numerous (and complex) queries, intensive processing tasks, and routine maintenance like data compaction often compete for limited resources. This leads to unpredictable performance and slower response times, and common solutions often introduce operational complexity. Achieving reliable, high-performance database operations requires strategic resource management and precise workload isolation.

InfluxDB 3 Enterprise addresses these challenges with specialized node “modes” that simplify multi-node deployments. With distinct modes tailored to handle specific types of workloads, InfluxDB enables seamless scaling, higher availability, and consistent performance for your time series data infrastructure. With Enterprise’s new diskless architecture, data is solely persisted to object storage, creating fault-tolerant setups without information-sharing between nodes. The only requirement is for all nodes to share the same object store connection. This ensures your system can efficiently handle increasing data volumes, minimize downtime, and deliver reliable insights in real-time, even as demand fluctuates—all without increasing operational complexity.

Understanding server modes

InfluxDB 3 Enterprise supports five distinct server modes that let you precisely tailor node functionality to your specific workload requirements:

  • ingest: Handles incoming writes of time series data without contention from query or compaction processes.
  • query: Dedicated mode optimized exclusively for handling read operations and queries, ensuring fast response times and consistent performance. Scales horizontally by adding nodes and pointing them at object storage.
  • compact: Runs compaction processes to optimize storage efficiency, manage storage costs, and maintain high query performance by reducing data fragmentation. You can only have one node running in compact mode per cluster.
  • process: Specializes in handling internal processing engine tasks and is ideal for custom transformations and real-time analytics operations.
  • all: Combines ingest, query, compact, and processing functionality in one node. Perfect for small-scale deployments or testing but may impact performance under heavy workloads.

This granular approach to node configuration lets you scale individual workloads independently, isolate tasks to dedicated infrastructure, and achieve more predictable, manageable, and efficient performance across your InfluxDB cluster.

Building a basic two-node setup

Let’s start by building the simplest multi-node configuration. We’ll have one node running all modes and a second node running ingest, query, and process. We do not specify compact on the second node since we can only have one compactor per cluster. This setup gives us basic high availability for ingestion and querying.

Node 1 (All):

influxdb3 serve \
  --cluster-id cluster01 \
  --node-id node01 \
  --mode all \
  --object-store s3 \
  --bucket influxdb-storage \
  --aws-access-key-id [AWS_ACCESS_KEY_ID] \
  --aws-secret-access-key [AWS_SECRET_ACCESS_KEY]

Node 2 (Ingest, Query):

influxdb3 serve \
  --cluster-id cluster01 \
  --node-id node02 \
  --mode ingest,query,process \
  --object-store s3 \
  --bucket influxdb-storage \
  --aws-access-key-id [AWS_ACCESS_KEY_ID] \
  --aws-secret-access-key [AWS_SECRET_ACCESS_KEY]

2-Node Setup for Basic HA

With this setup, if one node fails, the other seamlessly handles both writes and queries, ensuring uninterrupted operation.

Enhance performance with dedicated compaction

While the two-node setup gives you basic HA, it has some limitations; for starters, one node has to handle compaction, so your nodes won’t serve reads and writes with equal performance. For optimal query and ingestion performance, separate compaction tasks from your primary nodes. This setup avoids resource contention and maintains smooth operations, in addition to creating a single compactor that you can vertically scale on its own to handle more intense workloads.

Nodes 1 & 2 (Ingest, Query, Process):

Configure each ingest/query node the same way, with a simple adjustment to the --node-id option.

influxdb3 serve \
  --cluster-id cluster01 \
  --node-id [NODE_ID] \
  --mode ingest,query,process \
  --object-store s3 \
  --bucket influxdb-storage \
  --aws-access-key-id [AWS_ACCESS_KEY_ID] \
  --aws-secret-access-key [AWS_SECRET_ACCESS_KEY]

Node 3 (Compactor)

To set up a dedicated compactor, we simply set the --mode to compact.

influxdb3 serve \
  --cluster-id cluster01 \
  --node-id compactor01 \
  --mode compact \
  --object-store s3 \
  --bucket influxdb-storage \
  --aws-access-key-id [AWS_ACCESS_KEY_ID] \
  --aws-secret-access-key [AWS_SECRET_ACCESS_KEY]

3-Node Setup with Dedicated Compactor

In this setup, we’ve isolated the compactor. This ensures that Node A and Node B can serve reads and writes with predictable and similar performance, assuming similar architecture and load distribution. If either A or B fails, compaction will not be impacted, preventing workload build-up on restart.

Full workload isolation with dedicated nodes

We’ve now created a compaction-isolated HA setup but there’s more we can do to achieve even better results. Nodes 1 and 2 both serve reads and writes, which can cause inconsistent performance. For example, if queries come in at manual intervals, ingestion will not be as performant while queries are being served.

For best results, we can leverage full workload isolation by assigning dedicated nodes for ingestion, querying, compaction, and the Processing Engine. This approach optimizes resource utilization and delivers predictable performance.

Nodes 1 & 2 (Ingest):

These nodes handle exclusively write operations. Again, they have the same startup command running in ingest mode, with a change to the --node-id (e.g., ingest_node_1, ingest_node_2).

influxdb3 serve \
  --cluster-id cluster01 \
  --node-id [NODE_ID] \
  --mode ingest \
  --object-store s3 \
  --bucket influxdb-storage \
  --aws-access-key-id [AWS_ACCESS_KEY_ID] \
  --aws-secret-access-key [AWS_SECRET_ACCESS_KEY]

Node 3 & 4 (Query):

These nodes exclusively handle query operations. They run solely in query mode, with a slight change to the --node-id (e.g., query_node_1, query_node_2).

influxdb3 serve \
  --cluster-id cluster01 \
  --node-id [NODE_ID] \
  --mode query \
  --object-store s3 \
  --bucket influxdb-storage \
  --aws-access-key-id [AWS_ACCESS_KEY_ID] \
  --aws-secret-access-key [AWS_SECRET_ACCESS_KEY]

Node 5 (Compactor):

We’ll run a single compactor alongside our ingest and query nodes.

influxdb3 serve \
  --cluster-id cluster01 \
  --node-id compact01 \
  --mode compact \
  --object-store s3 \
  --bucket influxdb-storage \
  --aws-access-key-id [AWS_ACCESS_KEY_ID] \
  --aws-secret-access-key [AWS_SECRET_ACCESS_KEY]

Node 6 (Processing Engine):

Finally, we’ll create a dedicated node for the Processing Engine. This ensures all ETL runs in a dedicated environment. Similar to the compactor, this also enables vertical scaling to improve performance solely on processing power, rather than also scaling ingest/query capacity.

influxdb3 serve \
  --cluster-id cluster01 \
  --node-id process01 \
  --mode process \
  --object-store s3 \
  --bucket influxdb-storage \
  --aws-access-key-id [AWS_ACCESS_KEY_ID] \
  --aws-secret-access-key [AWS_SECRET_ACCESS_KEY]

6-Node Setup with Full Workload Isolation

That’s it! With only a few changes to each node’s configuration, you can run a workload-isolated setup without complex overhead. Each instance will work with the others seamlessly, without having to send constant communication streams back and forth, all from the simple configuration pattern.

Summary

We intentionally built InfluxDB 3 Enterprise to simplify the complexities of multi-node deployments. These specialized server modes offer powerful flexibility so you can scale effectively, ensure high availability, and isolate intense processing tasks within their own environments.

The above examples show how you can start small and slowly build a robust setup that fully isolates your workload across multiple nodes. By strategically deploying nodes based on their specific functions, you can optimize performance, reliability, and operational simplicity.

To explore further, check out the InfluxDB 3 Enterprise documentation, where you can learn about its configurability, best practices for HA setups, the Processing Engine, and more.