Fluentd and Datadog Integration

Input and output integration overview

The Fluentd Input Plugin gathers metrics from Fluentd’s in_monitor plugin endpoint. It provides insights into various plugin metrics while allowing for custom configurations to reduce series cardinality.

This plugin writes to the Datadog Metrics API and requires an apikey which can be obtained for the account. This plugin supports the v1 API.

Integration details

Fluentd

This plugin gathers metrics from the Fluentd plugin endpoint provided by the in_monitor plugin. It reads data from the /api/plugin.json resource and allows exclusion of specific plugins based on their type.

Datadog

Datadog metric names are formed by joining the Telegraf metric name and the field key with a . character.

Field values are converted to floating point numbers. Strings and floats that cannot be sent over JSON, namely NaN and Inf, are ignored.

Setting rate_interval to non-zero will convert count metrics to rate and divide its value by this interval before submitting to Datadog. This allows Telegraf to submit metrics alongside Datadog agents when their rate intervals are the same (Datadog defaults to 10s). Note that this only supports metrics ingested via inputs.statsd given the dependency on the metric_type tag it creates. There is only support for counter metrics, and count values from timing and histogram metrics.

Configuration

Fluentd

[[inputs.fluentd]]
  ## This plugin reads information exposed by fluentd (using /api/plugins.json endpoint).
  ##
  ## Endpoint:
  ## - only one URI is allowed
  ## - https is not supported
  endpoint = "http://localhost:24220/api/plugins.json"

  ## Define which plugins have to be excluded (based on "type" field - e.g. monitor_agent)
  exclude = [
    "monitor_agent",
    "dummy",
  ]

Datadog

[[outputs.datadog]]
  ## Datadog API key
  apikey = "my-secret-key"

  ## Connection timeout.
  # timeout = "5s"

  ## Write URL override; useful for debugging.
  ## This plugin only supports the v1 API currently due to the authentication
  ## method used.
  # url = "https://app.datadoghq.com/api/v1/series"

  ## Set http_proxy
  # use_system_proxy = false
  # http_proxy_url = "http://localhost:8888"

  ## Override the default (none) compression used to send data.
  ## Supports: "zlib", "none"
  # compression = "none"

  ## When non-zero, converts count metrics submitted by inputs.statsd
  ## into rate, while dividing the metric value by this number.
  ## Note that in order for metrics to be submitted simultaenously alongside
  ## a Datadog agent, rate_interval has to match the interval used by the
  ## agent - which defaults to 10s
  # rate_interval = 0s

Input and output integration examples

Fluentd

1. Basic Configuration: Set up the Fluentd Input Plugin to gather metrics from your Fluentd instance’s monitoring endpoint, ensuring you are able to track performance and usage statistics.
2. Excluding Plugins: Use the exclude option to ignore specific plugins’ metrics that are not necessary for your monitoring needs, streamlining data collection and focusing on what matters.
3. Custom Plugin ID: Implement the @id parameter in your Fluentd configuration to maintain a consistent plugin_id, which helps avoid issues with high series cardinality during frequent restarts.

Datadog

1. Basic Metric Submission: Utilize the Datadog Output Plugin to transmit metrics from your Telegraf instance to Datadog. By configuring the apikey and enabling necessary metrics, you can easily monitor application performance over time.
2. Debugging Write URL: In cases where you need to troubleshoot your metric submissions, you can override the default write URL with a custom endpoint to debug the metrics being sent, ensuring that they are reaching the correct destination.
3. Proxy Configuration: If your network setup requires routing through a proxy for outgoing requests, use the http_proxy_url option to set the appropriate proxy. This allows for seamless integration in restrictive network environments.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.