Hashicorp Nomad and Dynatrace Integration

Input and output integration overview

This plugin allows users to collect metrics from Hashicorp Nomad agents in distributed environments.

The Dynatrace plugin allows users to send metrics collected by Telegraf directly to Dynatrace for monitoring and analysis. This integration enhances the observability of systems and applications, providing valuable insights into performance and operational health.

Integration details

Hashicorp Nomad

The Hashicorp Nomad input plugin is designed to gather metrics from every Nomad agent within a cluster. By deploying Telegraf on each node, it can connect to the local Nomad agent, typically available at ‘http://127.0.0.1:4646’. With this setup, users can systematically collect and monitor metrics related to the performance and status of their Nomad environment, ensuring they maintain a healthy and efficient cluster operational state. This plugin enables visibility into the operational aspects of Nomad, which is essential for maintaining reliable cloud infrastructure.

Dynatrace

The Dynatrace plugin for Telegraf facilitates the transmission of metrics to the Dynatrace platform via the Dynatrace Metrics API V2. This plugin can function in two modes: it can run alongside the Dynatrace OneAgent, which automates authentication, or it can operate in a standalone configuration that requires manual specification of the URL and API token for environments without a OneAgent. The plugin primarily reports metrics as gauges unless explicitly configured to treat certain metrics as delta counters using the available config options. This feature empowers users to customize the behavior of metrics sent to Dynatrace, harnessing the robust capabilities of the platform for comprehensive performance monitoring and observability. It’s crucial for users to ensure compliance with version requirements for both Dynatrace and Telegraf, thereby optimizing compatibility and performance when integrating with the Dynatrace ecosystem.

Configuration

Hashicorp Nomad

[[inputs.nomad]]
  ## URL for the Nomad agent
  # url = "http://127.0.0.1:4646"

  ## Set response_timeout (default 5 seconds)
  # response_timeout = "5s"

  ## Optional TLS Config
  # tls_ca = /path/to/cafile
  # tls_cert = /path/to/certfile
  # tls_key = /path/to/keyfile

Dynatrace

[[outputs.dynatrace]]
  ## For usage with the Dynatrace OneAgent you can omit any configuration,
  ## the only requirement is that the OneAgent is running on the same host.
  ## Only setup environment url and token if you want to monitor a Host without the OneAgent present.
  ##
  ## Your Dynatrace environment URL.
  ## For Dynatrace OneAgent you can leave this empty or set it to "http://127.0.0.1:14499/metrics/ingest" (default)
  ## For Dynatrace SaaS environments the URL scheme is "https://{your-environment-id}.live.dynatrace.com/api/v2/metrics/ingest"
  ## For Dynatrace Managed environments the URL scheme is "https://{your-domain}/e/{your-environment-id}/api/v2/metrics/ingest"
  url = ""

  ## Your Dynatrace API token.
  ## Create an API token within your Dynatrace environment, by navigating to Settings > Integration > Dynatrace API
  ## The API token needs data ingest scope permission. When using OneAgent, no API token is required.
  api_token = ""

  ## Optional prefix for metric names (e.g.: "telegraf")
  prefix = "telegraf"

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Optional flag for ignoring tls certificate check
  # insecure_skip_verify = false

  ## Connection timeout, defaults to "5s" if not set.
  timeout = "5s"

  ## If you want metrics to be treated and reported as delta counters, add the metric names here
  additional_counters = [ ]

  ## In addition or as an alternative to additional_counters, if you want metrics to be treated and
  ## reported as delta counters using regular expression pattern matching
  additional_counters_patterns = [ ]

  ## NOTE: Due to the way TOML is parsed, tables must be at the END of the
  ## plugin definition, otherwise additional config options are read as part of the
  ## table

  ## Optional dimensions to be added to every metric
  # [outputs.dynatrace.default_dimensions]
  # default_key = "default value"

Input and output integration examples

Hashicorp Nomad

1. Cluster Health Monitoring: Use the Hashicorp Nomad plugin to aggregate metrics across all nodes in a Nomad deployment. By monitoring health metrics such as allocation status, job performance, and resource utilization, operations teams can gain insights into the overall health of their deployment, quickly identify and resolve issues, and optimize resource allocation based on real-time data.

2. Performance Analytics for Job Execution: Leverage the metrics provided by Nomad to analyze job execution times and resource consumption. This use case enables developers to adjust job parameters effectively, optimize task performance, and illustrate trends over time, ultimately leading to increased efficiency and reduced costs in resource allocation.

3. Alerting on Critical Conditions: Implement alerting mechanisms based on metrics scraped from Nomad agents. By setting thresholds for critical metrics like CPU usage or failed job allocations, teams can proactively respond to potential issues before they escalate, ensuring higher uptime and reliability for applications running on the Nomad platform.

4. Integration with Visualization Tools: Use the data collected by the Hashicorp Nomad plugin to feed into visualization tools for real-time dashboards. This setup allows teams to monitor cluster workloads, job states, and system performance at a glance, facilitating better decision-making and strategic planning based on visual insights into the Nomad environment.

Dynatrace

1. Cloud Infrastructure Monitoring: Utilize the Dynatrace plugin to monitor a cloud infrastructure setup, feeding real-time metrics from Telegraf into Dynatrace. This integration provides a holistic view of resource utilization, application performance, and system health, enabling proactive responses to performance issues across various cloud environments.

2. Custom Application Performance Metrics: Implement custom application-specific metrics by configuring the Dynatrace output plugin to send tailored metrics from Telegraf. By leveraging additional counters and dimension options, development teams can gain insights that are precisely aligned with their application’s operational requirements, allowing for targeted optimization efforts.

3. Multi-Environment Metrics Management: For organizations running multiple Dynatrace environments (e.g., production, staging, and development), use this plugin to manage metrics for all environments from a single Telegraf instance. With proper configuration of endpoints and API tokens, teams can maintain consistent monitoring practices throughout the SDLC, ensuring that performance anomalies are detected early in the development process.

4. Automated Alerting Based on Metrics Changes: Integrate the Dynatrace output plugin with an alerting mechanism that triggers notifications when specific metrics exceed defined thresholds. This scenario involves configuring additional counters to monitor crucial application performance indicators, enabling swift remediation actions to maintain service availability and user satisfaction.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.