Amazon CloudWatch and Dynatrace Integration

Input and output integration overview

This plugin will pull Metric Statistics from Amazon CloudWatch, streamlining the process of monitoring and analyzing AWS resources.

The Dynatrace plugin allows users to send metrics collected by Telegraf directly to Dynatrace for monitoring and analysis. This integration enhances the observability of systems and applications, providing valuable insights into performance and operational health.

Integration details

Amazon CloudWatch

The Amazon CloudWatch Plugin allows users to pull detailed metric statistics from Amazon’s CloudWatch service. As a monitoring solution, CloudWatch enables users to track various metrics related to AWS resources and applications, facilitating improved operational and performance insights. The plugin uses a structured authentication method that prioritizes security and flexibility through a combination of STS (Security Token Service), shared credentials, environment variables, and EC2 instance profiles, ensuring robust access control to AWS resources. Key features include the ability to define specific metric namespaces, aggregated periods for metrics, and optional inclusion of linked accounts for cross-account monitoring. A significant aspect of this plugin is its capacity to handle both sparse and dense metric formats, allowing for varied output structures depending on user preference. Thus, it supports versatile use cases in cloud monitoring and analytics by providing comprehensive, timely data directly from CloudWatch.

Dynatrace

The Dynatrace plugin for Telegraf facilitates the transmission of metrics to the Dynatrace platform via the Dynatrace Metrics API V2. This plugin can function in two modes: it can run alongside the Dynatrace OneAgent, which automates authentication, or it can operate in a standalone configuration that requires manual specification of the URL and API token for environments without a OneAgent. The plugin primarily reports metrics as gauges unless explicitly configured to treat certain metrics as delta counters using the available config options. This feature empowers users to customize the behavior of metrics sent to Dynatrace, harnessing the robust capabilities of the platform for comprehensive performance monitoring and observability. It’s crucial for users to ensure compliance with version requirements for both Dynatrace and Telegraf, thereby optimizing compatibility and performance when integrating with the Dynatrace ecosystem.

Configuration

Amazon CloudWatch

[[inputs.cloudwatch]]
  region = "us-east-1"
  # access_key = ""
  # secret_key = ""
  # token = ""
  # role_arn = ""
  # web_identity_token_file = ""
  # role_session_name = ""
  # profile = ""
  # shared_credential_file = ""
  # include_linked_accounts = false
  # endpoint_url = ""
  # use_system_proxy = false
  # http_proxy_url = "http://localhost:8888"
  period = "5m"
  delay = "5m"
  interval = "5m"
  #recently_active = "PT3H"
  # cache_ttl = "1h"
  namespaces = ["AWS/ELB"]
  # metric_format = "sparse"
  # ratelimit = 25
  # timeout = "5s"
  # batch_size = 500
  # statistic_include = ["average", "sum", "minimum", "maximum", sample_count]
  # statistic_exclude = []
  # [[inputs.cloudwatch.metrics]]
  #  names = ["Latency", "RequestCount"]
  #  [[inputs.cloudwatch.metrics.dimensions]]
  #    name = "LoadBalancerName"
  #    value = "p-example"

Dynatrace

[[outputs.dynatrace]]
  ## For usage with the Dynatrace OneAgent you can omit any configuration,
  ## the only requirement is that the OneAgent is running on the same host.
  ## Only setup environment url and token if you want to monitor a Host without the OneAgent present.
  ##
  ## Your Dynatrace environment URL.
  ## For Dynatrace OneAgent you can leave this empty or set it to "http://127.0.0.1:14499/metrics/ingest" (default)
  ## For Dynatrace SaaS environments the URL scheme is "https://{your-environment-id}.live.dynatrace.com/api/v2/metrics/ingest"
  ## For Dynatrace Managed environments the URL scheme is "https://{your-domain}/e/{your-environment-id}/api/v2/metrics/ingest"
  url = ""

  ## Your Dynatrace API token.
  ## Create an API token within your Dynatrace environment, by navigating to Settings > Integration > Dynatrace API
  ## The API token needs data ingest scope permission. When using OneAgent, no API token is required.
  api_token = ""

  ## Optional prefix for metric names (e.g.: "telegraf")
  prefix = "telegraf"

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Optional flag for ignoring tls certificate check
  # insecure_skip_verify = false

  ## Connection timeout, defaults to "5s" if not set.
  timeout = "5s"

  ## If you want metrics to be treated and reported as delta counters, add the metric names here
  additional_counters = [ ]

  ## In addition or as an alternative to additional_counters, if you want metrics to be treated and
  ## reported as delta counters using regular expression pattern matching
  additional_counters_patterns = [ ]

  ## NOTE: Due to the way TOML is parsed, tables must be at the END of the
  ## plugin definition, otherwise additional config options are read as part of the
  ## table

  ## Optional dimensions to be added to every metric
  # [outputs.dynatrace.default_dimensions]
  # default_key = "default value"

Input and output integration examples

Amazon CloudWatch

1. Cross-Account Monitoring: Utilize this plugin to monitor resources across multiple AWS accounts by enabling the include_linked_accounts option. This scenario allows companies managing multiple AWS accounts to aggregate metrics into a central monitoring dashboard, providing a unified view of all metrics while ensuring secure data access and compliance through proper role management.

2. Dynamic Alerting System: Integrate this plugin with alerting tools to create an automated system that triggers alerts based on defined thresholds for CloudWatch metrics. For instance, if latency metrics exceed specified limits, alerts can be sent to relevant teams, enabling proactive responses to performance issues and reducing downtime.

3. Cost Management Dashboard: Use the metrics gathered from the plugin to build a cost management dashboard that visualizes AWS service usage metrics over time. By correlating these metrics with billing data, organizations can identify high-cost services and take informed actions to optimize their resource usage and spending.

4. Performance Benchmarking for Applications: Leverage the metrics collected from applications running on AWS to perform performance benchmarks. For example, by tracking latency and request count metrics for an ELB, developers can assess the impact of application changes on its performance, making data-driven decisions for optimization.

Dynatrace

1. Cloud Infrastructure Monitoring: Utilize the Dynatrace plugin to monitor a cloud infrastructure setup, feeding real-time metrics from Telegraf into Dynatrace. This integration provides a holistic view of resource utilization, application performance, and system health, enabling proactive responses to performance issues across various cloud environments.

2. Custom Application Performance Metrics: Implement custom application-specific metrics by configuring the Dynatrace output plugin to send tailored metrics from Telegraf. By leveraging additional counters and dimension options, development teams can gain insights that are precisely aligned with their application’s operational requirements, allowing for targeted optimization efforts.

3. Multi-Environment Metrics Management: For organizations running multiple Dynatrace environments (e.g., production, staging, and development), use this plugin to manage metrics for all environments from a single Telegraf instance. With proper configuration of endpoints and API tokens, teams can maintain consistent monitoring practices throughout the SDLC, ensuring that performance anomalies are detected early in the development process.

4. Automated Alerting Based on Metrics Changes: Integrate the Dynatrace output plugin with an alerting mechanism that triggers notifications when specific metrics exceed defined thresholds. This scenario involves configuring additional counters to monitor crucial application performance indicators, enabling swift remediation actions to maintain service availability and user satisfaction.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.