VMware vSphere and Datadog Integration

Input and output integration overview

The VMware vSphere Telegraf plugin provides a means to collect metrics from VMware vCenter servers, allowing for comprehensive monitoring and management of virtual resources in a vSphere environment.

The Datadog Telegraf Plugin enables the submission of metrics to the Datadog Metrics API, facilitating efficient monitoring and data analysis through a reliable metric ingestion process.

Integration details

VMware vSphere

This plugin connects to VMware vSphere servers to gather a variety of metrics from virtual environments, enabling efficient monitoring and management of virtual resources. It interfaces with the vSphere API to collect statistics regarding clusters, hosts, resource pools, VMs, datastores, and vSAN entities, presenting them in a format suitable for analysis and visualization. The plugin is particularly valuable for administrators who manage VMware-based infrastructures, as it helps to track system performance, resource usage, and operational issues in real-time. By aggregating data from multiple sources, the plugin empowers users with insights that facilitate informed decision-making regarding resource allocation, troubleshooting, and ensuring optimal system performance. Additionally, the support for secret-store integration allows secure handling of sensitive credentials, promoting best practices in security and compliance assessments.

Datadog

This plugin writes to the Datadog Metrics API, enabling users to send metrics for monitoring and performance analysis. By utilizing the Datadog API key, users can configure the plugin to establish a connection with Datadog’s v1 API. The plugin supports various configuration options including connection timeouts, HTTP proxy settings, and data compression methods, ensuring adaptability to different deployment environments. The ability to transform count metrics into rates enhances the integration of Telegraf with Datadog agents, particularly beneficial for applications that rely on real-time performance metrics.

Configuration

VMware vSphere

[[inputs.vsphere]]
  vcenters = [ "https://vcenter.local/sdk" ]
  username = "[email protected]"
  password = "secret"

  vm_metric_include = [
    "cpu.demand.average",
    "cpu.idle.summation",
    "cpu.latency.average",
    "cpu.readiness.average",
    "cpu.ready.summation",
    "cpu.run.summation",
    "cpu.usagemhz.average",
    "cpu.used.summation",
    "cpu.wait.summation",
    "mem.active.average",
    "mem.granted.average",
    "mem.latency.average",
    "mem.swapin.average",
    "mem.swapinRate.average",
    "mem.swapout.average",
    "mem.swapoutRate.average",
    "mem.usage.average",
    "mem.vmmemctl.average",
    "net.bytesRx.average",
    "net.bytesTx.average",
    "net.droppedRx.summation",
    "net.droppedTx.summation",
    "net.usage.average",
    "power.power.average",
    "virtualDisk.numberReadAveraged.average",
    "virtualDisk.numberWriteAveraged.average",
    "virtualDisk.read.average",
    "virtualDisk.readOIO.latest",
    "virtualDisk.throughput.usage.average",
    "virtualDisk.totalReadLatency.average",
    "virtualDisk.totalWriteLatency.average",
    "virtualDisk.write.average",
    "virtualDisk.writeOIO.latest",
    "sys.uptime.latest",
  ]

  host_metric_include = [
    "cpu.coreUtilization.average",
    "cpu.costop.summation",
    "cpu.demand.average",
    "cpu.idle.summation",
    "cpu.latency.average",
    "cpu.readiness.average",
    "cpu.ready.summation",
    "cpu.swapwait.summation",
    "cpu.usage.average",
    "cpu.usagemhz.average",
    "cpu.used.summation",
    "cpu.utilization.average",
    "cpu.wait.summation",
    "disk.deviceReadLatency.average",
    "disk.deviceWriteLatency.average",
    "disk.kernelReadLatency.average",
    "disk.kernelWriteLatency.average",
    "disk.numberReadAveraged.average",
    "disk.numberWriteAveraged.average",
    "disk.read.average",
    "disk.totalReadLatency.average",
    "disk.totalWriteLatency.average",
    "disk.write.average",
    "mem.active.average",
    "mem.latency.average",
    "mem.state.latest",
    "mem.swapin.average",
    "mem.swapinRate.average",
    "mem.swapout.average",
    "mem.swapoutRate.average",
    "mem.totalCapacity.average",
    "mem.usage.average",
    "mem.vmmemctl.average",
    "net.bytesRx.average",
    "net.bytesTx.average",
    "net.droppedRx.summation",
    "net.droppedTx.summation",
    "net.errorsRx.summation",
    "net.errorsTx.summation",
    "net.usage.average",
    "power.power.average",
    "storageAdapter.numberReadAveraged.average",
    "storageAdapter.numberWriteAveraged.average",
    "storageAdapter.read.average",
    "storageAdapter.write.average",
    "sys.uptime.latest",
  ]

  datacenter_metric_include = [] ## if omitted or empty, all metrics are collected
  datacenter_metric_exclude = [ "*" ] ## Datacenters are not collected by default.

  vsan_metric_include = [] ## if omitted or empty, all metrics are collected
  vsan_metric_exclude = [ "*" ] ## vSAN are not collected by default.

  separator = "_"
  max_query_objects = 256
  max_query_metrics = 256
  collect_concurrency = 1
  discover_concurrency = 1
  object_discovery_interval = "300s"
  timeout = "60s"
  use_int_samples = true
  custom_attribute_include = []
  custom_attribute_exclude = ["*"]
  metric_lookback = 3
  ssl_ca = "/path/to/cafile"
  ssl_cert = "/path/to/certfile"
  ssl_key = "/path/to/keyfile"
  insecure_skip_verify = false
  historical_interval = "5m"
  disconnected_servers_behavior = "error"
  use_system_proxy = true
  http_proxy_url = ""

Datadog

[[outputs.datadog]]
  ## Datadog API key
  apikey = "my-secret-key"

  ## Connection timeout.
  # timeout = "5s"

  ## Write URL override; useful for debugging.
  ## This plugin only supports the v1 API currently due to the authentication
  ## method used.
  # url = "https://app.datadoghq.com/api/v1/series"

  ## Set http_proxy
  # use_system_proxy = false
  # http_proxy_url = "http://localhost:8888"

  ## Override the default (none) compression used to send data.
  ## Supports: "zlib", "none"
  # compression = "none"

  ## When non-zero, converts count metrics submitted by inputs.statsd
  ## into rate, while dividing the metric value by this number.
  ## Note that in order for metrics to be submitted simultaenously alongside
  ## a Datadog agent, rate_interval has to match the interval used by the
  ## agent - which defaults to 10s
  # rate_interval = 0s

Input and output integration examples

VMware vSphere

1. Dynamic Resource Allocation: Utilize this plugin to monitor resource usage across a fleet of VMs and automatically adjust resource allocations based on performance metrics. This scenario could involve triggering scaling actions in real time based on CPU and memory usage metrics collected from the vSphere API, ensuring optimal performance and cost-efficiency.

2. Capacity Planning and Forecasting: Leverage the historical metrics gathered from vSphere to conduct capacity planning. Analyzing the trends of CPU, memory, and storage usage over time helps administrators anticipate when additional resources will be needed, avoiding outages and ensuring that the virtual infrastructure can handle growth.

3. Automated Alerting and Incident Response: Integrate this plugin with alerting tools to set up automated notifications based on the metrics gathered. For example, if the CPU usage on a host exceeds a specified threshold, it could trigger alerts and automatically initiate predefined remediation steps, such as migrating VMs to less utilized hosts.

4. Performance Benchmarking Across Clusters: Use the metrics collected to compare the performance of clusters in different vCenters. This benchmarking provides insights into which cluster configurations yield the best resource efficiency and can guide future infrastructure enhancements.

Datadog

1. Real-Time Infrastructure Monitoring: Use the Datadog plugin to monitor server metrics in real-time by sending CPU usage and memory statistics directly to Datadog. This integration allows IT teams to visualize and analyze system performance metrics in a centralized dashboard, enabling proactive response to any emerging issues, such as resource bottlenecks or server overloads.

2. Application Performance Tracking: Leverage this plugin to submit application-specific metrics, such as request counts and error rates, to Datadog. By integrating with application monitoring tools, teams can correlate infrastructure metrics with application performance, providing insights that enable them to optimize code performance and improve user experience.

3. Anomaly Detection in Metrics: Configure the Datadog plugin to send metrics that can trigger alerts and notifications based on unusual patterns detected by Datadog’s machine learning features. This proactive monitoring helps teams swiftly react to potential outages or performance degradation before customers are impacted.

4. Integrating with Cloud Services: By utilizing the Datadog plugin to send metrics from cloud resources, IT teams can gain visibility into cloud application performance. Monitoring metrics like latency and error rates helps with ensuring service-level agreements (SLAs) are met and also assists in optimizing resource allocation across cloud environments.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.