VMware vSphere and Loki Integration
Powerful performance with an easy integration, powered by Telegraf, the open source data connector built by InfluxData.
5B+
Telegraf downloads
#1
Time series database
Source: DB Engines
1B+
Downloads of InfluxDB
2,800+
Contributors
Table of Contents
Powerful Performance, Limitless Scale
Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.
See Ways to Get Started
Input and output integration overview
The VMware vSphere Telegraf plugin provides a means to collect metrics from VMware vCenter servers, allowing for comprehensive monitoring and management of virtual resources in a vSphere environment.
The Loki plugin allows users to send logs to Loki for aggregation and querying, leveraging Loki’s efficient storage capabilities.
Integration details
VMware vSphere
This plugin connects to VMware vSphere servers to gather a variety of metrics from virtual environments, enabling efficient monitoring and management of virtual resources. It interfaces with the vSphere API to collect statistics regarding clusters, hosts, resource pools, VMs, datastores, and vSAN entities, presenting them in a format suitable for analysis and visualization. The plugin is particularly valuable for administrators who manage VMware-based infrastructures, as it helps to track system performance, resource usage, and operational issues in real-time. By aggregating data from multiple sources, the plugin empowers users with insights that facilitate informed decision-making regarding resource allocation, troubleshooting, and ensuring optimal system performance. Additionally, the support for secret-store integration allows secure handling of sensitive credentials, promoting best practices in security and compliance assessments.
Loki
This Loki plugin integrates with Grafana Loki, a powerful log aggregation system. By sending logs in a format compatible with Loki, this plugin allows for efficient storage and querying of logs. Each log entry is structured in a key-value format where keys represent the field names and values represent the corresponding log information. The sorting of logs by timestamp ensures that the log streams maintain chronological order when queried through Loki. This plugin’s support for secrets makes it easier to manage authentication parameters securely, while options for HTTP headers, gzip encoding, and TLS configuration enhance the adaptability and security of log transmission, fitting various deployment needs.
Configuration
VMware vSphere
[[inputs.vsphere]]
vcenters = [ "https://vcenter.local/sdk" ]
username = "[email protected]"
password = "secret"
vm_metric_include = [
"cpu.demand.average",
"cpu.idle.summation",
"cpu.latency.average",
"cpu.readiness.average",
"cpu.ready.summation",
"cpu.run.summation",
"cpu.usagemhz.average",
"cpu.used.summation",
"cpu.wait.summation",
"mem.active.average",
"mem.granted.average",
"mem.latency.average",
"mem.swapin.average",
"mem.swapinRate.average",
"mem.swapout.average",
"mem.swapoutRate.average",
"mem.usage.average",
"mem.vmmemctl.average",
"net.bytesRx.average",
"net.bytesTx.average",
"net.droppedRx.summation",
"net.droppedTx.summation",
"net.usage.average",
"power.power.average",
"virtualDisk.numberReadAveraged.average",
"virtualDisk.numberWriteAveraged.average",
"virtualDisk.read.average",
"virtualDisk.readOIO.latest",
"virtualDisk.throughput.usage.average",
"virtualDisk.totalReadLatency.average",
"virtualDisk.totalWriteLatency.average",
"virtualDisk.write.average",
"virtualDisk.writeOIO.latest",
"sys.uptime.latest",
]
host_metric_include = [
"cpu.coreUtilization.average",
"cpu.costop.summation",
"cpu.demand.average",
"cpu.idle.summation",
"cpu.latency.average",
"cpu.readiness.average",
"cpu.ready.summation",
"cpu.swapwait.summation",
"cpu.usage.average",
"cpu.usagemhz.average",
"cpu.used.summation",
"cpu.utilization.average",
"cpu.wait.summation",
"disk.deviceReadLatency.average",
"disk.deviceWriteLatency.average",
"disk.kernelReadLatency.average",
"disk.kernelWriteLatency.average",
"disk.numberReadAveraged.average",
"disk.numberWriteAveraged.average",
"disk.read.average",
"disk.totalReadLatency.average",
"disk.totalWriteLatency.average",
"disk.write.average",
"mem.active.average",
"mem.latency.average",
"mem.state.latest",
"mem.swapin.average",
"mem.swapinRate.average",
"mem.swapout.average",
"mem.swapoutRate.average",
"mem.totalCapacity.average",
"mem.usage.average",
"mem.vmmemctl.average",
"net.bytesRx.average",
"net.bytesTx.average",
"net.droppedRx.summation",
"net.droppedTx.summation",
"net.errorsRx.summation",
"net.errorsTx.summation",
"net.usage.average",
"power.power.average",
"storageAdapter.numberReadAveraged.average",
"storageAdapter.numberWriteAveraged.average",
"storageAdapter.read.average",
"storageAdapter.write.average",
"sys.uptime.latest",
]
datacenter_metric_include = [] ## if omitted or empty, all metrics are collected
datacenter_metric_exclude = [ "*" ] ## Datacenters are not collected by default.
vsan_metric_include = [] ## if omitted or empty, all metrics are collected
vsan_metric_exclude = [ "*" ] ## vSAN are not collected by default.
separator = "_"
max_query_objects = 256
max_query_metrics = 256
collect_concurrency = 1
discover_concurrency = 1
object_discovery_interval = "300s"
timeout = "60s"
use_int_samples = true
custom_attribute_include = []
custom_attribute_exclude = ["*"]
metric_lookback = 3
ssl_ca = "/path/to/cafile"
ssl_cert = "/path/to/certfile"
ssl_key = "/path/to/keyfile"
insecure_skip_verify = false
historical_interval = "5m"
disconnected_servers_behavior = "error"
use_system_proxy = true
http_proxy_url = ""
Loki
[[outputs.loki]]
## The domain of Loki
domain = "https://loki.domain.tld"
## Endpoint to write api
# endpoint = "/loki/api/v1/push"
## Connection timeout, defaults to "5s" if not set.
# timeout = "5s"
## Basic auth credential
# username = "loki"
# password = "pass"
## Additional HTTP headers
# http_headers = {"X-Scope-OrgID" = "1"}
## If the request must be gzip encoded
# gzip_request = false
## Optional TLS Config
# tls_ca = "/etc/telegraf/ca.pem"
# tls_cert = "/etc/telegraf/cert.pem"
# tls_key = "/etc/telegraf/key.pem"
## Sanitize Tag Names
## If true, all tag names will have invalid characters replaced with
## underscores that do not match the regex: ^[a-zA-Z_:][a-zA-Z0-9_:]*.
# sanitize_label_names = false
## Metric Name Label
## Label to use for the metric name to when sending metrics. If set to an
## empty string, this will not add the label. This is NOT suggested as there
## is no way to differentiate between multiple metrics.
# metric_name_label = "__name"
Input and output integration examples
VMware vSphere
-
Dynamic Resource Allocation: Utilize this plugin to monitor resource usage across a fleet of VMs and automatically adjust resource allocations based on performance metrics. This scenario could involve triggering scaling actions in real time based on CPU and memory usage metrics collected from the vSphere API, ensuring optimal performance and cost-efficiency.
-
Capacity Planning and Forecasting: Leverage the historical metrics gathered from vSphere to conduct capacity planning. Analyzing the trends of CPU, memory, and storage usage over time helps administrators anticipate when additional resources will be needed, avoiding outages and ensuring that the virtual infrastructure can handle growth.
-
Automated Alerting and Incident Response: Integrate this plugin with alerting tools to set up automated notifications based on the metrics gathered. For example, if the CPU usage on a host exceeds a specified threshold, it could trigger alerts and automatically initiate predefined remediation steps, such as migrating VMs to less utilized hosts.
-
Performance Benchmarking Across Clusters: Use the metrics collected to compare the performance of clusters in different vCenters. This benchmarking provides insights into which cluster configurations yield the best resource efficiency and can guide future infrastructure enhancements.
Loki
-
Centralized Logging for Microservices: Utilize the Loki plugin to gather logs from multiple microservices running in a Kubernetes cluster. By directing logs to a centralized Loki instance, developers can monitor, search, and analyze logs from all services in one place, facilitating easier troubleshooting and performance monitoring. This setup streamlines operations and supports rapid response to issues across distributed applications.
-
Real-Time Log Anomaly Detection: Combine Loki with monitoring tools to analyze log outputs in real-time for unusual patterns that could indicate system errors or security threats. Implementing anomaly detection on log streams enables teams to proactively identify and respond to incidents, thereby improving system reliability and enhancing security postures.
-
Enhanced Log Processing with Gzip Compression: Configure the Loki plugin to utilize gzip compression for log transmission. This approach can reduce bandwidth usage and improve transmission speeds, especially beneficial in environments where network bandwidth may be a constraint. It’s particularly useful for high-volume logging applications where every byte counts and performance is critical.
-
Multi-Tenancy Support with Custom Headers: Leverage the ability to add custom HTTP headers to segregate logs from different tenants in a multi-tenant application environment. By using the Loki plugin to send different headers for each tenant, operators can ensure proper log management and compliance with data isolation requirements, making it a versatile solution for SaaS applications.
Feedback
Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.
Powerful Performance, Limitless Scale
Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.
See Ways to Get Started
Related Integrations
Related Integrations
HTTP and InfluxDB Integration
The HTTP plugin collects metrics from one or more HTTP(S) endpoints. It supports various authentication methods and configuration options for data formats.
View IntegrationKafka and InfluxDB Integration
This plugin reads messages from Kafka and allows the creation of metrics based on those messages. It supports various configurations including different Kafka settings and message processing options.
View IntegrationKinesis and InfluxDB Integration
The Kinesis plugin allows for reading metrics from AWS Kinesis streams. It supports multiple input data formats and offers checkpointing features with DynamoDB for reliable message processing.
View Integration