OpenStack and Azure Data Explorer Integration
Powerful performance with an easy integration, powered by Telegraf, the open source data connector built by InfluxData.
5B+
Telegraf downloads
#1
Time series database
Source: DB Engines
1B+
Downloads of InfluxDB
2,800+
Contributors
Table of Contents
Powerful Performance, Limitless Scale
Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.
See Ways to Get Started
Input and output integration overview
This plugin collects metrics from essential OpenStack services, facilitating the monitoring and management of cloud infrastructures.
The Azure Data Explorer plugin allows integration of metrics collection with Azure Data Explorer, enabling users to analyze and query their telemetry data efficiently. With this plugin, users can configure ingestion settings to suit their needs and leverage Azure’s powerful analytical capabilities.
Integration details
OpenStack
The OpenStack plugin allows users to collect performance metrics from various OpenStack services such as CINDER, GLANCE, HEAT, KEYSTONE, NEUTRON, and NOVA. It supports multiple OpenStack APIs to fetch critical metrics related to these services, enabling comprehensive monitoring and management of cloud resources. As organizations increasingly adopt OpenStack for their cloud infrastructure, this plugin plays a vital role in providing insights into resource usage, availability, and performance across the cloud environment. Configuration options allow for customized polling intervals and filtering unwanted tags to optimize performance and cardinals.
Azure Data Explorer
The Azure Data Explorer plugin allows users to write metrics, logs, and time series data collected from various Telegraf input plugins into Azure Data Explorer, Azure Synapse, and Real-Time Analytics in Fabric. This integration serves as a bridge, allowing applications and services to monitor their performance metrics or logs efficiently. Azure Data Explorer is optimized for analytics over large volumes of diverse data types, making it an excellent choice for real-time analytics and monitoring solutions in cloud environments. The plugin empowers users to configure metrics ingestion based on their requirements, define table schemas dynamically, and set various ingestion methods while retaining flexibility regarding roles and permissions needed for database operations. This supports scalable and secure monitoring setups for modern applications that utilize cloud services.
Configuration
OpenStack
[[inputs.openstack]]
## The recommended interval to poll is '30m'
## The identity endpoint to authenticate against and get the service catalog from.
authentication_endpoint = "https://my.openstack.cloud:5000"
## The domain to authenticate against when using a V3 identity endpoint.
# domain = "default"
## The project to authenticate as.
# project = "admin"
## User authentication credentials. Must have admin rights.
username = "admin"
password = "password"
## Available services are:
## "agents", "aggregates", "cinder_services", "flavors", "hypervisors",
## "networks", "nova_services", "ports", "projects", "servers",
## "serverdiagnostics", "services", "stacks", "storage_pools", "subnets",
## "volumes"
# enabled_services = ["services", "projects", "hypervisors", "flavors", "networks", "volumes"]
## Query all instances of all tenants for the volumes and server services
## NOTE: Usually this is only permitted for administrators!
# query_all_tenants = true
## output secrets (such as adminPass(for server) and UserID(for volume)).
# output_secrets = false
## Amount of time allowed to complete the HTTP(s) request.
# timeout = "5s"
## HTTP Proxy support
# http_proxy_url = ""
## Optional TLS Config
# tls_ca = /path/to/cafile
# tls_cert = /path/to/certfile
# tls_key = /path/to/keyfile
## Use TLS but skip chain & host verification
# insecure_skip_verify = false
## Options for tags received from Openstack
# tag_prefix = "openstack_tag_"
# tag_value = "true"
## Timestamp format for timestamp data received from Openstack.
## If false format is unix nanoseconds.
# human_readable_timestamps = false
## Measure Openstack call duration
# measure_openstack_requests = false
Azure Data Explorer
[[outputs.azure_data_explorer]]
## The URI property of the Azure Data Explorer resource on Azure
## ex: endpoint_url = https://myadxresource.australiasoutheast.kusto.windows.net
endpoint_url = ""
## The Azure Data Explorer database that the metrics will be ingested into.
## The plugin will NOT generate this database automatically, it's expected that this database already exists before ingestion.
## ex: "exampledatabase"
database = ""
## Timeout for Azure Data Explorer operations
# timeout = "20s"
## Type of metrics grouping used when pushing to Azure Data Explorer.
## Default is "TablePerMetric" for one table per different metric.
## For more information, please check the plugin README.
# metrics_grouping_type = "TablePerMetric"
## Name of the single table to store all the metrics (Only needed if metrics_grouping_type is "SingleTable").
# table_name = ""
## Creates tables and relevant mapping if set to true(default).
## Skips table and mapping creation if set to false, this is useful for running Telegraf with the lowest possible permissions i.e. table ingestor role.
# create_tables = true
## Ingestion method to use.
## Available options are
## - managed -- streaming ingestion with fallback to batched ingestion or the "queued" method below
## - queued -- queue up metrics data and process sequentially
# ingestion_type = "queued"
Input and output integration examples
OpenStack
-
Cross-Cloud Management: Leverage the OpenStack plugin to monitor and manage multiple OpenStack clouds from a single Telegraf instance. By aggregating metrics across different clouds, organizations can gain insights into resource utilization and optimize their cloud architecture for cost and performance.
-
Automated Scaling Based on Metrics: Integrate the metrics gathered from OpenStack into an automated scaling solution. For example, if the plugin detects that a specific service’s performance is degraded, it can trigger auto-scaling rules to launch additional instances, ensuring that system performance remains optimal under varying workloads.
-
Performance Monitoring Dashboard: Use data collected by the OpenStack Telegraf plugin to power real-time monitoring dashboards. This setup provides visualizations of key metrics from OpenStack services, enabling stakeholders to quickly identify trends, pinpoint issues, and make data-driven decisions in managing their cloud infrastructure.
-
Reporting and Analysis of Service Availability: By utilizing the metrics collected from various OpenStack services, teams can generate detailed reports on service availability and performance over time. This information can help identify recurring issues, improve service delivery, and make informed decisions regarding changes in infrastructure or service configuration.
Azure Data Explorer
-
Real-Time Monitoring Dashboard: By integrating metrics from various services into Azure Data Explorer using this plugin, organizations can build comprehensive dashboards that reflect real-time performance metrics. This allows teams to respond proactively to performance issues and optimize system health without delay.
-
Centralized Log Management: Utilize Azure Data Explorer to consolidate logs from multiple applications and services. By utilizing the plugin, organizations can streamline their log analysis processes, making it easier to search, filter, and derive insights from historical data accumulated over time.
-
Data-Driven Alerting Systems: Enhance monitoring capabilities by configuring alerts based on metrics sent via this plugin. Organizations can set thresholds and automate incident responses, significantly reducing downtime and improving the reliability of critical operations.
-
Machine Learning Model Training: By leveraging the data sent to Azure Data Explorer, organizations can perform large-scale analytics and prepare the data for feeding into machine learning models. This plugin enables the structuring of data that can subsequently be used for predictive analytics, leading to enhanced decision-making capabilities.
Feedback
Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.
Powerful Performance, Limitless Scale
Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.
See Ways to Get Started
Related Integrations
Related Integrations
HTTP and InfluxDB Integration
The HTTP plugin collects metrics from one or more HTTP(S) endpoints. It supports various authentication methods and configuration options for data formats.
View IntegrationKafka and InfluxDB Integration
This plugin reads messages from Kafka and allows the creation of metrics based on those messages. It supports various configurations including different Kafka settings and message processing options.
View IntegrationKinesis and InfluxDB Integration
The Kinesis plugin allows for reading metrics from AWS Kinesis streams. It supports multiple input data formats and offers checkpointing features with DynamoDB for reliable message processing.
View Integration