Supervisor and Azure Data Explorer Integration

Input and output integration overview

This plugin gathers information about processes running under Supervisor using the XML-RPC API.

The Azure Data Explorer plugin allows integration of metrics collection with Azure Data Explorer, enabling users to analyze and query their telemetry data efficiently. With this plugin, users can configure ingestion settings to suit their needs and leverage Azure’s powerful analytical capabilities.

Integration details

Supervisor

The Supervisor plugin for Telegraf is designed to collect metrics about processes managed by the Supervisor process control system using its XML-RPC API. The plugin is able to track various metrics, including process states and uptime, and provides options for configuring which metrics to collect through include or exclude lists. This integration is particularly useful for monitoring applications running under Supervisor, providing insights into their operational status and performance metrics. A minimum tested Supervisor version is 3.3.2, and it is recommended to secure the HTTP server with basic authentication for better security.

Azure Data Explorer

The Azure Data Explorer plugin allows users to write metrics, logs, and time series data collected from various Telegraf input plugins into Azure Data Explorer, Azure Synapse, and Real-Time Analytics in Fabric. This integration serves as a bridge, allowing applications and services to monitor their performance metrics or logs efficiently. Azure Data Explorer is optimized for analytics over large volumes of diverse data types, making it an excellent choice for real-time analytics and monitoring solutions in cloud environments. The plugin empowers users to configure metrics ingestion based on their requirements, define table schemas dynamically, and set various ingestion methods while retaining flexibility regarding roles and permissions needed for database operations. This supports scalable and secure monitoring setups for modern applications that utilize cloud services.

Configuration

Supervisor

[[inputs.supervisor]]
  ## Url of supervisor's XML-RPC endpoint if basic auth enabled in supervisor http server,
  ## than you have to add credentials to url (ex. http://login:pass@localhost:9001/RPC2)
  # url="http://localhost:9001/RPC2"
  ## With settings below you can manage gathering additional information about processes
  ## If both of them empty, then all additional information will be collected.
  ## Currently supported supported additional metrics are: pid, rc
  # metrics_include = []
  # metrics_exclude = ["pid", "rc"]

Azure Data Explorer

[[outputs.azure_data_explorer]]
  ## The URI property of the Azure Data Explorer resource on Azure
  ## ex: endpoint_url = https://myadxresource.australiasoutheast.kusto.windows.net
  endpoint_url = ""

  ## The Azure Data Explorer database that the metrics will be ingested into.
  ## The plugin will NOT generate this database automatically, it's expected that this database already exists before ingestion.
  ## ex: "exampledatabase"
  database = ""

  ## Timeout for Azure Data Explorer operations
  # timeout = "20s"

  ## Type of metrics grouping used when pushing to Azure Data Explorer.
  ## Default is "TablePerMetric" for one table per different metric.
  ## For more information, please check the plugin README.
  # metrics_grouping_type = "TablePerMetric"

  ## Name of the single table to store all the metrics (Only needed if metrics_grouping_type is "SingleTable").
  # table_name = ""

  ## Creates tables and relevant mapping if set to true(default).
  ## Skips table and mapping creation if set to false, this is useful for running Telegraf with the lowest possible permissions i.e. table ingestor role.
  # create_tables = true

  ##  Ingestion method to use.
  ##  Available options are
  ##    - managed  --  streaming ingestion with fallback to batched ingestion or the "queued" method below
  ##    - queued   --  queue up metrics data and process sequentially
  # ingestion_type = "queued"

Input and output integration examples

Supervisor

1. Centralized Monitoring Dashboard: Implement this plugin to feed Supervisor metrics directly into a centralized monitoring dashboard, allowing teams to visualize the health and performance of their applications in real-time. This integration enables quick identification of issues, helps track service performance over time, and aids in capacity planning based on observed trends.

2. Alerting for Process Failures: Utilize the metrics gathered by the Supervisor plugin to create an alerting mechanism that notifies engineers when critical processes go down or enter a fatal state. By setting thresholds in your monitoring system, teams can respond proactively to potential problems, minimizing downtime and ensuring system reliability.

3. Historical Analysis of Process States: Store the metrics collected over time to analyze process state changes and patterns. By examining historical data, teams can identify recurring issues, track the impact of deployment changes, and optimize resource allocation based on process trends, leading to improved overall system performance.

4. Integration with Incident Management Systems: Configure the Supervisor plugin to automatically send alerts to incident management systems like PagerDuty or OpsGenie when a process reaches a critical state. This integration streamlines the incident response process, ensuring that the right team members are notified promptly and can take action without delay.

Azure Data Explorer

1. Real-Time Monitoring Dashboard: By integrating metrics from various services into Azure Data Explorer using this plugin, organizations can build comprehensive dashboards that reflect real-time performance metrics. This allows teams to respond proactively to performance issues and optimize system health without delay.

2. Centralized Log Management: Utilize Azure Data Explorer to consolidate logs from multiple applications and services. By utilizing the plugin, organizations can streamline their log analysis processes, making it easier to search, filter, and derive insights from historical data accumulated over time.

3. Data-Driven Alerting Systems: Enhance monitoring capabilities by configuring alerts based on metrics sent via this plugin. Organizations can set thresholds and automate incident responses, significantly reducing downtime and improving the reliability of critical operations.

4. Machine Learning Model Training: By leveraging the data sent to Azure Data Explorer, organizations can perform large-scale analytics and prepare the data for feeding into machine learning models. This plugin enables the structuring of data that can subsequently be used for predictive analytics, leading to enhanced decision-making capabilities.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.