SNMP and Clickhouse Integration
Powerful performance with an easy integration, powered by Telegraf, the open source data connector built by InfluxData.
5B+
Telegraf downloads
#1
Time series database
Source: DB Engines
1B+
Downloads of InfluxDB
2,800+
Contributors
Table of Contents
Powerful Performance, Limitless Scale
Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.
See Ways to Get Started
Input and output integration overview
The SNMP plugin allows you to collect a variety of metrics from SNMP (Simple Network Management Protocol) agents. It provides flexibility in how data is retrieved, whether collecting single metrics or entire tables.
Telegraf’s SQL plugin sends collected metrics to an SQL database using a straightforward table schema and dynamic column generation. When configured for ClickHouse, it adjusts DSN formatting and type conversion settings to ensure seamless data integration.
Integration details
SNMP
This plugin uses polling to gather metrics from SNMP agents, supporting retrieval of individual OIDs and complete SNMP tables. It can be configured to handle multiple SNMP versions, authentication, and other features.
Clickhouse
Telegraf’s SQL plugin is engineered to write metric data into an SQL database by dynamically creating tables and columns based on incoming metrics. When configured for ClickHouse, it utilizes the clickhouse-go v1.5.4 driver, which employs a unique DSN format and a set of specialized type conversion rules to map Telegraf’s data types directly to ClickHouse’s native types. This approach ensures optimal storage and retrieval performance in high-throughput environments, making it well-suited for real-time analytics and large-scale data warehousing. The dynamic schema creation and precise type mapping enable detailed time-series data logging, crucial for monitoring modern, distributed systems.
Configuration
SNMP
[[inputs.snmp]]
agents = ["udp://127.0.0.1:161"]
[[inputs.snmp.field]]
oid = "RFC1213-MIB::sysUpTime.0"
name = "sysUptime"
conversion = "float(2)"
[[inputs.snmp.field]]
oid = "RFC1213-MIB::sysName.0"
name = "sysName"
is_tag = true
[[inputs.snmp.table]]
oid = "IF-MIB::ifTable"
name = "interface"
inherit_tags = ["sysName"]
[[inputs.snmp.table.field]]
oid = "IF-MIB::ifDescr"
name = "ifDescr"
is_tag = true
Clickhouse
[[outputs.sql]]
## Database driver
## Valid options include mssql, mysql, pgx, sqlite, snowflake, clickhouse
driver = "clickhouse"
## Data source name
## For ClickHouse, the DSN follows the clickhouse-go v1.5.4 format.
## Example DSN: "tcp://localhost:9000?debug=true"
data_source_name = "tcp://localhost:9000?debug=true"
## Timestamp column name
timestamp_column = "timestamp"
## Table creation template
## Available template variables:
## {TABLE} - table name as a quoted identifier
## {TABLELITERAL} - table name as a quoted string literal
## {COLUMNS} - column definitions (list of quoted identifiers and types)
table_template = "CREATE TABLE {TABLE} ({COLUMNS})"
## Table existence check template
## Available template variables:
## {TABLE} - table name as a quoted identifier
table_exists_template = "SELECT 1 FROM {TABLE} LIMIT 1"
## Initialization SQL (optional)
init_sql = ""
## Maximum amount of time a connection may be idle. "0s" means connections are never closed due to idle time.
connection_max_idle_time = "0s"
## Maximum amount of time a connection may be reused. "0s" means connections are never closed due to age.
connection_max_lifetime = "0s"
## Maximum number of connections in the idle connection pool. 0 means unlimited.
connection_max_idle = 2
## Maximum number of open connections to the database. 0 means unlimited.
connection_max_open = 0
## Metric type to SQL type conversion for ClickHouse.
## The conversion maps Telegraf metric types to ClickHouse native data types.
[outputs.sql.convert]
conversion_style = "literal"
integer = "Int64"
text = "String"
timestamp = "DateTime"
defaultvalue = "String"
unsigned = "UInt64"
bool = "UInt8"
real = "Float64"
Input and output integration examples
SNMP
- Basic SNMP Configuration: Collect metrics from a local SNMP agent using typical SNMP community string settings. This setup is ideal for local monitoring of device performance.
- Advanced SNMPv3 Setup: Securely collect metrics using SNMPv3 with authentication and encryption to enhance security. This configuration is recommended for production environments.
- Collect Interface Metrics: Configure the plugin to collect interface metrics from the device’s SNMP table. Utilize fields to capture specific data points for traffic analysis.
- Join Two SNMP Tables: By using translation fields, join data from two SNMP tables for a comprehensive view of correlated performance metrics.
Clickhouse
-
Real-Time Analytics for High-Volume Data: Use the plugin to feed streaming metrics from large-scale systems into ClickHouse. This setup supports ultra-fast query performance and near real-time analytics, ideal for monitoring high-traffic applications.
-
Time-Series Data Warehousing: Integrate the plugin with ClickHouse to create a robust time-series data warehouse. This use case allows organizations to store detailed historical metrics and perform complex queries for trend analysis and capacity planning.
-
Scalable Monitoring in Distributed Environments: Leverage the plugin to dynamically create tables per metric type in ClickHouse, making it easier to manage and query data from a multitude of distributed systems without prior schema definitions.
-
Optimized Storage for IoT Deployments: Deploy the plugin to ingest data from IoT sensors into ClickHouse. Its efficient schema creation and native type mapping facilitate the handling of massive volumes of data, enabling real-time monitoring and predictive maintenance.
Feedback
Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.
Powerful Performance, Limitless Scale
Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.
See Ways to Get Started
Related Integrations
Related Integrations
HTTP and InfluxDB Integration
The HTTP plugin collects metrics from one or more HTTP(S) endpoints. It supports various authentication methods and configuration options for data formats.
View IntegrationKafka and InfluxDB Integration
This plugin reads messages from Kafka and allows the creation of metrics based on those messages. It supports various configurations including different Kafka settings and message processing options.
View IntegrationKinesis and InfluxDB Integration
The Kinesis plugin allows for reading metrics from AWS Kinesis streams. It supports multiple input data formats and offers checkpointing features with DynamoDB for reliable message processing.
View Integration