Zipkin and Clickhouse Integration

Input and output integration overview

The Zipkin Input Plugin allows for the collection of tracing information and timing data from microservices. This capability is essential for diagnosing latency troubles within complex service-oriented environments.

Telegraf’s SQL output plugin sends collected metrics to an SQL database using a straightforward table schema and dynamic column generation. When configured for ClickHouse, it adjusts DSN formatting and type conversion settings to ensure seamless data integration.

Integration details

Zipkin

This plugin implements the Zipkin HTTP server to gather trace and timing data necessary for troubleshooting latency issues in microservice architectures. Zipkin is a distributed tracing system that helps gather timing data across various microservices, allowing teams to visualize the flow of requests and identify bottlenecks in performance. The plugin offers support for input traces in JSON or thrift formats based on the specified Content-Type. Additionally, it utilizes span metadata to track the timing of requests, enhancing the observability of applications that adhere to the OpenTracing standard. As an experimental feature, its configuration and schema may evolve over time to better align with user requirements and advancements in distributed tracing methodologies.

Clickhouse

The SQL output plugin is designed to store Telegraf metrics in an SQL database using a simple, hard-coded schema. Each metric type gets its own table, and columns are generated for every tag and field, with an optional timestamp column. For ClickHouse, the plugin leverages a specialized DSN format as defined by clickhouse-go v1.5.4 and customizes metric type conversion to align with ClickHouse data types. This ensures that integers, texts, timestamps, booleans, and real numbers are mapped to ClickHouse’s native types such as Int64, String, DateTime, UInt8, and Float64 respectively.

Configuration

Zipkin

[[inputs.zipkin]]
  ## URL path for span data
  # path = "/api/v1/spans"

  ## Port on which Telegraf listens
  # port = 9411

  ## Maximum duration before timing out read of the request
  # read_timeout = "10s"
  ## Maximum duration before timing out write of the response
  # write_timeout = "10s"

Clickhouse

[[outputs.sql]]
  ## Database driver
  ## Valid options include mssql, mysql, pgx, sqlite, snowflake, clickhouse
  driver = "clickhouse"

  ## Data source name
  ## For ClickHouse, the DSN follows the clickhouse-go v1.5.4 format.
  ## Example DSN: "tcp://localhost:9000?debug=true"
  data_source_name = "tcp://localhost:9000?debug=true"

  ## Timestamp column name
  timestamp_column = "timestamp"

  ## Table creation template
  ## Available template variables:
  ##  {TABLE}        - table name as a quoted identifier
  ##  {TABLELITERAL} - table name as a quoted string literal
  ##  {COLUMNS}      - column definitions (list of quoted identifiers and types)
  table_template = "CREATE TABLE {TABLE} ({COLUMNS})"

  ## Table existence check template
  ## Available template variables:
  ##  {TABLE} - table name as a quoted identifier
  table_exists_template = "SELECT 1 FROM {TABLE} LIMIT 1"

  ## Initialization SQL (optional)
  init_sql = ""

  ## Maximum amount of time a connection may be idle. "0s" means connections are never closed due to idle time.
  connection_max_idle_time = "0s"

  ## Maximum amount of time a connection may be reused. "0s" means connections are never closed due to age.
  connection_max_lifetime = "0s"

  ## Maximum number of connections in the idle connection pool. 0 means unlimited.
  connection_max_idle = 2

  ## Maximum number of open connections to the database. 0 means unlimited.
  connection_max_open = 0

  ## Metric type to SQL type conversion for ClickHouse.
  ## The conversion maps Telegraf metric types to ClickHouse native data types.
  [outputs.sql.convert]
    conversion_style = "literal"
    integer          = "Int64"
    text             = "String"
    timestamp        = "DateTime"
    defaultvalue     = "String"
    unsigned         = "UInt64"
    bool             = "UInt8"
    real             = "Float64"

Input and output integration examples

Zipkin

1. Latency Monitoring in Microservices: Use the Zipkin Input Plugin to capture and analyze tracing data from a microservices architecture. By visualizing the request flow and pinpointing latency sources, development teams can optimize service interactions, improve response times, and ensure a smoother user experience across services.

2. Performance Optimization in Essential Services: Integrate the plugin within critical services to monitor not only the response times but also track specific annotations that could highlight performance issues. The ability to gather span data can help prioritize areas needing performance enhancements, leading to targeted improvements.

3. Dynamic Service Dependency Mapping: With the collected trace data, automatically map service dependencies and visualize them in dashboards. This helps teams understand how different services interact and the impact of failures or slowdowns, ultimately leading to better architectural decisions and faster resolutions of issues.

4. Anomaly Detection in Service Latency: Combine Zipkin data with machine learning models to detect unusual patterns in service latencies and request processing times. By automatically identifying anomalies, operations teams can respond proactively to emerging issues before they escalate into critical failures.

Clickhouse

1. Basic ClickHouse Integration: Configure the plugin by setting the driver to ‘clickhouse’ and providing the appropriate DSN format as required by clickhouse-go v1.5.4. This ensures that Telegraf can connect and write metrics to your ClickHouse database.

2. Customized Table Schemas: Leverage the table creation and existence check templates to tailor the database schema. This allows you to predefine column types and even disable automatic table creation if you prefer manual schema management.

3. Advanced Type Conversion: Utilize the ClickHouse-specific conversion settings to map Telegraf metric types directly to ClickHouse data types (e.g., mapping integers to Int64 and timestamps to DateTime). This ensures data is stored with the correct precision and format.

4. Initialization and Connection Tuning: Use the init_sql setting to run custom SQL commands upon connection, and adjust connection pool settings (like connection_max_idle_time and connection_max_open) to optimize performance for high-throughput environments.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.