Jenkins and SQLite Integration

Input and output integration overview

The Jenkins plugin collects vital information regarding jobs and nodes from a Jenkins instance through its API, facilitating comprehensive monitoring and analysis.

Telegraf’s SQL output plugin stores metrics in an SQL database by creating tables dynamically for each metric type. When configured for SQLite, it utilizes a file-based DSN and a minimal SQL schema tailored for lightweight, embedded database usage.

Integration details

Jenkins

The Jenkins Telegraf plugin allows users to gather metrics from a Jenkins instance without needing to install any additional plugins on Jenkins itself. By utilizing the Jenkins API, the plugin retrieves information about nodes and jobs running in the Jenkins environment. This integration provides a comprehensive overview of the Jenkins infrastructure, including real-time metrics that can be used for monitoring and analysis. Key features include configurable filters for job and node selection, optional TLS security settings, and the ability to manage request timeouts and connection limits effectively. This makes it an essential tool for teams that rely on Jenkins for continuous integration and delivery, ensuring they have the insights they need to maintain optimal performance and reliability.

SQLite

The SQL output plugin writes Telegraf metrics to an SQL database using a dynamic schema where each metric type corresponds to a table. For SQLite, the plugin uses the modernc.org/sqlite driver and requires a DSN in the format of a file URI (e.g., ‘file:/path/to/telegraf.db?cache=shared’). This configuration leverages standard ANSI SQL for table creation and data insertion, ensuring compatibility with SQLite’s capabilities.

Configuration

Jenkins

[[inputs.jenkins]]
  ## The Jenkins URL in the format "schema://host:port"
  url = "http://my-jenkins-instance:8080"
  # username = "admin"
  # password = "admin"

  ## Set response_timeout
  response_timeout = "5s"

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use SSL but skip chain & host verification
  # insecure_skip_verify = false

  ## Optional Max Job Build Age filter
  ## Default 1 hour, ignore builds older than max_build_age
  # max_build_age = "1h"

  ## Optional Sub Job Depth filter
  ## Jenkins can have unlimited layer of sub jobs
  ## This config will limit the layers of pulling, default value 0 means
  ## unlimited pulling until no more sub jobs
  # max_subjob_depth = 0

  ## Optional Sub Job Per Layer
  ## In workflow-multibranch-plugin, each branch will be created as a sub job.
  ## This config will limit to call only the lasted branches in each layer,
  ## empty will use default value 10
  # max_subjob_per_layer = 10

  ## Jobs to include or exclude from gathering
  ## When using both lists, job_exclude has priority.
  ## Wildcards are supported: [ "jobA/*", "jobB/subjob1/*"]
  # job_include = [ "*" ]
  # job_exclude = [ ]

  ## Nodes to include or exclude from gathering
  ## When using both lists, node_exclude has priority.
  # node_include = [ "*" ]
  # node_exclude = [ ]

  ## Worker pool for jenkins plugin only
  ## Empty this field will use default value 5
  # max_connections = 5

  ## When set to true will add node labels as a comma-separated tag. If none,
  ## are found, then a tag with the value of 'none' is used. Finally, if a
  ## label contains a comma it is replaced with an underscore.
  # node_labels_as_tag = false

SQLite

[[outputs.sql]]
  ## Database driver
  ## Valid options: mssql (Microsoft SQL Server), mysql (MySQL), pgx (Postgres),
  ## sqlite (SQLite3), snowflake (snowflake.com), clickhouse (ClickHouse)
  driver = "sqlite"

  ## Data source name
  ## For SQLite, the DSN is a filename or URL with the scheme "file:".
  ## Example: "file:/path/to/telegraf.db?cache=shared"
  data_source_name = "file:/path/to/telegraf.db?cache=shared"

  ## Timestamp column name
  timestamp_column = "timestamp"

  ## Table creation template
  ## Available template variables:
  ##  {TABLE}        - table name as a quoted identifier
  ##  {TABLELITERAL} - table name as a quoted string literal
  ##  {COLUMNS}      - column definitions (list of quoted identifiers and types)
  table_template = "CREATE TABLE {TABLE} ({COLUMNS})"

  ## Table existence check template
  ## Available template variables:
  ##  {TABLE} - table name as a quoted identifier
  table_exists_template = "SELECT 1 FROM {TABLE} LIMIT 1"

  ## Initialization SQL (optional)
  init_sql = ""

  ## Maximum amount of time a connection may be idle. "0s" means connections are never closed due to idle time.
  connection_max_idle_time = "0s"

  ## Maximum amount of time a connection may be reused. "0s" means connections are never closed due to age.
  connection_max_lifetime = "0s"

  ## Maximum number of connections in the idle connection pool. 0 means unlimited.
  connection_max_idle = 2

  ## Maximum number of open connections to the database. 0 means unlimited.
  connection_max_open = 0

  ## Metric type to SQL type conversion
  ## The values on the left are the data types Telegraf has and the values on the right are the SQL types used when writing to SQLite.
  #[outputs.sql.convert]
  #  integer       = "INT"
  #  real          = "DOUBLE"
  #  text          = "TEXT"
  #  timestamp     = "TIMESTAMP"
  #  defaultvalue  = "TEXT"
  #  unsigned      = "UNSIGNED"
  #  bool          = "BOOL"

Input and output integration examples

Jenkins

1. Continuous Integration Monitoring: Use the Jenkins plugin to monitor the performance of continuous integration pipelines by collecting metrics on job durations and failure rates. This can help teams identify bottlenecks in the pipeline and improve overall build efficiency.

2. Resource Allocation Analysis: Leverage Jenkins node metrics to assess resource usage across different agents. By understanding how resources are allocated, teams can optimize their Jenkins architecture, potentially reallocating agents or adjusting job configurations for better performance.

3. Job Execution Trends: Analyze historical job performance metrics to identify trends in job execution over time. With this data, teams can proactively address potential issues before they grow, making adjustments to the jobs or their configurations as needed.

4. Alerting for Job Failures: Implement alerts that leverage the Jenkins job metrics to notify team members in case of job failures. This proactive approach can enhance operational awareness and speed up response times to failures, ensuring that critical jobs are monitored effectively.

SQLite

1. Local Monitoring Storage: Configure the plugin to write metrics to a local SQLite database file. This is ideal for lightweight deployments where setting up a full-scale database server is not required.
2. Embedded Applications: Use SQLite as the backend for applications embedded in edge devices, benefiting from its file-based architecture and minimal resource requirements.
3. Quick Setup for Testing: Leverage SQLite’s ease of use to quickly set up a testing environment for Telegraf metrics collection without the need for external database services.
4. Custom Schema Management: Adjust the table creation templates to predefine your schema if you require specific column types or indexes, ensuring compatibility with your application’s needs.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.