Bring Your Own Telegraf

By David Flanagan / Use Cases, Developer, Product
Jun 06, 2019

Navigate to:

I have the privilege of telling people I work for a truly open source company. What do I mean by that?

90% of the InfluxData code is open source, with that other 10% being our enterprise additions to our database, InfluxDB. That 10% fuels the company and it’s open source commitment.

Our commitment to open source doesn’t stop there. We also actively encourage, maintain, and support contributions that integrate our products with other products, OSS or otherwise; even those which some may consider our “competitors”. Alas, we don’t see it this way. We welcome and encourage an open and supporting ecosystem around time series, monitoring, and alerting.

Of course, this comes at a cost. While Telegraf supports reading and writing to over 200 software components; such as DataDog, Elasticsearch, Kafka, MongoDB, OpenTSDB, Prometheus, Wavefront, and much much more…this comes at a cost…a cost of bytes.

Plugins everywhere<figcaption> Plugins everywhere</figcaption>

Presently, the Telegraf binary weighs in at 60MiB (amd64 binary), 244MiB (Docker Image), and 80MiB (Docker Image Alpine).

While not massive, depending on your usage of our plugins, this might be too much for you to chew. As more and more of our community adopts Kubernetes, embracing patterns like sidecars, the size of utilising multiple Telegraf instances can become problematic.

Introducing Bring Your Own Telegraf (BYOT)

BYOT is a Docker-based framework for building Docker Images with a custom, slim-line, Telegraf binary; that only includes the plugins you need. This does not require any special configuration. You just need to provide your Telegraf configuration, as you normally would.

This will result in substantially smaller image sizes. Let’s take a look at a comparison. Below are the image sizes for the normal Telegraf image, and the Alpine variant. Next to it is a custom Kubernetes image I built with only the following plugins:

  • Inputs
    • Prometheus
  • Outputs
    • InfluxDB
$ docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
telegraf            1.10                c14f9454c5aa        13 days ago         244MB
telegraf            1.10-alpine         f6d6ea3f4972        4 days ago          80MB
rawkode/telegraf    kubernetes          3437fe686f23        4 days ago          25.4MB

Getting started

My recommended approach is to provide a Git repository for each Telegraf configuration within your infrastructure. Let’s assume we want to build a Telegraf with the following plugins:

  • Inputs
    • Docker
  • Outputs
    • InfluxDB 2

Configuring Telegraf

First, we’d create our Telegraf configuration. The build process expects there to be a telegraf directory in the root of your build context. This directory is no different from your /etc/telegraf directory on any other system. You can use a single file, telegraf.conf, or a directory-based approach, telegraf.d. For simplicity, this example uses a single file.

[agent]
  interval = "5s"
  omit_hostname = true

[[inputs.docker]]
  endpoint = "unix:///var/run/docker.sock"

[[outputs.influxdb_v2]]
  urls = ["http://influxdb2:9999"]
  token = "$INFLUXDB_2_TOKEN"
  organization = "InfluxData"
  bucket = "default"

Next you need to add a Dockerfile to your repository:

FROM rawkode/telegraf:byo AS build
FROM alpine:3.7 AS telegraf
COPY --from=build /etc/telegraf /etc/telegraf
COPY --from=build /go/src/github.com/influxdata/telegraf/telegraf /bin/telegraf
ENTRYPOINT [ "/bin/telegraf" ]

Build

That’s it! Have your CI system automagically build this image on every commit and push it to your registry and you’re good to go. You can specify the version of Telegraf to build with a build-arg.

docker image build --build-arg VERSION=1.10 -t my_telegraf

A custom Telegraf with only the plugins you require, within a few minutes!

How does it work?

BYOT utilizes the often forgotten ONBUILD instruction available within a Dockerfile. This allows us to define instructions that are deferred until a subsequent calling, such as your docker image build command. Feel free to peruse my code, or checkout my example repository, to understand it a little better.

Any other improvements?

During my initial tests, which are not extensive, I’ve noticed that the memory usage of Telegraf is about a 1/3 (6MiB vs. 12MiB) of running the normal Telegraf binary with only 4 plugins (system, cpu, mem, disk) enabled. This could be circumstantial and I will publish more results on this in due course.

What's next

Currently, BYOT supports custom building with input and output plugins based on your configuration. I plan to add support for aggregators and processors shortly also, which will bring our image size down even further.

Thanks for listening. I’d love to know what you think, so please reach out in the comments section or via Twitter.