TL;DR InfluxDB Tech Tips - Using InfluxDB To Write and Export Data In Line Protocol

By Regan Kuchan / Developer
Nov 03, 2016

Navigate to:

In this weekly post we recap the most interesting InfluxDB and TICK-stack related issues, using InfluxDB to write and export data in line protocol, workarounds, how-tos and Q&A from GitHub, IRC and the InfluxDB Google Group that you might have missed in the last week or so.

Write data from a file

Q: I have a lot of points in line protocol that I’d like to write to InfluxDB with the HTTP API. Is there a way to do this besides separating each point in the request body with a newline?

A: You can use the HTTP API to write data to InfluxDB from a file. For example:

$ curl -i -XPOST "http://localhost:8086/write?db=mydb" --data-binary @bridges.txt

A sample of the data in bridges.txt:

bridges,type=cable_stayed visitors=607 1478133069000000000
bridges,type=cable_stayed visitors=608 1478133071000000000
bridges,type=suspension visitors=231 1478133069000000000
bridges,type=suspension visitors=234 1478133071000000000

Export data

Q: I’d like to export data from InfluxDB in line protocol format. How can I do this?

A: The influx_inspect tool allows you to do just that. The example below exports data from the mydb database and the autogen retention policy to a file called export.

influx_inspect export -datadir "/var/lib/influxdb/data" -waldir "/var/lib/influxdb/wal" -out "export" -database mydb -retention autogen

A sample of the data in export:

# INFLUXDB EXPORT: 1677-09-21T00:12:43Z - 2262-04-11T23:47:16Z
# DDL
CREATE DATABASE mydb WITH NAME autogen
# DML
# CONTEXT-DATABASE:mydb
# CONTEXT-RETENTION-POLICY:autogen
# writing tsm data
mymeas,color=blue value=800 1478129348953855972
mymeas,color=blue value=900 1478129351427777943
# writing wal data
bridges,type=cable_stayed visitors=607 1478133069000000000
bridges,type=cable_stayed visitors=608 1478133071000000000
bridges,type=suspension visitors=231 1478133069000000000
bridges,type=suspenstion visitors=234 1478133071000000000

Note that the example uses syntax available in the 1.1 version of the influx_inspect tool.  The release candidate for version 1.1 is available on the downloads page.

Set a (nearly) infinite shard group duration

Q: I’m trying to create a retention policy with an infinite shard group duration. The CREATE RETENTION POLICY query seems to succeed but the shard group duration is set to seven days, not INF.

> CREATE RETENTION POLICY "ever_after" ON "final_frontier" DURATION INF REPLICATION 1 SHARD DURATION INF

> SHOW RETENTION POLICIES ON "final_frontier"
name        duration  shardGroupDuration  replicaN  default
----        --------  ------------------  --------  -------
autogen     0s        168h0m0s            1         true
ever_after  0s        168h0m0s            1         false

A: Shard group durations cannot be infinite. As a workaround, we recommend specifying a 1000w shard group duration to achieve an extremely long shard group duration. For more on shard duration management, see the Schema Design documentation.

There’s an open issue on GitHub for InfluxDB to return an error if the CREATE RETENTION POLICY query attempts to set the shard group duration to INF.

For more InfluxDB tips, see our Frequently Asked Questions page and feel free to post your questions in the InfluxDB users group!

What's next?

  • Download and get started with InfluxDB!
  • Schedule a FREE 20 minute consultation with a Solutions Architect to review your InfluxDB project.
  • Attend one of our FREE virtual training seminars.
  • Got a question and need an immediate answer from the InfluxData Support team? Support subscriptions with unlimited incidents start at just $399 a month. Check out all the support options here.