TL;DR InfluxDB Tech Tips - Writing Historical Data, Setting Timestamp Precision in InfluxDB, and Finding Your Log Files

By Regan Kuchan / Product, Developer
Mar 16, 2017

Navigate to:

In this weekly post we recap the most interesting InfluxDB and TICK-stack related issues, workarounds, how-tos and Q&A from GitHub, IRC and the InfluxData Community that you might have missed in the last week or so.

Writing historical data

Q:  I’m writing a lot of sparse, historical data to InfluxDB. My data start in 1800, end in 2016, and they record the average number of French Bulldogs per day.

I’ve noticed that InfluxDB supports historical data but a lot of the default settings seem to target the real-time data use case. Are there any configuration recommendations or schema guidelines for the sparse, historical data use case?

A: We have a couple recommendations for users who want to write sparse, historical data to InfluxDB:

First, lengthen your retention policy’s shard group duration to cover several years. The default shard group duration is one week and if your data cover 216 years - well, that’s a lot of shards! Having an extremely high number of shards is inefficient for InfluxDB. Increase the shard group duration for your data’s retention policy with the ALTER RETENTION POLICY query.

Second, we recommend temporarily lowering the cache-snapshot-write-cold-duration configuration setting. If you’re writing a lot of historical data, the default setting (10m) can cause the system to hold all of your data in cache for every shard. Temporarily lowering the cache-snapshot-write-cold-duration setting to 10s while you write the historical data makes the process more efficient.

Setting timestamp precision for writes in InfluxDB's CLI

Q: I’m writing data using InfluxDB’s Command Line Interface (CLI). I see that I can specify the timestamp precision in query returns but what about writes? Can I specify a second-precision timestamp in with the INSERT command? My attempts seem to indicate that I can’t:

> insert blocks,color=blue number=90 1489529196s
ERR: 
{"error":"unable to parse 'blocks,color=blue number=90 1489529196s': bad timestamp"}

A: The CLI’s precision command actually sets the timestamp precision for both queries and writes. Here’s an example:

$ influx    <---- Connect to the CLI
Connected to http://localhost:8086 version 1.2.2
InfluxDB shell version: 1.2.2

> precision s    <---- Set the query and write precision to seconds

> USE "tldr"
Using database tldr

> INSERT blocks,color=blue number=90 1489529196    <---- Write a point with a second-
                                                         precision timestamp

> SELECT * FROM "blocks"
name: blocks
time color number
---- ----- ------
1489529196 blue 90    <---- Admire your work

Note that the precision command’s rfc3339 option applies to queries only. If you specify precision rfc3339, InfluxDB assumes that all written timestamps are in nanoseconds.

Oh where oh where can my log files be

Q: I can’t seem to find my InfluxDB logs! I’m using the open source version 1.2.1. Can you help me out?

A: Starting with version 1.0.0, systemd operating systems no longer write files to /var/log/influxdb by default. You can access the InfluxDB logs with the command journaldctl -u influxdb or you can write the logs to a file with journalctl -u influxdb > influxd.log.

On System V operating systems logs are stored under /var/log/influxdb/.

What’s next:

  • Downloads for the TICK-stack are live on our "downloads" page.
  • Deploy on the Cloud: Get started with a FREE trial of InfluxDB Cloud featuring fully-managed clusters, Kapacitor and Grafana.
  • Deploy on Your Servers: Want to run InfluxDB clusters on your servers? Try a FREE 14-day trial of InfluxDB Enterprise featuring an intuitive UI for deploying, monitoring, and rebalancing clusters, plus managing backups and restores. 
  • Tell Your Story: Over 300 companies have shared their story on how InfluxDB is helping them succeed. Submit your testimonial and get a limited edition hoodie as a thank you.