Announcing InfluxDB v0.10 Beta w/ TSM Storage Engine & Backup/Restore

By Paul Dix / Product
Jan 13, 2016

We are excited to announce the immediate availability of InfluxDB v0.10.0-beta1 today. The big news with this release is that the Time Structured Merge Tree, or TSM for short, is now the default storage engine. TSM is a massive improvement on our previous releases supporting write loads in the hundreds of thousands of data points per second and compression that can bring the space used per data point down to as little as 2.2 bytes. The TSM engine supports online backups with offline restores. Finally, we’ve changed to building with Go 1.4.3. Read on for more details.

TSM Storage

If you’re upgrading in place, you can use the offline conversion utility to convert all your old data to the TSM engine. Details on how to convert data can be found here. Optionally, you can keep your data in the old format and upgrade without converting. However, backup and restore will only work with the TSM storage engine format.

The conversion utility is quite fast and can convert multi-GB databases in a few minutes. The biggest advantage to migrating your data to the new format is a huge reduction in the on-disk size of the database. In our testing we’ve seen up to a 99% reduction in disk space used.

If you are currently using InfluxDB v0.8x and need help migrating to v0.10, contact us for how we can help with best practices and utilities.

Backup/Restore

The TSM engine supports online backups and offline restores. Users are able to backup an entire database, retention policy, or an individual shard. This means that an operator can wait for a shard to become cold for writes and then back it up entirely, or backup just the hot shards multiple times to get incremental backups.

We’ll be writing up more details on the backup/restore functionality, but in the meantime execute this command to get usage instructions for backup and restore:

influxd backup

Building with Go 1.4.3

For the 0.10.0 release we’ll be building with Go 1.4.3. We had previously been building the nightly releases with 1.5.2, but found that the performance was significantly better in our testing and in a few customers real world production workloads with 1.4.3. More details can be seen in this comment on #4977 and on #5217.

We’ll be doing extensive testing against Go 1.6 when it’s released.

The next release

We’re putting out this beta for early testing. We’ve already done extensive testing, but getting use from other workloads is helpful for turning up any issues that we may not have uncovered.

We’ll be cutting another beta release soon with an updated clustering implementation. We’ll have more details on that when it becomes available.