Getting Started with Flux

By Navdeep Sidhu / Company, Product
Jan 04, 2019

Recently we concluded a series of six ‘office-hour’ webinars on Flux, InfluxData’s new functional data scripting language designed for querying, analyzing, and interacting with data. The goal was to answer any Flux-related questions and also get community feedback to improve the language. In these sessions, InfluxData’s Flux engineer, Adam Anthony showed cool features in the products, did some amazing demos and even replicated a user’s InfluxQL-based query and re-wrote it in Flux language. These sessions clearly showed the versatility of the language and the capabilities which make it an obvious choice for  time series data and beyond.

'Flux - Office Hour' Recordings and Session Overviews

In each webinar, we highlighted a different set of capabilities. Each session was recorded so that it can be replayed by users who could not join. We are highlighting key topics covered in each session in the summary below. The summary will help you jump to the interesting parts in the video if you do not have the time to watch all of them.

Session 1 (Nov. 15th) - In this session you will see an example of switching a complex query using NON_NEGATIVE_DERIVATIVES from InfluxQL to Flux language (FFWD to 17:15). This is a great example to see side-by-side construction of a Flux query which explains from, range, withMeasurement and filter which are basic constructs of Flux to using group, mean, sort, derivative and window functions. Note that the usage of group function has changed since this session and we highlight that in a later session. Adam also shows running sandbox (FFWD 41:40) which is the fastest way to get your hands on Flux.

Session 2 (Nov. 20th) - We revisit using the Sandbox from the last webinar and use Sandbox commands to start with basic constructs of Flux (FFWD 8:10) which helps in getting started with Flux - using the CLI tool to write a Flux query. The test scripts on Github and how you can use them is explained (FFWD 14:01), these scripts reside in flux/functions/transformations/testdata folder and provide the scripts which show how to use for example the derivative function and the expected output as well. Table: key concept is explained (FFWD 24:40) which determines which determines the table structure and how data streams are stored. Group and Window functions are explained (FFWD 29:50) and how adding _start and _stop times can get data into separate groups. Pivot function is explained (FFWD 40:45) which collects values stored vertically (column-wise) in a table and aligns them horizontally (row-wise) into logical sets.

Session 3 (Nov. 27th) - In addition to answering several questions about Flux, our resident expert Adam explains a brand new function called aggregateWindow (FFWD 22:44) which enables users to aggregate data in fixed windows of time. In addition to explaining the function, Adam also explains how to use this function in a Flux query. The percentile functions estimate_tdigest, exact_mean and exact_selector are explained (FFWD 32:30)  with an example including input and output data. Furthermore, the use of Pivot function is explained to show (FFWD 38:07) the join of ‘downloaded’ and ‘installed’ data stored in different tables. The use of map function is explained (FFWD 48:53) to calculate ratio of ‘downloaded’ vs. ‘installed’ measurements.

Session 4 (Dec. 6th) - This session explains the new group function. Note that the earlier sessions showed the older implementation of the group function. As the Flux language evolves, the implementation of functions can change as well. Adam discusses the new implementation of the group function (FFWD 20:25) and explains the reasons for the change as well. This is one of the most important functions in the language as it lets developers reshape the data and how you can group, re-group and ungroup data to achieve desired results.

Session 5 (Dec. 13th) - We revisit the breaking change in the syntax of group function and this time with a demo of the new syntax. Adam explains the new syntax for the  group function (FFWD 4:05) and how this new syntax is available in the nightlies first and will show up in the next OSS version soon. Adam also shows the new method for testing code (FFWD 15:10) in testingTest which tests a function called simpleMax and loads both input and output files in Flux tables and compares them using assertEquals. The import and standard library design proposals are explained (FFWD 21:02) which make Flux more organized and support packages and namespaces. Additionally, Adam highlights another way of doing maths across measurements (FFWD 29:20) across httpd and write streams using pivot function.

Session 6 (Dec. 20th) - This was the last webinar in the office hour series in 2018. Adam walks through the Github repo for Flux (FFWD 5:52) and ‘helpers.go’ function in flux/functions/tests which is a great way to setup tests using Flux. This is a great way to learn other Flux functions as well. Upcoming Flux features (FFWD 11:50) are highlighted including how to define and import a package. Adam also writes another query example for doing distinct (FFWD 15:05) on multiple columns. The concept of writing helper functions is also explained.

Additional Flux Resources

If you were unable to join the office-hour webinar series, we encourage you to view the recordings linked above and start writing Flux queries using the sandbox which is the fastest way to get your hands on Flux. As a reminder, Flux is still being developed, and we recommend that Flux not be used for Production uses at the moment.

In addition to the Flux documentation and forums, the blog posts will be a good source of getting up to speed. So keep an eye on InfluxData’s blog page and look out for Flux-related posts to learn more about Flux.