Querying InfluxDB Cloud with the C++ Flight SQL Client

By Anais Dotis-Georgiou / Developer
Jun 26, 2023

Navigate to:

InfluxDB Cloud 3.0 is a versatile time series database built on top of the Apache ecosystem. You can query InfluxDB Cloud with the Apache Arrow Flight SQL interface, which provides SQL support for working with time series data. In this tutorial, we will walk through the process of querying InfluxDB Cloud with Flight SQL, using C++. The C++ Flight SQL Client is part of Apache Arrow Flight, a framework for building high-performance data services. It provides a way to efficiently transmit large datasets over gRPC, a modern high-performance RPC framework. Try out querying InfluxDB Cloud with the C++ Flight SQL Client for yourself with this repo.

Requirements and setup

This tutorial assumes that you already have a free InfluxDB Cloud account and that you have Docker running on your machine.

Finally you’ll need to create or obtain the following InfluxDB resources:

  • A bucket
  • A token
  • Your organization (usually the email you used to sign up for your account)

You’ll also need to write data to your InfluxDB account. The simplest way to do this is to write some line protocol manually through the UI. Navigate to Load Data > Buckets > +Add Data > Line Protocol > Enter Manually, select the bucket you want to write to, and write a point to InfluxDB. For example you could write measurementName,tagKey=tagValue fieldKey=1.0. Or, if you want real-world line protocol data, try the NOAA air sensor dataset. You can also check out the following documentation on writing data to InfluxDB Cloud 3.0 for other methods of writing data to InfluxDB Cloud.

Code walkthrough

Let’s break down the code into smaller pieces to understand what’s happening.

  1. Import required classes: We start by importing the required classes from Apache Arrow Flight and other necessary libraries. 
    #include <cstdlib>
#include <iostream>

#include <arrow/flight/client.h>
#include <arrow/flight/sql/client.h>
#include <arrow/table.h>
#include <gflags/gflags.h>
#include <arrow/flight/client_auth.h>
#include <grpcpp/grpcpp.h>
#include <grpcpp/support/channel_arguments.h>
#include <arrow/flight/api.h>
#include <arrow/flight/client_middleware.h>

namespace flight = arrow::flight;
namespace flightsql = arrow::flight::sql;
  2. Gather your authentication credentials and the query you want to execute: 
    const char* myhost = std::getenv("HOST");
const char* mytoken = std::getenv("TOKEN");
const char* mydatabase = std::getenv("DATABASE_NAME");

DEFINE_string(host, myhost , "The host of the Flight SQL server.");
DEFINE_string(token, mytoken, "The token for InfluxDB");
DEFINE_string(database, mydatabase, "The database to query from");
DEFINE_int32(port, 443, "The port of the Flight SQL server.");
DEFINE_string(query, "SELECT * FROM 'measurementName'", "The query to execute.");
  3. Define the main class: We define a C++ class with a Main method where our code will execute. Also connect to the Arrow client. 
    arrow::Status Main() {
ARROW_ASSIGN_OR_RAISE(auto location,
                        flight::Location::ForGrpcTls(FLAGS_host, FLAGS_port));
  std::cout << "Connecting to " << location.ToString() << std::endl;
  4. Set up the Flight SQL Client and add headers for the Bearer token and database. 
    // Set up the Flight SQL client
std::unique_ptr<flight::FlightClient> flight_client;
ARROW_ASSIGN_OR_RAISE(flight_client, flight::FlightClient::Connect(location));
std::unique_ptr<flightsql::FlightSqlClient> client(
new flightsql::FlightSqlClient(std::move(flight_client)));

flight::FlightCallOptions call_options;
call_options.headers.emplace_back("authorization", "Bearer " + FLAGS_token);
call_options.headers.emplace_back("database", FLAGS_database);
  5. Next use the Execute method and pass in the headers and query to the client to execute the query and fetch the result. 
    // Execute the query, getting a FlightInfo describing how to fetch the results
std::cout << "Executing query: '" << FLAGS_query << "'" << std::endl;
ARROW_ASSIGN_OR_RAISE(std::unique_ptr<flight::FlightInfo> flight_info,
client->Execute(call_options, FLAGS_query));
  6. Finally fetch the results into a stream, read them into an Arrow Table, and convert them into a string. 
    ARROW_ASSIGN_OR_RAISE(auto stream, client->DoGet(call_options, endpoint.ticket));
// Read all results into an Arrow Table, though we can iteratively process record
// batches as they arrive as well
ARROW_ASSIGN_OR_RAISE(auto table, stream->ToTable());
std::cout << "Read one chunk:" << std::endl;
std::cout << table->ToString() << std::endl;

You can find the full script here.

Query InfluxDB Cloud with C++ Flight SQL

To run the example, first clone the corresponding repo, navigate to that directory, and follow the following steps as outlined in the README.md or:

  1. In your terminal, set the following environment variables. 
    # Set environment variables

export INFLUX_DATABASE=<your bucket> && \
export INFLUX_HOST=<your host url i.e. us-east-1-1.aws.cloud2.influxdata.com> && \
export INFLUX_TOKEN=<your token>
  2. Run the following commands to utilize the shell script that builds an image with the name ccflight: 
    sh ./influxdb-build.sh build
  3. To start the application, run docker run <IMAGE_NAME> in your terminal: 
    docker run ccflight

Take a look at the following documentation. It helped me build this example, and it can help you on your journey with querying InfluxDB Cloud:

  1. Reference documentation for Arrow Flight
  2. InfluxDB Cloud documentation for Querying data with Arrow Flight SQL in Python
  3. A blog post on InfluxDB, Flight SQL, Pandas, and Jupyter Notebooks Tutorial
  4. A blog post on TL;DR InfluxDB Tech Tips: Downsampling with Flight SQL and AWS Lambda

I hope this blog post inspires you to explore InfluxDB Cloud and take advantage of Flight SQL to transport large datasets from InfluxDB for data processing with the tools of your choice. If you need any help, please reach out using our community site or Slack channel. I’d love to hear about what you’re trying to achieve and what features you’d like InfluxDB to have.