How to Build a Streaming Pipeline for IoT Data Flows with Landoop & InfluxDB
In this webinar, Stefan Bocutiu, Head of Engineering at Landoop (now Lenses.io), shows how to build an end-to-end data streaming pipeline for massive IoT data flows. He uses the Kafka ecosystem to orchestrate the cornerstones of IoT data platforms: MQTT, a protocol optimized for connected devices and M2M protocol, to analyze device data on the fly and persist the results into an InfluxDB Time Series Database.
Watch the Webinar
Watch the webinar “How to Build a Streaming Pipeline for Massive IoT Data Flows with Landoop and InfluxDB” by filling out the form and clicking on the download button on the right. This will open the recording.
[et_pb_toggle _builder_version="3.17.6" title="Transcript" title_font_size="26" border_width_all="0px" border_width_bottom="1px" module_class="transcript-toggle" closed_toggle_background_color="rgba(255,255,255,0)"]
Here is an unedited transcript of the webinar “How to Build a Streaming Pipeline for Massive IoT Data Flows with Landoop and InfluxDB.” This is provided for those who prefer to read than watch the webinar. Please note that the transcript is raw. We apologize for any transcribing errors.
Speakers:
- Chris Churilo: Director Product Marketing, InfluxData
- Stefan Bocutiu: Head of Engineering, Landoop
Chris Churilo 00:00:07.116 All right. Welcome, everybody. Thanks for joining our webinars at InfluxData. Today, we have a fantastic webinar lined up from Landoop, in particular from Stefan. My name is Chris Churilo, and I run product marketing, and I conduct these webinars as well as our trainings, and just going to give another little plug for InfluxDays. So once we get the videos all edited, I will post them, so you can take a look at the fantastic presentations that we had at InfluxDays New York. And we’ll be doing something similar in London on June 14th and 15th, and you will be getting-if you’re interested, just go to influxdays.com, and you’ll be able to see the agenda as we’re starting to pull it together. So with that, just want to remind everybody, if you have any questions, feel free to put your questions in the chat or the Q&A panel, whichever one is easier for you. And we will actually reserve the questions for the end of the presentation today with Stefan. So with that, I am going to put myself on mute, and I’ll let Stefan take it away.
Stefan Bocutiu 00:01:14.766 Thank you, Chris. First and foremost, hello to everyone and thank you for joining this webinar. I know it’s quite early for the people joining from the US West Coast, so good morning to you. I am Stefan Bocutiu, and in this session, I will demonstrate how to build a common streaming data pipeline for IoT data flows by leveraging Apache Kafka, Landoop’s Lenses platform and InfluxData Time Series Database. Before I go any further, let me set the agenda so you know what you get out of this talk. I’ll start with a brief introduction, then I’ll talk about the data challenges that companies face today in the space of IoT, and after that, I’m actually going to go and build the data pipeline. At the end of the session, there will be a Q&A section, so let’s get going.
Stefan Bocutiu 00:02:03.722 I’m part of the great Landoop team, and at Landoop, we do believe that companies’ data needs to be turned into action. And that’s why we built the data streaming platform, data integration platform, on top of Apache Kafka and Kubernetes that connects, reshapes, and delivers actionable data. Today, we live in the ever-more connected world, so that means we do collect a lot of data. We use Fitbit devices. We have smart home devices. We track livestock. We even track farming vehicles. The Gartner prediction says that by the year 2020, there’s going to be more than 26 billion connected devices in the whole world. Surprising or not, 94% of the companies in the IoT space face challenges in collecting and processing IoT data. The challenge comes out in extracting value from an ever more increasing data volume. The reason we want to extract value is to provide better services and timely services.
Stefan Bocutiu 00:03:13.404 A typical IoT data flow involves a vast network of sensors publishing data periodically. That flux of information is then processed on the fly to extract patterns, trends, or do predictions. Those end results and the initial data might be stored for further analysis. You might have spotted here the typical ETL scenario. At Landoop, we like to name it connect, process and analyze, and connect again. There are various use cases for this scenario that I’ve just described. To give you a quick example, the UK largest energy provider is using smart meters data to predict consumption. They do process over five billion messages every day, and that number is only supposed to go up. In today’s demo, I will build a data pipeline that will be streaming temperature and humidity levels. Those will be aggregated to produce minimum, maximum average value. And the end result, I’ll be storing them for further analysis.
Stefan Bocutiu 00:04:26.600 So now that I’ve set up the desired outcome, how do we go and implement it? The solution that I’m going to propose today is leveraging Apache Kafka, Kafka Connect and Kafka Streams, Lenses SQL, and the popular choice for storage, Time Series Database in the space of IoT, InfluxDB. So for the purpose of this exercise, I will be using Lenses Box to take care of all the infrastructure, and for that, I need to run its Docker image. To get my InfluxDB instance, I’ll run the InfluxData Docker image as well because setting up a network of sensors goes beyond the scope of this webinar. I’ve prepared upfront a small application that generates the data, but, normally, the devices would. So what is it that we’re going to build today? So we’re going to have device data being generated. That data is going to be probing to Apache Kafka via Connect. We’re going to do a bit of stream analysis. We store the results back into Apache Kafka, and you will see how using InfluxDB Sink Connector, you can move data from Kafka into your InfluxDB instance. So let’s get going.
Stefan Bocutiu 00:05:55.681 If I were to define Kafka in one sentence, I would say that it’s an open source, streaming framework with messaging semantics. Every record needs a key-value pair, and it’s stored into a topic and partition. Kafka is a good fit for building IoT systems, and there’s a couple of reasons for that. When we talk IoT, we talk a large number of messages. It can be millions or even billion of messages, and Kafka can handle that quite easily. We know that IoT data can come at irregular intervals and can have various volumes, and that fits quite well with the producer and consumer being decoupled in Kafka. Because they are decoupled, each one can go on their own pace without impacting the other. That basically solves the problem of back pressure. We don’t have to worry about it. When we build an IoT system, we want our data to be processed usually by more than one service, so we can use Kafka to stream that data to our microservices. If we want to do stream analytics, we can make use of Kafka Streams API. We know that IoT data has many sources, so we can leverage Kafka Connect to bring in the data into Kafka reliably and with fault tolerance out of the box.
Stefan Bocutiu 00:07:25.812 Lenses Box is Landoop’s free offering for the Lenses platform, and it provides the entire Apache Kafka ecosystem in one docker command. I’m going to go and actually start with docker image. You have the entire command here, so docker is a prerequisite. Do not worry. You don’t have to learn the entire command by hand. You’ll get this when you download Lenses, so let’s start it. Enter my password. When it starts, the docker image will provision one Kafka broker, one Zookeeper node, will spin up the schema registry, and will also provision a Connect worker. Alongside that, we pack in a couple of synthetic data generators such that when you open the Lenses UI, you have some data to play with. Also in the Docker image, you will find out that we packed the Stream Reactor component. It is Landoop open source collection of Kafka Connect sinks and sources. Normally, it takes around 45 seconds for the box to start. While it does that, then we also launch the MQTT program. I’ve prepared it up front. The program takes two arguments. First is the port that I’m going to bind the MQTT broker to, and the second one is the MQTT copy that I’ll be sending the data to. So I am going to run this as well. As you can see on the screen, it prints the data as if it was coming from the devices. So great, now we have data that we can work with.
Stefan Bocutiu 00:09:21.104 Kafka Connect is the framework that sits on top of Kafka, and it allows it to move data in and out of Kafka reliably with fault tolerance out of the box. Stream Reactor is an open source collection of Kafka sources and sinks. A Kafka Connect source is meant to bring data in. A Kafka Connect sink is meant to move data out into the targets. MQTT stands for Message Queuing Telemetry Transport, and it is a publish, subscribe, very simple and lightweight messaging protocol. It has been designed for low bandwidth, high-latency, and unreliable networks, and that makes it quite ideal for the world of IoT. With Stream Reactor is another component that we offer as open source in Landoop, and that’s the Kafka Connect Query Language. This allows us to embed sequel-like syntax into the connected configuration in order to make it easier to set it up. As you can see at the bottom of the screen, that’s the syntax to bring data from the MQTT source topic into a Kafka topic named Kafka on this core topic. So let’s go ahead and actually provision that source for that.
Stefan Bocutiu 00:10:42.136 I’m going to log in to my Lenses UI. I’m going to navigate to localhost:3030, and I’m going to use the default credentials: admin, admin. The page that I land into is the dashboard that Lenses provides. As you can see, we do have one Kafka broker. We have the Zookeeper instance, and we have a Connect cluster. Let me go ahead and provision the connector. For that, I’m going to go to the connector’s page. From the list of connectors, I’m going to choose MQTT. Let me zoom in. I’ve already prepared the configuration. When you set up a Kafka Connect source or sink, there’s a couple of keys that you have to provide. One of them is name. You have to provide a unique name for your sink or source. The next one will be the class that the Connect will instantiate. And if you want to scale your connector, all you have to do is set up the number of tasks you want to run to a higher number than one. I’m going to highlight also the KCQL syntax that instructs the connector what to do. One of the important things here is the keyword with key, which basically tells the connector to pick up the ID field from incoming entity payload and store it as the Kafka record key.
Stefan Bocutiu 00:12:12.923 Of course, you can integrate your CI-CD tools with Kafka Connect and indeed with Lenses. You can see here that the curl command that you can use to do so. Let’s go ahead and create the connector. It usually takes a couple of seconds to start up. As you can see already on my screen, the task is being provisioned. I can see the configuration here that is running. Obviously, I can restart a task, and I can manage my connector. What you get also out of the box with Lenses is full auditing. I go to the audit page. I should see an entry that says that I’ve added a new connector with all the details about it. Well, now that I’ve set my connector, I should see data coming through into Kafka, so let’s go to the topics page. I’m into the topics page now, and there’s a topic with a name Sensor Unscored Data as I’ve configured it to be. Let’s click on that. You can see on the screen I already have data as it arrives in Kafka, and we achieved that with a simple SQL select for the lag subscription. But what if you want to actually see data that already landed in Kafka? For that, you can navigate to the data tab, and you can browse by using the paging, or if you feel adventurous, you can switch to the SQL part.
Stefan Bocutiu 00:13:47.617 There’s options here that if I want to actually see the data has arrived from a specific device data, I can filter. I know my Kafka record key holds the device that I did, so I can filter on that. SB01, for example. I don’t need a thousand records. I think I’m good with two. I hit the run query, and I get my two records that have arrived from my device. I can also go further and I can say: “Maybe I want to filter and see those records where the temperature value has been great, as in bigger than 25 degrees.” So all I have to do is actually filter on the temperature, and run my query, and that’s my data. So great. Now that we have entered the data, which we landed into Kafka, we can move on to the next step, and that’s to do the stream analytics on it.
Stefan Bocutiu 00:14:49.817 To run the stream analytics, I’ll rely on Lenses SQL engine for Apache Kafka. Lenses, with its SQL engine, allows me to declare, deploy, scale and monitor SQL processors. SQL processors are basically Kafka’s [inaudible] application that I’ve encoded with LSQL. LSQL is the acronym for Lenses SQL. What you get out of the box with Lenses is native integration with Kubernetes to scale up and down your processor instances. Also, for each processor, you get the visual topology that shows you how the data is being manipulated and transformed. That could give you the opportunity to spot room for improvements. Let’s go ahead and actually code the processor that computes the average values for humidity and temperature and also calculates the minimum and maximum overtime limit.
Stefan Bocutiu 00:15:47.157 I’m going to go back to the Lenses UI. This time, to the SQL processor page and click on the new processor, and I’m going to name it IoT. Now, there on this setting on the screen. If I were to be running in Kubernetes, we’re translating the number of odds that I’ll be running. Lenses integrates natively with Kubernetes. It provisions the pod but also monitors or restarts the pod if need be. Let’s see the actual LSQL code, how it looks. I’m going to go briefly over what it does. It picks data from the sensor unscored data. That’s how I landed my data from MQTT to Kafka. It groups the record by their key. I know the data holds the device ID. Sorry, the key holds the device ID, and it computes into many results, and it does so over a time window of two seconds. Two seconds is not the typical aggregation time in the world of IoT, but I’m doing so for the sake of the exercise.
Stefan Bocutiu 00:16:57.488 The last part of the query is basically utilizing the computer results and inserts them into this Kafka topic. So let’s go ahead and actually create the processor. Takes a couple of seconds for it to be spinned up, and there you go. What you have on the screen is the processor code, its topology, and at the bottom of the screen, you should see actual live data coming through. The topology graph that you see on the screen is not started. I can actually go and interact with the node to see in detail what each one does. For example, I can go to groupBy, and I’ll see that it groups by the key. If I want to see the source data coming through, I can just click on the source topic, and I see that the data is actually aggregated. So now we have data that has been running through Kafka and has been aggregated. We can move to the next step to actually store the data into the popular time series database, which is InfluxDB.
Stefan Bocutiu 00:18:02.631 I’m sure most of the audience in here today knows what InfluxDB is, but it will be good to actually go over it again. InfluxDB, it’s an open source Time Series Database open in the space of IoT. It’s been written in Go and can handle high writes and query loads. typical data you would push into InfluxDB could be application metrics or sensor IoT data. Stream Reactor delivers the InfluxDB Sink Connector. And I’m not going to say this just because of this webinar-it is by far the most used connector out of all of them. The InfluxDB Sink Connector comes with full support for AVRO and JSON payloads in Kafka. It also allows you to set up various error policies. For example-and we’ve seen this in production-it happens at times that bad record lands into Kafka. In that case, you don’t want to stop the world. You just want that exception to be ignored, and the connector should go on doing its job. However, by using Noop, and in case of network outage or, let’s say, a timeout when writing data into InfluxDB could lead to data being lost, and that’s where the Retry option comes to play. But when using Retry, the connector will insert the Connect framework to replay the messages. There’s obviously a max retry until the connector will actually be failed. There’s also scenarios where I just want to stop the connector and just take the appropriate action, and that’s when the Throw option should be used.
Stefan Bocutiu 00:19:52.995 Being part of Stream Reactor, it comes with support for the KCQL language, and that makes it very easy for me to set up the behavior for the InfluxDB connector. With simple sequel-like syntax, I can route my Kafka topic data into InfluxDB Measurements. I can route the same data into more than one Measurement in InfluxDB by simply specifying two Insert statements where I can route any number of Kafka topic data into any number of InfluxDB Measurements. The same connector allows you to specify the tags. Before I try to talk about the tags, let me explain briefly about the points being inserted into InfluxDB. Every time you insert the point, the key for that point is made out of the measurement made, the set of tags, and the timestamp. So going back now on the tags, I can say that you can set up the tag to be a value that is coming in the payload, or you can set up a constant value as you’ve seen below. For this timestamp, you can do the same thing via KCQL. You can pick up a field that is coming in the payload, or if you do not specify it, it will be the time when the connective processes direct. InfluxDB Sink Connector allows you to set how long you want the records to stay or to be kept into InfluxDB. At the bottom of the screen, you get to see the simple syntax for pulling data out of sensor unscored data average. That’s where we have sent our computed values and stored into sensor measurement. Each point is tagged with the value of the ID field, and we also set a constant tag, which is poweredBy, and the value is Lenses.
Stefan Bocutiu 00:21:49.614 Before I go on and actually provision the sink, I actually want to talk a little bit about the drawbacks of the sink, if I can call them by that. Until Java 9, the sink is written with the JVM. That’s because Connect runs on the JVM. There’s no way you can get the timestamp with nanosecond accuracy. We had to code that into the sink. It would have been great if that would have been part of the Java client library that InfluxDB supports. We’re looking in the future to add filter support to the KCQL. You may want to insert records into your InfluxDB only when the temperature is not null. Also, we want to add function support. You may want to concatenate fields or you may want to actually do arithmetics on the values. One of the challenges with an InfluxDB connector is the throughput. Because it runs on top of Kafka, it requires the Kafka domain knowledge to extract the best performance out of it. Kafka Connect is a great framework, but it’s not perfect. It’s been acknowledged by the community that it has its drawbacks, and the drawbacks come in the form of rebalances. When we provision sinks and sources, and then we add the new instance, or different sink, or a different source, the Connect will have to stop the existing running connectives, and that means downtime that sometimes is not acceptable. So the best practice requires you to run Kafka Connect clusters in isolation. That means, for example, if you want to push the same data both into InfluxDB, but, let’s say, also Elasticsearch, you should run two Kafka Connect clusters, one doing it with InfluxDB, and the other one with Elasticsearch.
Stefan Bocutiu 00:23:45.951 Now with all this talk out of the window, I can go actually set my-or push the data from Kafka into InfluxDB. The first thing I need to do is actually start my InfluxDB instance. So I’m going to run the Docker image first, put down my password, and stop it, and then I need to create a database. I’m going to name that database IoT. I picked up the wrong curl command. Bear with me one second. I’ve created a database. Let’s just make sure it was actually created. I’m going to run another query to just show the databases, so it’s there on my screen. Now I have my InfluxDB ready, and I can move in to actually provision the sink. Normally, you can just hit the Kafka Connect REST endpoints to provision the sync, but with the Lenses UI, you get a high-level tool to actually help with the task. Let’s go back to the connectors page, and this time from the list, I’ll pick InfluxDB. I have prepared the configuration already. Similar to MQTT, you have to set up the name. In this case, I’m sending to one, but if you have large volumes of data that’s coming through the topic, you should partition the Kafka topic, and maybe you partition into 16 partitions. The maximum scale you can achieve is by running 16 tasks for your connective instance.
Stefan Bocutiu 00:25:38.830 I’m just going to highlight the KCQL syntax, and to find out when you set up the sink, you have to say which Kafka topics you want to subscribe to. Let’s go ahead and create the connector. Takes a couple of seconds, and there you go. The task has been created. Click on the task. It’s running. I can see my configuration. Go back to the audit page. I should see an entry that says that I’ve added a connector. It’s here with all the details. Also, we track and audit the actions done on the processes. So now that I’ve set up the connector, I should see data coming into InfluxDB. Just go ahead and just basically run a Select All from our essential sensor measure. You can see now on the screen I have the aggregated data that was computed based on the SQL processor. So great, we actually managed to build the flow that we wanted to.
Stefan Bocutiu 00:26:47.552 There’s one thing I want to talk about, and that is it will be great to see the journey my data takes up to InfluxDB. I would want to see the entire end-to-end topology. For that, I can go back to the Lenses UI, and at the top of the page, I get the topology link. I click on the topology. I’ll be taken to a page-well, indeed, I will see the entire end-to-end topology. This is not a static graph. Of course, the more data flows that I add, the more complex this graph will be. But as I said earlier, it’s not a static graph. I can actually interact with any nodes in my topology. I can click on the processing, and I can see its own topology that we set up earlier. I can click on my sensor data average topic, and I can get a few information about the topic. And indeed, I can see by clicking on the sink-or InfluxDB sink, I can get the instance of my InfluxDB details, but also the details about the connector, like how many tasks the worker addressed and so on.
Stefan Bocutiu 00:27:59.813 So now we’re done. We actually managed to build our pipeline that moves data from your devices into Kafka. It aggregates that data and stores the result into InfluxDB. Adding the data into InfluxDB is great because the TICK Stack that InfluxData provides, you can set up your rich dashboards. You have seen how using the right tools enables you to build IoT streaming pipelines in a couple of minutes and allows you to focus on extracting values out of your data rather than building infrastructure. You can download Lenses and Stream Reactor, and you’re welcome to contribute to Stream Reactor and Kafka Connect Query Language. You can do all of that by following us on www.landoop.com. Thank you very much for having the patience to listen to my small demo. We can open now the Q&A section.
Chris Churilo 00:28:56.733 Excellent. Thank you so much. So let’s see. Let’s go into the chat window. We’ve got our first question. But everyone else, as we start going through these questions, feel free to put your questions either in the chat or the Q&A. So, Stefan, the first question we have from Vishwas Rao is: Can one connector handle all the data being pushed from Kafka?
Stefan Bocutiu 00:29:24.082 In terms of InfluxDB, yes. Is it? I mean, you will set the connector, but the way you scale it is with connected tasks. Depending on your data volume, you will want to set up multiple tasks when you configure your connector. I think I said in the talk, if you have, let’s say, 16 partitions on your Kafka topic, you will want 16 tasks in your connector. Obviously, it depends on the data volume. Also, one thing to look at, you should not run one Connect worker. In that case, you will want to spin up a couple of Connect workers so you get your failover. So, yeah.
Chris Churilo 00:30:08.544 So, hopefully, that answered your question. If not, just go ahead and type in any kind of a follow-up question that you might have to that. Let me just check. Looks like that’s all we have for now, but don’t be shy. I know you guys always have questions during our webinars. So I guess I’ll just ask a question. So how did you decide to store the data into InfluxDB, Stefan?
Stefan Bocutiu 00:30:39.121 Well, we’ve seen through our community and through the Google Analytics page that InfluxDB, which has been there for I think about two years now, has been quite hot. I mean, as I said, it’s the most used connector out of all of them, and we have 20-plus connectors there, sinks and sources. So we’ve identified that InfluxDB time series database is a very good solution for the IoT world.
Chris Churilo 00:31:09.729 Excellent. And it’s always nice to be able to rely on analytics to determine which one is the most used, which one is the most popular.
Stefan Bocutiu 00:31:17.480 Oh, and just to give a bit of an insight, one of the biggest GPU manufacturers is using that InfluxDB Sink Connector in production.
Chris Churilo 00:31:30.181 Oh, cool. Very nice. So as I mentioned earlier, Stefan will be doing this presentation live at a Kafka meetup at the end of April, so we’ll make sure that we hand that information to you. And as we wait for questions to come in, just want to remind everybody that this session is recorded. So I will post the recording later, and we’ll send the links out to everybody so you can take another listen to it. And if you do have questions later on, feel free to just shoot me an email, and I’ll be happy to forward the questions to Stefan. And, Stefan, you’re lucky today. Everyone is very-
Stefan Bocutiu 00:32:11.804 Yeah, looks like-
Chris Churilo 00:32:12.834 Very shy.
Stefan Bocutiu 00:32:13.113 Yeah. Yeah. I would say they’re shy, yes. Or maybe they’re still digesting the information.
Chris Churilo 00:32:20.935 All right. So Brad asks, “What would be the advantage/disadvantage of using something like Telegraf versus the InfluxDB connectors with Kafka?”
Stefan Bocutiu 00:32:31.378 Well, I’m not here to-I don’t want to actually do a comparison. If you are already using Kafka in production, this is the way we advise to move data out of Kafka into InfluxDB. If you’re already working with a TICK Stack, there’s no point in setting up Kafka since you can use the InfluxDB connector. It depends on the entire requirements you have. How do you want to move data, what you need to do with that data, and so on. I know the TICK Stack is quite powerful as well. At the end of the day, it’s just basically moving a full analysis and see what it works best for you.
Chris Churilo 00:33:14.207 All right. We have another question. So what capabilities do you like about InfluxDB?
Stefan Bocutiu 00:33:22.633 For me, it was because I’ve been using the-I mean, the ease of the client library is very easy. I mean, also the fact that the InfluxDB deals with primitive data makes it also very easy. But, yeah, the library’s very helpful and easy to use. Apart from the fact that I said the-it would be nice to have the timestamp with nanosecond precision for running on the JVM 8, 7, and so on. It will be fixed in JVM 9.
Chris Churilo 00:34:00.860 Excellent. Yeah, well that’s one of our key goals is to make sure that-i
n fact, Paul says it all the time, but “Time to Awesome” is one of our key values that we strive to achieve for all of our developer customers. So the idea that you could just download any part of the TICK Stack and just get up and running very quickly is super important for us, so we’re glad to hear that you appreciate it too. All right. We’ll keep the lines open for the next couple of minutes. Stefan, you really are lucky.
Stefan Bocutiu 00:34:36.505 Yeah. I’m okay with that, so.
Chris Churilo 00:34:38.339 Yeah, I know. You were worried that you would get bombarded, but trust me, they’ll come later. They always come later with the questions.
Stefan Bocutiu 00:34:44.140 Okay. I can go on holiday and come back, and then I-no, I’m kidding. I’ll do my best to answer as soon as possible.
Chris Churilo 00:34:52.505 And just to fill some time, just want to remind everybody that I will be posting the presentations from InfluxDays. We had InfluxDays New York last week, and we had some great customers chatting there. So you’ll be able to take a listen to that. Hopefully, I can get them done before Wednesday, just have to do some quick edits. And we will be hosting InfluxDays London on June 14th, and, hopefully, Stefan will be able to join us, and you’ll be able to meet him if you happen to join us there. All right. We have another question from Anya. Are there future plans to use the other components of the InfluxData platform in Lenses?
Stefan Bocutiu 00:35:38.072 We were planning to add the source as well, look into the Telegraf and actually bring data from the TICK Stack into-from InfluxDB into Kafka. But at the moment, any other functionality in terms of integrating is not yet on the road map.
Chris Churilo 00:35:59.297 Thank you. So maybe you could just describe to the audience, what are some interesting use cases that your customers have? What’s some data that they’re collecting, or some interesting stories about what they’re using that data for?
Stefan Bocutiu 00:36:16.632 I mean, we can’t track everyone that is using the connector because it’s open source, so anyone can just go and pull it up and use it on their own. But what we’ve seen with the GPU, they track at nanosecond precision various parameters of their machines, and that’s quite impressive if you think about it. They raised the bargain. We had to fix that for them because otherwise, they were losing data.
Chris Churilo 00:36:46.605 So most of your customers are pulling data from various sensors?
Stefan Bocutiu 00:36:50.989 Yes, and then pushing it to IoT. I mean, it’s mostly machines and machine data, and, obviously, we’ve seen the space of utility and electricity companies using it a lot.
Chris Churilo 00:37:04.565 Right. And it makes a lot of sense. I mean, there’s a lot of information that they need to collect. Not only about the state of their devices, but also the amount of energy you produced and where that energy is actually being sold to.
Stefan Bocutiu 00:37:17.816 That’s right. Most are seeing InfluxData stack use as an opportunity for Prometheus, to be honest. You’ll actually monitor on the TICK Stack if you do not want to use Prometheus. We’ve seen that in action as well.
Chris Churilo 00:37:33.510 Yeah. We actually have a lot of customers that use both. They’ll use InfluxDB as a long-term store, so they can use-and we built that capability in last year, so it can be either as an alternative or you can use it together. Okay, Stefan. I think you’re lucky. I don’t think any questions are going to come in here, but I do want to thank you for presenting today. I think you did a really wonderful job. And-
Stefan Bocutiu 00:38:00.849 Thank you.
Chris Churilo 00:38:00.959 -for all the attendees that are still hanging on-and, actually, almost everybody’s still here-I know you probably have some questions, so feel free to just shoot me an email and I will forward it off to Stefan. If you get a chance to be in London, either, at the end of April, please go to the Kafka meetup, and we’ll be pushing that out on social media to support that. Or come join us at InfluxDays in London, and we will try our best to get Stefan to come join as well so you can meet him in either of those venues.
[/et_pb_toggle]
