How to Store and Visualize CAN Bus Telematic Data with InfluxDB Cloud and Grafana

Session date: Apr 27, 2021 08:00am (Pacific Time)

CSS Electronics develop & manufacture professional-grade, simple-to-use CAN bus data loggers. Their plug-and-play CANedge2 logger records time-stamped raw CAN data to an extractable industrial SD card - and connects via WiFi/3G/4G access points to upload the data to the end user’s own servers. The CANedge2 is ideal for collecting automotive sensor metrics like speed, temperatures, state of charge, GPS and more. Learn how to create your own telematics dashboard built on InfluxDB in minutes by attending this webinar!

Join us as Martin Falch dives into:

• CSS Electronics' approach to improving R&D field testing, diagnostics, fleet management and predictive maintenance
• The CANedge's methodology for collecting IIoT data from cars, trucks and machines
• How they process time series data from S3 via Python to store it in InfluxDB and visualize it with Grafana

Click here for presentation

Watch the Webinar

Watch the webinar “How to Store and Visualize CAN Bus Telematic Data with InfluxDB Cloud and Grafana” by filling out the form and clicking on the Watch Webinar button on the right. This will open the recording.

[et_pb_toggle _builder_version=”3.17.6” title=”Transcript” title_font_size=”26” border_width_all=”0px” border_width_bottom=”1px” module_class=”transcript-toggle” closed_toggle_background_color=”rgba(255,255,255,0)”]

Here is an unedited transcript of the webinar “How to Store and Visualize CAN Bus Telematic Data with InfluxDB Cloud and Grafana”. This is provided for those who prefer to read than watch the webinar. Please note that the transcript is raw. We apologize for any transcribing errors.

Speakers:

• • • Caitlin Croft: Customer Marketing Manager, InfluxData
    • Martin Falch: Co-Owner, CSS Electronics

Caitlin Croft: 00:00:04.111 Welcome to today’s webinar. My name is Caitlin Croft, and I’m very excited to have Martin joining us from CSS Electronics. And he will be discussing how to store and visualize CAN bus telematic data with InfluxDB Cloud and Grafana. Once again, please feel free to post any questions you may have for Martin in the Q&A, and we’ll answer them at the end. Well, without further ado, I’m going to hand things off to Martin.

Martin Falch: 00:00:36.533 Thanks, Caitlin. All right. So thanks, everybody, for joining. And today, we will be talking about how to visualize CAN bus data and specifically from the CANedge2, and how to do it using Grafana and InfluxDB. And if some of you can see me on the video, I’ll be talking a lot about this small device here, the CANedge2, which is essentially our data logger with Wi-Fi and which is the solution that we’ll be discussing as part of today’s session. To give you a quick overview, we’ll be running through just a bit about CSS Electronics, about CAN bus so that you have a basic understanding if you’re not already familiar with the protocol and the CANedge2. And then, after that, we will run into the actual telematics dashboards, essentially why we have chosen InfluxDB and Grafana, how we implemented it. And then the cool part, which will be running through some of the actual dashboards that we have made, as well as some of the cool stuff that some of our users have made using the integration.

Martin Falch: 00:01:43.851 But let’s start with CSS and myself. Now I have this on top here. So I’m one of the co-owners at CSS Electronics. I’ve been with the company for five years, and my responsibilities are somewhat broad. I’m head of sales but also helping, you can say, all of our customers and users with tech support. Besides that, I’m also responsible for our marketing. So that involves our website, videos, search engine optimization, and other things. And then a bit with my left hand, I do some software development both in, React, Python, Ruby, and other tools and other languages. My background is within management consulting and corporate strategy, and I have a degree in financial economics, which are two things that are not immediately or obviously relevant to the stuff I do today. And that’s how I think sometimes it turned out. I live in Denmark with my wife and three kids.

Martin Falch: 00:02:39.602 And just a bit about CSS. As a company, we started in 2015, and today we supply 2,000-plus companies globally across 80 countries. Our specialty is CAN bus hardware and specifically CAN bus data loggers. So that means recording data to an SD card. But we are also branching out into additional modules, and that’s stuff we’ll get into in a moment. We do PCB layouts. So the stuff inside the devices, we do that from scratch, which is one of our specialties. All of our firmware is done in embedded C, which might be interesting to some of you, so we’re not using a Linux or other type of system inside. And the assembly of our products is done by our US partner for many years. Other than that, have been carbon neutral since 2019 through offsets. And you can say as an overall perspective - or overall, our products are used particularly by automotive and industrial OEMs. So the users that we cater to are engineers. And they are typically engineers in research and development teams, for example, trying to diagnose new equipment that they are producing or manufacturing for various automotive applications.

Martin Falch: 00:03:59.638 In order to understand today’s webinar, I thought it would be useful just to have a very quick basic on - or basic intro on what CAN bus is. CAN bus is a network. It’s essentially like the nervous system inside your car, but it’s not just your car. It’s essentially anything with wheels. And you can see it a bit in this next picture here. So pretty much anything that moves uses CAN bus today, whether it’s within aerospace, maritime and the two-wheelers, four-wheelers, trucks, mining, forklifts, agriculture, many different things, but also even down to small e-scooters. Those are some of the typical segments that we cater to, but it is also widely used within industrial automation. So things like robotics and production machinery also use CAN bus. I won’t go fully in depth on how CAN bus works. You can read the small intro that we link to in the slides here, but suffice to say, it is an extremely popular and extremely widely used protocol. And what we do is essentially to help record the CAN bus data from different applications.

Martin Falch: 00:05:14.846 You can say the way that works is that if you take your car, for example, today in order for your car to operate, the CAN bus traffic essentially helps - or the CAN bus network helps communicate traffic between different parts of the car in order to facilitate communication so that the car is able to operate. But all of this traffic is usually not captured, meaning that it is used instantaneously and consumed by the different ECUs within a car. But it is not recorded in any way. It’s not stored on a server; it’s not stored on an SD card. And most of the time, that’s obviously not necessary either. But if you want to diagnose something, if you want to monitor something from a vehicle, then you need a way to record this CAN bus data. And that’s where we come in.

Martin Falch: 00:06:05.916 To get a bit of an understanding of what this looks like, I’ve just taken one sort of deep dive into the whole CAN bus logging aspect, and that is because I think it is important for those that are not familiar with CAN bus data logging to understand how it works. If you take a usual IoT device, you might be thinking of a temperature sensor or GPS sensor that monitors a specific value and sends that as a physical value, as we would call it. That could be a degree Celsius, or it could be a longitude value or a temperature of other types. That’s not how CAN bus works by default. By default, you will record what you see on the left-hand side here on this slide, which is essentially a timestamp data with identifiers and data bytes. And if you’re not familiar with CAN bus data, the stuff on the left side over here will look like nonsense to you, and you will not be able to understand it. On the right-hand side, you have what we typically call physical values or decoded CAN bus data. And here you have stuff like engine speed, like rpm in your car, speed in kilometers per hour, fuel level and percentages, etc. So how do you get from here to here? That’s what we illustrate by this small drawing down here, that you would record some raw CAN data. And then there is this metric over here called a DBC file or essentially a small database. What this file does is that it tells you how to interpret the raw data over here and what it means when you’re seeing a specific ID and a specific set of data bytes. So if you have a software tool that understands these two components, you’re able to produce the stuff on the right-hand side, which is what we will be focusing on today. So essentially time series data.

Martin Falch: 00:08:04.385 If we look at what kind of stuff we do and what type of challenge we solve with our products, you can take a typical situation from one of our users. So you might have a forklift battery OEM that needs to collect data from a prototype vehicle. It could be a prototype battery. And they may need to do so across hundreds of units and across different warehouses located in different states, for example. So that would be a typical use case. There are some complications to this setup. First of all, one complication is that these users are not your average users. They are engineers, and they have very specific requirements to what a product like this needs to be able to do. I’m not going to include all of them. But essentially, it relates to the state or the capabilities of the device in terms of performance and in terms of different functionalities and different configuration options.

Martin Falch: 00:09:04.445 The CANedge2 that we deliver essentially cater to these requirements, and it enables users to collect their data in the specific way that they need to do it and to do it in an automated way. So instead of installing a device in the forklift and then manually sending a technician to fetch the data on-site, the CANedge2 is essentially able to store data from the forklift onto the SD card, as you can see over here. And when the CANedge2 moves within range of a Wi-Fi access point, it is able to automatically off-load the data from the SD card onto a server hosted by the end user. The server could be a cloud service, as we’ll get a bit into, or it could be a local self-hosted server. It could even be a server hosted on a Raspberry Pi if you want. And on this server, you can then manipulate and process the data in different ways, as we’ll look into today in terms of dashboards, for example. I’m not going to go deep into the CANedge2, but if you want to read a bit more about how this device works and the functionality, there are some links in the presentation that you can run through afterwards. But just to give you the basics in terms of the understanding of how the machinery in this works, the first step you would do if you were to deploy CANedge2 would be to set up your own server. We use a server concept called S3, which is like a modern equivalent to FTP servers. S3 servers are used very broadly. Amazon S3 is the most common one, but there are also many other cloud options like the Google Cloud has a S3 version, and you can use self-hosted servers as well, like MiniO S3 servers.

Martin Falch: 00:10:54.273 Setting up an S3 server can sound a bit - are you able to do this if you’ve not worked with servers before? But it’s actually very easy. Once you’ve done this, you would then configure our device with your server and Wi-Fi details. So it could be a Wi-Fi router in a warehouse, or it could be a USB hot spot with a SIM card inside. When you have configured the device, and you plug it into the vehicle, it will automatically start recording the data. And when a new log file is created, for example, after five minutes, that log file is uploaded to your server, and once it’s uploaded successfully, it’s deleted from the SD card. So that’s essentially how the logging process works. And on your server, you can then process the data, for example, using Python APIs, as we’ll look a bit into. There are many use cases for this, so we talked about warehouses as an example. But it’s also very commonly used within agriculture, like if you want to monitor agricultural vehicles across the harvest season, for example. It’s also used a lot within automotive when OEMs need to basically monitor and debug equipment within cars. It’s also used within ships and boats in the maritime vessels. It’s used a lot within heavy-duty trucks and transit buses, and then it’s also used within industrial automation, like robotics.

Martin Falch: 00:12:20.259 Just a bit about what makes our solution different from some of the other solutions in the markets. We essentially try to put ourselves in between two different markets. So a big market today is within “pro specs”, as we call them, professional-grade automotive data loggers. There are quite a few of these on the market, both from European and US manufacturers. They’re typically three to four times more expensive than our devices. So our starting point was essentially to create a [inaudible] or lower-cost alternative to these devices while still providing professional-grade specs that can compete with them. More recently, a large share of our focus and also sales have moved into the CANedge2, which adds the Wi-Fi functionality and the ability to remotely collect your data. And by doing so, we also gradually shift a bit into another segment, vehicle telematics, which is your classical having a small dongle in a truck doing fleet management, these type of things. But we do not cater typically to large fleets or aftermarket customers. We typically cater to OEMs and engineers at OEMs that need to get this type of professional data logging, but they also need a convenient way to collect it at scale. So that’s a bit about sort of the concept.

Martin Falch: 00:13:44.566 Another thing that sort of characterizes our approach is we do not charge for our software. So we’re really passionate about open source solutions. Personally, we dislike it when you charge subscriptions to a product like ours. So we try to move that outside of our offering, and we do so by integrating with many different open source tools, and we do so by standardizing our integrations and our interfaces. And one of the perhaps last things that’s important to know is, as a result of these things, we also do not host anything. We do not host servers. We do not sort of have a cloud server that we collect all of our customers’ data both because the data is typically extremely sensitive to our customers but also because it’s just not part of our concept. And the reason we can do this is, again, that our customers are quite smart and they know what they’re doing. Otherwise, this would be an impossible setup. That was a bit about CSS Electronics and just what we do and how the CANedge2 works, and if there are any questions at the end of the webinar on that, we can definitely run through that a bit more.

Martin Falch: 00:14:57.099 Then let’s talk a bit about the actual InfluxDB and Grafana inspiration and how that works, and why we did this. So the starting point for us when we looked into - when we looked into the solution, was that we had a lot of users communicate and report to us that while they liked the CANedge, they were lacking a solution for visualizing the data in the browser. And the whole concept of dashboards was typically mentioned and typically came up as what they were looking for. And before we looked at this inspiration, we had various tools for the CANedge, which we still do today. We have configuration editors for easily configuring your device. We have the conversion tools for easily getting the log files into other formats. We have some standard PC tools for decoding the data using the PC files and visualizing it, as you can see down here. We also had a Python API if you wanted to automate your processing of the data. And then, we have a browser-based tool for managing devices to data, but not for visualizing the data. So to us, it felt like the browser dashboard integration would be a really nice addition and would supplement many of these tools really well. So we looked into how do we best create a dashboard inspiration for our device, and the obvious challenge is, how do you? Our users, they want to visualize their CAN bus data in the browser, and they want to create a classic telematics dashboard if you want. But there are some sort of complications to this.

Martin Falch: 00:16:41.251 One of the complications that we had is that when you look at dashboards when you look at Grafana, there are typical use cases. So InfluxDB and many of the typical use cases shown there, you’re typically looking at something like server monitoring or PC monitoring, figuring out if there are errors like an IT department administration dashboard kind of tool. And whatever characterizes those type of solutions is that it’s often fine to have a 10-second resolution, and if you’re going down to 5- or 1- second resolutions, then you are pretty crazy. In our case, we need often to be able to visualize data and to drill down at the 1-millisecond resolution, so that’s 1,000 times per second. And that puts some restrictions on how do we actually set this up, how do we get performance into this, and what kind of integrations would we require.

Martin Falch: 00:17:38.775 Another thing is because we do not host anything and because we do not cater to one specific use case or one specific segment of users. We need to enable our users to customize their dashboards for exactly their own use case. We don’t know if this is going to be used in a plane or in a ship or in a car or in a truck or if it’s going to be used in a drone or for something completely different. Each of these use cases will require different types of custom dashboards, so it needs to be very easy for our users to actually do this. And if it’s not easy, then we’re going to drown in support. So the setup of this also has to be released. In addition, because of our concept, as mentioned before, we wanted something that is optionally at least 100% free and open source. At the same time, because many of our users are not just using 1 or 2 or 5 or 10 devices, but many of them use several hundred devices in the field, it needs to be possible to scale this solution in a convenient way.

Martin Falch: 00:18:46.179 The solution that we found to be the most sort of suitable for this challenge here was to essentially use our Python API for processing the log file data and then to write the decoded data into InfluxDB and visualize it in Grafana. And the way we do this is to provide a step-by-step guide and some plug-and-play scripts that I’ll show in a moment. But before we sort of jump into how it works and how we did it, I think it’s useful to just understand why specifically Grafana, why InfluxDB. So it doesn’t have to be these two solutions. In principle, you could use our Python API to integrate with other solutions. But for us, we needed something that we thought worked well, as at least the way to get started for many of our users. Grafana, we chose because it’s a very popular tool in our opinion, and in our research, it’s perhaps the most popular open source dashboard tool available, and you can see it a bit from the 40K GitHub stars that it has. Grafana has a really nice and intuitive front end for customizing your panels and for customizing your dashboards. It’s free and open source. It has a nice optional cloud starter account, so you can essentially just log in with your Google details or your mail and create an account that is quite unrestricted in the free version. Another cool thing is that Grafana has hundreds of plug-ins, including geo map plug-ins, and it has easy support for variables and alerts that I’ll show in a moment. So we originally selected Grafana. And then we needed to find - fine, Grafana is the front end. What is going to be the solution for our database, where we essentially need to store the decoded data or the physical values?

Martin Falch: 00:20:40.779 And we looked at InfluxDB, again, because it’s popular and it is natively supported by the Grafana. And it came up in our research as one of the most popular data sources for Grafana. It’s free and open source. And then importantly, InfluxDB has an optional cloud, including a free starter edition. So not all of the time series databases that are available has this option. And the interface we found for InfluxDB is really nice. So as you can probably imagine, the combination of a cloud edition of Grafana and a cloud edition of Influx, that combination eliminates a lot of starter issues such as firewalls and port forwarding, and other things. And combining these can be done very fast. Then we also found that it was quite easy to integrate in Python with InfluxDB, which was important for our use case, and it has a very nice UI, user interface. If you log in to the cloud, you can essentially manage things without knowing scripts or anything like that. And then, while not mentioned here, one of the important things about InfluxDB is that it also supports intra-second resolution, in contrast to some of the other time series databases out there.

Martin Falch: 00:22:02.558 So to give you a quick idea of how we actually implemented this. The way we took - or the route we took was essentially to have an article that is quite high level to explain the concept and what you can do and how it works. The reason this is important is that traditionally within this type of segment, the way it’s usually done is that the company like ours would be hosting both the database and the dashboards. And users would simply be provided with a log-in, and then they have the solution available. And there are pros and cons to this concept, but it’s definitely a lot different than what we do. And therefore, we need to explain that up front so users actually know what they’re getting. Secondly, our users, they need to be able to set this up themselves. They need to set up an InfluxDB account themselves. They need to set up a Grafana account themselves, and they need to use the Python script and deploy that themselves. So in order for users not to drown in this and not to get frustrated, it is critical for our solution that the “time to awesome” is really small - or really short. So therefore, we need to create a simple solution as the starter solution, and you can see it up here and visualize a bit how we went around it. So we have a guide that I’ll show just in a moment where we recommend users to set up the free cloud starter versions. So users may have, for example, an Amazon S3 server where they are uploading data to from their CANedge2, but they could also be one of the other S3 server options available, like the self-hosted options.

Martin Falch: 00:23:47.610 Then we recommend as the starting point that users set up an InfluxDB Cloud account and a Grafana Cloud account. With these, you can essentially then deploy the Python script that we promote that we have created as a plug-and-play script. You can run that on your own PC that will then fetch data from your Amazon cloud server or other S3 server. It will process that data to essentially get or extract the physical values, which it will then write into the InfluxDB database. And then, you connect Grafana with the InfluxDB Cloud, which is really easy because both of them are clouds. So there is no sort of funky stuff going on. It’s just very, very plug and play. And once you’ve done this, you are essentially ready. You will have your data available from Influx. You can start creating panels in Grafana. Sp really, really easy. And just for the sort of technical side, for those of you that are familiar a bit with InfluxDB, we essentially decided in terms of our structure that we try to use measurements to those we set equal to our device. Serial number show each CANedge2 has a unique eight-character serial number. And then, each parameter, like speed or rpm or temperatures, will be equal to fields within the context of an InfluxDB database.

Martin Falch: 00:25:13.880 So now I’m going to jump a bit out of the presentation and just show you a bit of things from how this is actually set up just to give you a better idea of how it works. The first thing I just want to show is essentially the guide. So within our introduction to the CANedge2, we essentially provide a step-by-step guide on how to set this up. And I’m just going to run through it really briefly to just give you an idea of what steps are actually involved. So the first thing, as we discussed, is to set up an InfluxDB account and get the credentials. And as you can see in our guide, what we tried to do in order to make this easier for users is to visualize, to show with videos and GIFs and the like, where do you actually get each of the details and then show examples of each of the details you need, like the bucket, token, organization ID and the endpoint for your Influx database. And then, you set up a Grafana account, and you link it to your InfluxDB database. And again, we try to visualize how you do it, but I would say that part is really simple.

Martin Falch: 00:26:17.987 We recommend users to start with the cloud editions, but if they want to self-host everything using Docker, for example, we also have a small guide for how you would do that inside the guide here. But assuming you have the cloud account set up, the next step would be to actually start writing data using our Python script into InfluxDB. And what we promote here - or what we provide here for our users is a script that is designed to be plug and play. So if you do not know Python, you do not have to know it in order to actually use this tool. It’s an advantage I would say to have worked with some type of coding in the past, in particular, if you want to do more advanced stuff. But in principle, you can follow the guide in here, install Python, download the script along with our sample data, and then you can actually use that sample data to write some parameters into InfluxDB without having a CANedge2. And that is important for us because we typically prefer users to try this out before they decide whether they want to buy a CANedge2. Because if they don’t like the whole setup of how it works, then it’s better for everybody that you do not buy the device at all.

Martin Falch: 00:27:34.846 Once you’ve set up the script and you set it to run the first time, then the next step that you would typically want to do is to automate it. So let’s imagine now that instead of using a sample data, you have set up the script using some of the input variables so that it fetches data from your own server and it writes the data to your InfluxDB account. Now you could want to automate it, which you could do in many ways. Let’s say you’re running it on your PC. You could use something as basic as Windows Task Scheduler to run the script on a periodic basis so that every half hour, it checks your server for new log files that have not been processed before, and it drives the data from those into InfluxDB. An alternative way that some of our users also do is that they use event triggers. So, for example, if you have an Amazon cloud server, you can use something called AWS Lambda to trigger a script execution every time a new log file is uploaded. So it depends a bit on your preference and what type of scale you’re running this at, what you prefer. But there are different options we describe in here. Once you’ve done this automation, you should essentially start having data run into your InfluxDB instant, and you can then customize and set up your Grafana dashboard. And we provide a bit of tips and tricks in here, and Grafana also has a good tutorial in getting started with this. So what we try to do this to essentially provide users with some plug and play current examples so that it’s easy for them to get started if you just want to visualize one variable, or, as in this case here, the two variables. How do you do that? So here we hard code these, and we also hard code the serial number of the device that you’re showing this for. So I’ll show a bit of variations on this in a moment.

Martin Falch: 00:29:26.159 Those of you familiar with Influx will know that this is the Flux query language which we selected, basically, based on some research, and because it seems like Flux is the one - or the newest of the Influx languages. And hence we wanted to use something that would most likely be the more long-term supported one. But I’m sure Caitlin can comment on that in the end what might be relevant here. So this is essentially the guide, and as you can probably imagine, one thing is going through a bit of a dry guide like this, so it might be interesting to some of you how we actually try to promote this so you can see what you’re getting from this. And what we do a lot on our website is we try to make it very visual and very sort of easy to see how do the tools work, how does the software work. So we have this small overview here. And when it comes to the dashboard, we actually have a playground that you can jump into.

Martin Falch: 00:30:25.004 So this is a public playground, and you can go in, and you can actually play around with it and see how it works. So as an example here, we have that playground that shows some data from a truck. So this is data - or this data is called J1939, is a variation of CAN bus data that is used within heavy-duty vehicles like trucks and buses and other applications. The way it works in here is essentially that you create these panels. So if I wanted to create a new one, I could do this. And within this panel here, you would put in a piece of code. And if we just jump back into the example from before here, you can essentially take this. Let’s see if it works. Yeah, it actually worked. So you can see that what we’re doing here is essentially to fetch data from a specific device, and we are fetching the two parameters here, rpm and wheel-based vehicle speed. And when I save this query in here, it’s essentially fetching this stuff from InfluxDB and visualizing it up here.

Martin Falch: 00:31:28.998 So if you’re familiar with Grafana, there is not a lot of hocus-pocus in here. But if you’re not, I think it’s useful to just jump into the playground after this webinar and try and play around with it. You can do a lot of cool stuff in terms of visualization and how you want to show things in here, both in terms of the visual presentation of the panels but also in terms of different options for the queries here. To give you an example some of this stuff - just going to delete this here. But some of this stuff we found really, really important for our type of use case was the ability to enable users to have some level of control over the dashboards. So it’s one thing to show aesthetic dashboard with certain parameters that somebody who created this dashboard decided to show. But it’s nice to be able to have the capability in a live way and from your browser to show other parameters. And that’s what you can do with the Flux query language as well. Essentially, what you can see down here is that we are specifying in some of these panels that instead of taking a specific predefined device, it should fetch the device or the multiple devices selected up here from a dropdown. And instead of selecting a specific predefined parameter, it should select the parameters that I select from this dropdown here and then show those. And so that’s what it’s saying down here.

Martin Falch: 00:33:01.789 And this current example, of course, is also shown in our guide. And it’s a nice way to allow any front-end user of this dashboard in their own session here to make changes and to visualize different data. Of course, you can also quickly jump in and out in terms of the time range. So if I wanted to zoom in on something, I can easily do that. And I’m going to show an example of why that can be extra important in a moment. Another thing that was useful for us with Grafana is a plugin called TrackMap. That’s what you’re seeing on the left-hand side. So when I’m actually scrolling through our data here, you can see that the GPS position on the map is changing as well. That means that I’m actually able to see what is a specific parameter value at a specific point in time. And that’s often useful if you’re doing diagnostics or you’re doing other type of analysis of automotive equipment. Where was the vehicle when this occurred at a specific event, for example? Another nice thing about Grafana and also InfluxDB, I would say, is the ability to set up alerts. So you can essentially say that you want an alert if a specific event occurs, for example, vehicle speed going above a certain level. And again, here, you could be creative. It could be based on calculated signals that you specify in Python in a specific way. You can do essentially whatever you want with these alerts. The key thing is that it’s easy to set up, and you can get your alert notifications via text or via email or via slide notifications or other things. So that’s a bit of an example of a dashboard we used in J1939 or in the heavy-duty case. And if you go into the playground, you can actually find all of our dashboards by clicking over here and go through the manage section.

Martin Falch: 00:34:53.092 One of the other cases I’m just going to quickly show us here from a car. What is cool about this case is that we’re actually using one of our new products, a GPS module, which you can combine with our CANedge in order to add a GPS data to whatever data you’re recording. So what that looks like is essentially down here. I’m just going to show you quickly. So you’re essentially combining the CANedge2 with a module. And it could look like what you’re seeing in this picture here, where the module is pushing data into the CANedge through one of the channels, and at the same time, the CANedge is recording data from a car through the first channel here. The combination of this data allows you to show things, for example, what is the fuel level in the car that’s based on the car’s data through something called OBD2, but you’re also able to see things like the GPS position. And again, the cool thing is you can monitor how these variables are changing over time. So if you want to see, for example, has there been a period with a specific fuel consumption, you could show things like that. And you can also show, you can say, data around your dynamic vehicle behavior. So beyond GPS data, the module here also provides a different accelerometer-gyroscope data so that you can see accelerations whether you are driving over a road bump, if you’re doing harsh braking, that kind of stuff. Just to give you a bit of an idea of what type of applications this is also used in, this is the dashboard from one of our users who is using the device in a ship. And again, you can see that the principle is the same. The whole construction of the dashboard is the same, but the user is able to set up the dashboard in the way they want for a setup that might be suitable for a maritime in this application.

Martin Falch: 00:36:46.810 The last dashboard I want to show in this playground I walk through here,  is just an example of how the dashboard concept can also be used, and also I think how easy it is with InfluxDB and Grafana to set this up. So, again, this GPS module that we mentioned before. We have been developing this with InfluxDB and Grafana as one of the tools in evaluating performance and evaluating the functionality of this device here. So in order to do that, we have a run sort of a simple test where we are putting a CANedge2 in the trunk of a car, a Nissan Leaf, over here. And then we’re actually hooking up the two GPS modules in sync right next to each other so that they should ideally yield roughly equivalent data. The only sort of trick or sort of complexity to this is that on one device, we’re using a functionality called sensor fusion. And that functionality is essentially able to use some of the information from the accelerometer and gyroscope in order to provide a GPS position, even when the vehicle is inside a GPS hostile area such as a parking lot. And I think this is an excellent demonstration of what you can do with Grafana and InfluxDB because trying to visualize the effect of this in other tools is practically impossible. Trust me; I’ve tried.

Martin Falch: 00:38:18.541 But if you check it out here, you can actually see the process. So on the left-hand side with the orange line, you’re seeing the GPS module where sensor fusion is not enabled, and then on the right-hand side, you’re seeing the GPS module where this sensor fusion functionality is enabled. And what you can see is that they are equivalent here in the start as I’m driving out on an open road. But as I’m moving into the parking lot here, you can see that the fix down in the lower-left corner, which is essentially whether the device is able to catch the satellite signal that drops, and for the orange line, it drops to zero, meaning it has absolutely no idea where it is. And as a result, you can see the orange line. It just stops. But on the blue line, the fixed type is one, which means that it still has the functionality of sensor fusion. It still knows the last position of the car. And from there, it’s calculating based on the acceleration and the gyroscope data where does it think the car is. So you can actually see the two dots here, and the second one is actually moving through the parking lot inside an area with no GPS signal. And it’s actually able to track the route almost perfectly inside the parking lot all the way to the end of it, where both of them regain the signal. And this type of functionality is not only important when it comes to the position of a signal or the position of a vehicle, but also if you’re using that GPS module for other data like speed, altitude, heading, roll, pitch, and also odometer distance. Without this functionality, these values basically disappear, as you can see from the orange line. But with sensor fusion, you actually are able to continue having data in these areas.

Martin Falch: 00:40:08.752 So this is just a good example of how it works and also a nice example of something else you can do if you have a bit of a crowded dashboard. You can use these rows inside Grafana to sort of collect your panels within different sections that you might think would be related, and you might think would be interesting to monitor at the same time. The last thing I want to show here on sort of the dashboard examples is just a bit about how have some of our users done this. And what always impresses me is when I talk to our users, the stuff that they’re able to do is quite amazing. And therefore, we try to keep these case studies on our website, and you can go and check them out. We have around 40 case studies. And we try to filter them both based on category and title but also based on these types here. And one of the types that you can find in here is telematics. I have it here. If you click this, then it will filter for all of the telematics use cases that are available in here.

Martin Falch: 00:41:10.535 And it’s just a really cool - I mean, we have some users that deployed the CANedge2, for example, in electric transit buses to collect data to their own cloud servers. And then, they visualize the data in order to investigate the implementation of electric batteries. We have another use case here with a defense vehicle, a smaller sort of robotic vehicle or automated vehicle that is driving around here with a CANedge2 on it. And when it gets within range of a Wi-Fi access point, then it will off-load the data, and the data is then visualized in a dashboard that the user set up. And I think what is cool about this is - I mean, we haven’t helped this user set this up. They just do it themselves based on the guide. And there are plenty of examples in here that you can check out in terms of these dashboards, and there are more down here if you just scroll through them in here. But I think that’s probably some of the best illustration for us that this is a nice concept base that the users are able to do it themselves. And in the presentation, we’ll provide some links for some of the dashboards you’re seeing here so you can check that out afterwards and try the playgrounds yourselves.

Martin Falch: 00:42:22.663 And then just a bit here towards the end in terms of pros and cons of the implementation. For us, I think it’s important to say it like this because there are both good sides and bad sides to this way of doing it. So I think on the good side, it’s easy to get started with this because of the InfluxDB Cloud starter option, as well as the Grafana Cloud starter option. That makes it really, really fast to get started. And also, if you have ever worked with Python, you’ll find it very easy to run the plug-and-play script that we have for it, and you can test it out with the sample data. If you haven’t worked with Python at all, you might find a bit of learning curve just installing Python and getting that to work, but nothing that can’t be handled within [inaudible]. Then a nice thing about this is you can self-host everything if you want. And then it would be 100% free and open sourced. You wouldn’t pay anything, so you could use a self-host at S3 server. You could use a self-host of InfluxDB database, and you could self-host Grafana as well, and you could set everything up using Docker. So if you’re familiar with that, great, you can do it. If you don’t want to bother with that or you want to have something that scales very easily, you can also just opt in for the cloud versions of everything, which I think, again, makes it easy for users to choose what they like amongst these options. And then, users can fully customize the front end in Grafana without having to be developers and without doing any scripted part of it or any scripts involved in this.

Martin Falch: 00:43:59.882 A downside is probably that you can say if you were to create a more clean or elegant solution, that might be to visualize data directly from a stream so that you could directly fetch the data that is already on a stream and visualize it. The challenges, as I describe from the start, the data on your server uploaded by the CANedge2 will be in a raw form. It will be raw CAN bus identifiers and raw data bytes. So you need some sort of mechanism to convert this into time series data, and you might be able to do this live. But it’s difficult to get performance when doing this live. In particular, if you need to drill up and down across weeks of data at the same time as being able to go in and look at one millisecond data as might be relevant, for example, if you are analyzing acceleration data and other things. The other thing that can be a bit of a downside to this is that if you’re doing periodic uploads through the CANedge2. So let’s say you have a CANedge2 operating on a truck in the field for three days, and it’s accumulating data. Then it gets back to the garage, where there is a Wi-Fi access point. Great. It then starts uploading the data, but then suddenly, your data processing script will have to crunch a lot of data at once. So it’s not necessarily possible to, you can say, flatten out this data processing exercise over time, which means that you need to think about this when you set up your solution for scale. And then I think another thing is when we started this, at least the Flux support in Grafana was still semi-new. But I think a lot has happened here in the most recent months in terms of making this report more stable and more full-fledged.

Martin Falch: 00:45:49.684 I think that was pretty much it. Also, just leaving time for if anybody has some questions on this. And so what we wanted to do with this webinar was primarily just to illustrate how we use InfluxDB and Grafana to create a solution for dashboards. And I hope it inspires you to go and try it yourself using some of the same sample data and trying to set up your own accounts to get started with this. Great.

Caitlin Croft: 00:46:19.759 Awesome. Thank you, Martin. That was fantastic. So before we go into all the questions, I just want to remind everyone that we have InfluxDays coming up in May. It’s actually just in a few weeks. And Martin did touch on the Flux language, so if you’re interested in learning more, be sure to register for that training. All right. Martin, there’s tons of questions here for you. The first one is regarding the device. In addition to Wi-Fi connectivity for warehouse, could it offer LTE or 3G connectivity in case there is no Wi-Fi?

Martin Falch: 00:46:56.506 Yes. The concept that we use for our devices is very much based on modularity. So the CANedge2 in itself has the ability to connect to any Wi-Fi access point that could be like a router in your office, the router you have at home for your own Wi-Fi. You could put in the name of that Wi-Fi and the password, and the device would be able to connect through that. But if you are in a car and you want to upload data while on the road, so to speak, you may want to do so using a cellular hot spot instead of a fixed Wi-Fi access point. So what many of our users do is that they deploy the CANedge2 with a small USB hot spot like in this example here, where the CANedge2 is essentially powering the USB hot spot here, and then it connects to it via Wi-Fi. You could also just put the USB hot spot in your car’s USB power supply as if you were creating a Wi-Fi network in your car, and the CANedge2 would, of course, also be able to connect. But this solution here is just useful in the case where you cannot rely on there being an available power supply in the car. So that’s one way that some users do it. Others, they use different cellular routers, but the CANedge can essentially connect to any of them as long as it’s a Wi-Fi router, so to speak.

Caitlin Croft: 00:48:18.329 How about integration with NMEA 2000 data? Do you have a DBC file for that application?

Martin Falch: 00:48:25.882 Yes. It’s a great question. We have a guide. If you go to our website, on the guide section, you will see both our introduction guides to different protocols, but also these application examples that you can see down here for different types of applications. So one of the applications we have is within the NMEA 2000 protocol, so for boats and ships and the like. And we are actually going to add a section very shortly on how do you create a DBC file within this area. So in various basic terms, you can get the standard NMEA 2000 from the NMEA organization. It’s a very long and slightly cumbersome PDF, but we will provide a guide on how you can extract the data you need from this and create a DBC file. And this will probably be up within the coming week and will be added in this article. But you can also contact us for getting the guide immediately.

Caitlin Croft: 00:49:24.130 Could you also integrate with an existing SCADA system from a different vendor, for example, Siemens? Or is there an integration required?

Martin Falch: 00:49:33.728 What did you mention, Caitlin? System?

Caitlin Croft: 00:49:38.089 SCADA?

Martin Falch: 00:49:39.361 SCADA system from Siemens?

Caitlin Croft: 00:49:41.016 Yeah.

Martin Falch: 00:49:41.846 No. We don’t integrate specifically with it, for example, proprietary data systems or specific data systems. So we have use cases involving Siemens where they are using our devices, but we do not have a specific integration, for example, with Siemens’ products. But again, you can integrate with pretty much anything. So here in the presentation, we went through how you integrate the data with InfluxDB and Grafana. We have other users that utilize our Python API to write data into MySQL server - or MySQL databases [inaudible]. You can do different things. So everything is possible. It’s just a matter of whether you are able to do a bit of the work on your end on the integration.

Caitlin Croft: 00:50:29.559 Is it possible to process compressed log files from CANedge2 with the Python API?

Martin Falch: 00:50:36.911 That’s a great question. It will be soon. It is not a native feature in the API today, which I’m sad about because I think compression is basically a functionality of the CANedge2 that reduces the data size by 50 to 70 percent, which is really nice and should be used by everybody. The only reason not to use it right now is that our Python API does not natively handle compressed files, and that is on our road map and on our short-term road map. But you can do it now if you use our converter tools, and you can use those in a programmatic way, but it is a bit more cumbersome than if it was natively supported by our API. But for these things, I mean, definitely contact us and let us know your thoughts on it, and we will try to also push this in the coming weeks.

Caitlin Croft: 00:51:27.012 Do you need an interpreter somewhere? Because CAN messages consist of CAN-ID and that up to eight bytes payload and InfluxDB cannot handle

binary data.

Martin Falch: 00:51:39.629 That is exactly correct. So the dashboard-writer script, as we call it on our GitHub page, it essentially uses different API modules that we have for processing our CAN data. So what happens in this API - or as part of this API script here is essentially that it takes the raw CAN bus data and it turns it into decoded data. The way it does this is essentially shown in this very basic - or high-level script. So it essentially initializes the connection to your Influx database. You specify what time period you want to get data from, then you create a link to your file system, for example, your S3 server, to be able to get the raw data. Then you put in the path to your DBC files. And the DBC files are these magic keys, so to speak, if you’re not familiar with them. But they tell you how to go from binary or from raw CAN bus data into something useful and into time series data. And then you load your log files. And what happens here is essentially that you extract the raw data from the log files into a Pandas DataFrame. And this Pandas DataFrame is then processed by our API tool here in order to create a new Pandas DataFrame, which now contains time series data. So, for example, rpm, speed in kilometers per hour, and the like. And these time series are then simply written into InfluxDB using this last part here and using in some of these classes we have. So I can’t go in depth in this session here, but that’s the gist of how it works. And you can find all of our API modules, of course, on GitHub in here.

Caitlin Croft: 00:53:25.194 With port of the car or jack is used to connect the CANedge2? Which version of CAN 11-bits?

Martin Falch: 00:53:34.521 So if the question is how you would connect it to your car - was that the question or?

Caitlin Croft: 00:53:39.425 I think so, yes. Which port of the car would you use as well?

Martin Falch: 00:53:44.364 Yeah. It depends, a bit. I mean, let’s say you are a family dad who wants to monitor how his kids are driving around in the car. Then the typical way you would connect would be through the OBD2 board. So if you search near your steering wheel, just left to it typically a bit down, you will find an OBD2 connector that matches this adapter here. And you can plug this into your car through this connector and then the other end into the CANedge2 so that essentially you connected to this first port here. And then, you use a configuration where you request data from the car, and that’s how you get things like vehicle speed, fuel level, fuel consumption, and the like. If you are an engineer at, let’s say, Volkswagen, as in one of our case studies, you might want to get some data through the OBD2 connector. But you might also want to get some data through other means, like from some of the raw CAN bus data, because as an OEM, you typically know how to interpret both the OBD2 data but also the raw proprietary CAN bus data. And in such a case, you might want to use one of the channels of the CANedge2 to log through this OBD2 cable and the other channel you might use with a CANCrocodile, as we call them, to log data from the raw CAN bus wiring harness. So without going too much in depth on this, you can see on our products page a lot of the typical adapter cables you use, whether it’s for recording data from cars like this one, from trucks like this one, from industrial and boats like this one, or from more generic applications like these two cables here.

Caitlin Croft: 00:55:24.603 For customers in mining or other verticals where some additional decoding is necessary, is there something you can do to help with, or is there another path you would suggest? We have CAT and Sandvik trucks, [inaudible], and drills.

Martin Falch: 00:55:39.952 Yes. So this is a common question that I get a lot. And so one of the things to understand when it comes to CAN bus data is that recording the raw CAN bus is simple enough. You can do that from pretty much any of the applications we went through. Decoding that data requires that you know how to interpret the data. In some cases, you know how to interpret it. For example, the case of a car, any regular car, typically, you can decode OBD2 data because it is standardized and it’s public how you do it. The same goes for many heavy-duty vehicles if you use a J1939, a DBC file. Then in my experience, probably across 70% of heavy-duty vehicles, you can expect probably 60 to 80 percent of the data from them using the standard DBC file, whereas the rest of it will be proprietary. When it comes to mining vehicles, like Caterpillar vehicles, then you may face that some of the data will be possible to decode using a J1939 DBC file, whereas some of it will be proprietary. And to my knowledge, at least, there is no public database where you can find out whether a specific vehicle is supported or not. So what we recommend is that users, they essentially try out recording data from the different vehicles for a use case with a single device. And then, we offer that they can send the data to us to check whether it can be decoded using the J1939 DBC file. If that’s not possible, then pretty much the only other way is to reverse-engineer the data, which is hard or to get a deal with the OEM, which is hard if it’s Caterpillar, but feasible in some other cases.

Caitlin Croft: 00:57:30.455 Let’s see, someone’s saying, “We have 1,000 trucks to monitor, is it possible to use your hardware plus InfluxDB and Grafana?”

Martin Falch: 00:57:40.053 Yeah. I mean, it is possible. And you can use the CANedge2 as we discussed in here, and you can always contact us to learn what type of combination of, let’s say, cables and different accessories would be relevant. What I would say is that when you look at this, you should always look at whether this is the right solution and what you’re trying to do. And I would say for these type of use cases, when it comes to bigger volumes, what we recommend is to, step one, contact us to understand whether it’s a good fit, and we try to answer very, very fast on this. And step two is, if you think that it could be a good fit, then we recommend to try it out and do a proof of concept with one unit very early on. So I’ve seen a lot of cases where these bigger volume projects, they stall because no proof of concept is done, and you very late in the process figure out whether it’s feasible or not at all. So we try to encourage doing that very early on.

Caitlin Croft: 00:58:42.331 Is there an existing GPS tracker with RS-485 or RS-232 interfaces? Can we connect the CANedge? If yes, any specific manufacturer supported?

Martin Falch: 00:58:57.294 There are many GPS modules and also GPS to CAN bus modules on the market, so the one that we are coming up with is the one we recommend because it is designed using the same principles as our CANedge. And it is possible to power this module through a five-volt power supply, which means that you can use it on the second port of the CANedge in a very similar setup as shown here, for example. But you don’t have to use this one. We have customers that use other GPS modules. If you have a specific module in mind that you are considering, we always recommend to just send us a link for it, and we can verify whether it is compatible or not. But as a general rule of thumb, if it produces GPS data via CAN bus, then our device will be compatible. So as long as it’s producing CAN bus data, the CANedge2 is a CAN bus data logger, and it will record it. And so if it is CAN bus interfaces, then it’s fine.

Caitlin Croft: 00:59:57.575 Why did you opt for InfluxDB Cloud versus some of the other versions?

Martin Falch: 01:00:04.664 Yeah. I originally looked at some of the options. For example, in Grafana, they have some built-in data sources such as Graphite and other databases that could also be used, and you could use other time series databases. But take the example of the built-in Graphite in Grafana. One of the limitations here was that once the data would be more than a week old, it would basically be downsampled. So the max resolution would be 1 second, but after one week, it will be 10-second resolution. And that’s not always very useful. So if you take, for example, the data in here, data like our acceleration data from this module here, you might want to go down to very, very detailed levels of resolution, intra-second resolution in order to monitor how this data is moving in order to, for example, look at road bumps or other things. If the data is aggregated to 10-second resolution, then for many CAN bus use cases, it’s useless. So that was one of the things that we look for with InfluxDB where the default retention is 30 days, and the data is not downsampled within that period, which was important for us.

Caitlin Croft: 01:01:20.638 And why were you specifically interested in the cloud version of InfluxDB?

Martin Falch: 01:01:27.222 Yeah. The reason for that was that - I mean, if you’re familiar with Docker and other tools like that, you might find self-hosting is very easy. Why not just do that? If you’ve never used Docker before, then self-hosting a database, self-hosting a dashboard tool, then it becomes a bit much for many people. The cool thing about the InfluxDB Cloud is that we can provide a step-by-step guide that always works, regardless of whether a user is in one country or the other. They can go in, and they can add a new data source. And the data they need to input is generic, it’s standard, and it’s standard the way you would find it within the InfluxDB Cloud interface. And that is what allows us to provide a step-by-step guide where we can provide a small video like this one so that every user will have the same experience. Without this, then we would basically be bogged down in support on setting up Docker and other stuff, and it would not be possible for us to do this concept. And so I think, Caitlin, you also mentioned “time to awesome,” which is important for InfluxDB, and that is the key for us. We need to make it possible for users to see a proof of concept of their own custom dashboard within half an hour. If they use that half an hour to fail setting up Docker, then it’s not going to be a radical proof of concept that they can show their manager, and therefore the project may fail. But if they can get past this first part, then they can always shift to a self-hosted solution down the road if need be, or they might decide to use the paid route for whatever is most suitable for their project. So that was the key motivation.

Caitlin Croft: 01:03:09.327 Fantastic. So I know we have gone completely over. I’m just going to have a couple more - go through a couple of more questions. And then, if anyone has more questions for Martin, please feel free to email me, and I’m happy to connect you with him. And there were a few people that asked if this was being recorded. Yes, this session has been recorded, and the recording and the slides will be made available later today. Okay. So a couple more questions. Did you implement Telegraf as a data collector with Python, or do you have Python pushing data directly to InfluxDB through POST?

Martin Falch: 01:03:46.351 The way that the Python script works is that it takes the data from the log files, extracts the physical value data, and then it pushes that data into InfluxDB. So if you want to get a bit more of an idea of how it works in detail, what I will recommend is to check out the dashboard-writer script or the repository here. And specifically, you can look at the utils_db files in here, where you can see the logic of how we set up the Influx part of this. And specifically, there is this section regarding how we write signals. And essentially, what happens here is just that we provide a Pandas DataFrame with a time series, and then we write that into a InfluxDB. So you can see the process of how that works here. But no, we don’t use Telegraf for this.

Caitlin Croft: 01:04:41.441 Ah. Let’s see. Are the log files created by the CANedge devices with actual time and date rather than system time?

Martin Falch: 01:04:49.813 Yeah. The way it works is the CANedge2 has a small, real-time clock inside and a battery backup that allows it to know what the date and time is at any point in time with a 0.05-millisecond accuracy. That date and time information is then added for each and every single CAN frame in the raw data, and it is then carried through to the decoded data. So you will know that the vehicle’s speed was 55.5 kilometers per hour at March 14th down to the microsecond level. So that is something that the CANedge2 timestamps as part of the device itself, if that answers the question.

Caitlin Croft: 01:05:35.210 Perfect. Thank you, Martin. Thank you, everyone, for attending today’s webinar. I apologize that we weren’t able to get through everyone’s questions. I know there were a lot. So, once again, if you have any burning questions that you would still like Martin to answer, please feel free to email me, and I’m happy to connect you with him. Once again, thank you, everyone, for joining today’s webinar. Really appreciate it. And this session, once again, has been recorded and will be available for replay later today.

Martin Falch: 01:06:08.525 Thanks.

Caitlin Croft: 01:06:09.443 Thank you.

[/et_pb_toggle]