How to Use Grafana with InfluxDB
In this webinar, Gunnar Aasen from InfluxData and Carl Bergqyist will show you how to use Grafana with InfluxDB, and how to make the most of your InfluxDB/Grafana solution by sharing key tips, tricks, and insights. InfluxDB is a Time Series Database while Grafana is a tool for visualizing time series data. This webinar will explain how they work together so that you can get the most out of your time series data.
Watch the Webinar
Watch the webinar on “How to use Grafana with InfluxDB” by filling out the form and clicking on the download button on the right. This will open the recording.
[et_pb_toggle _builder_version="3.17.6" title="Transcript" title_font_size="26" border_width_all="0px" border_width_bottom="1px" module_class="transcript-toggle" closed_toggle_background_color="rgba(255,255,255,0)"]
Here is an unedited transcript of the webinar “How to use Grafana with InfluxDB.” This is provided for those who prefer to read than watch the webinar. Please note that the transcript is raw. We apologize for any transcribing errors.
Speakers: - Chris Churilo: Director Product Marketing, InfluxData - Gunnar Aasen: Support Engineer, InfluxData - Carl Bergqyist: Software Engineer, Grafana Labs
Gunnar Aasen 00:01.268 Hello, everyone, and thank you for joining us today. And we’re just going to go through and basically just explain InfluxDB and Grafana and how you can use them together to sort of get the most out of your time series data, and also visualize the things that you want and the way that you want using InfluxDB. So, first I’m just going to go through a quick basic setup-of how to set up Grafana and InfluxDB, sort of like some basic configuration options. And then Carl will sort of explain how to set up graphs in Grafana and then sort of a lot of the basic tags and query builders and things to make your graphs look very pretty. And then we’ll come back around to me, and I will then go through a few tips on how to make your solution more efficient when using Grafana and InfluxDB together. So just to kick it off if you’re not aware-InfluxDB is a Time Series Database. It’s a database specifically used for storing time series data, and Grafana is a visualization tool for time series data. Now, Grafana supports a bunch of different backends that store time series data, and one of those backends is InfluxDB. And Grafana has a pretty tight integration with InfluxDB, and that’s definitely one of the most popular visualization tools that we recommend using with InfluxDB. So just to sort of give people, who haven’t maybe set up InfluxDB and Grafana, an idea of what we’re talking about when we talk about using InfluxDB and Grafana together: the basic setup for InfluxDB and Grafana is to have InfluxDB and Grafana connected together, and InfluxDB has an API, and typically that defaults to the port 8086, and Grafana, their API is on port 3000. And Grafana will call the InfluxDB API whenever it wants to query data. So, basically, the way that when you set up a time series visualization stack, you typically have a collection agent collecting your metrics. In this case, for InfluxDB we typically recommend using Telegraf. It’s one of the tools that integrates very well with InfluxDB.
Gunnar Aasen 02:53.830 So if you put your collection agent or Telegraf on the instance that you would like to collect on, you can generate metrics and send those to InfluxDB. Now, InfluxDB is just a process that runs on a server and that can-that process can also run on the same box that Grafana runs on. And Grafana has a very lightweight server-side application, and then most of Grafana runs in the browser. So what you’ll find generally if you’re trying to set up InfluxDB and Grafana together, typically we find the easiest setup is just to set up, InfluxDB and Grafana on the same instance. InfluxDB will certainly be the more memory-intensive and CPU-intensive application out of the two, but that’s simply because a lot of Grafana’s work happens in the browser. So typically, we’ll recommend for Grafana that you use the latest browsers. And for InfluxDB we recommend, depending on your load, setting up on an instance cloud provider. So I’m just going to go through some of the configuration that we recommend for InfluxDB, and I’m going to switch the window I am sharing real quick. Okay. So I’m just going to go through very briefly some of the important configurations to keep in mind when you’re setting up InfluxDB and Grafana. First, I’m going to go through the InfluxDB configuration, and just point out some highlights that are useful to keep in mind. Also, I want to mention that both InfluxDB and Grafana can be downloaded from their respective sites. They’re very easy to install. There are a number of packaging solutions. So I highly recommend going into Docs, and the download page is on each one of the respective sites to get that information. So if you’re installing InfluxDB and Grafana on, in this case, this is a [inaudible] instance.
Gunnar Aasen 05:31.579 This is typically what you’ll find once you install the DB package in the scd/influxdb/influxdb.com. And a lot of the defaults set up for InfluxDB is pretty good. Some things you’ll want to note is that you can enable things like query log-query log which will log all queries when they’re executed or when they get to the InfluxDB API. It’s very useful for debugging Grafana issues. Going down further in the configuration, one of the most important things to keep in mind when setting up InfluxDB with Grafana is to set up this coordinator section of the config-specifically many of those useful options are being able to set the maximum number of concurrent queries. So if you have issues with a bunch of different Grafana users all hitting InfluxDB with different open browsers and sending a bunch of queries through and you’re having trouble with that, I recommend setting the max concurrent queries. You can also set query time-outs. You can also, very useful, set queries to be logged if they take longer than a certain amount of time. And then these last settings around max-select point, and max-select series, and max-select buckets, are also very useful in limiting the amount of results that can be returned. So if someone executes a particularly expansive query, this will allow that query to be killed before it potentially takes down the InfluxDB server or just takes a long time and ends up using a lot of resources and slows down InfluxDB queries for everyone else.
Gunnar Aasen 07:36.794 Going further down, InfluxDB has monitoring. So that’s very useful. You can check out how to set up InfluxDB monitoring of itself using the internal database. And I will show a little bit of that later. There’s also the http section. So it’s the actual section where you would set where InfluxDB has its API Listen to, so by default that is port 8086. You can turn on auth as well. We highly recommend turning on authentication to be able to set up users and prevent unauthorized access to your InfluxDB. And it can also enable logging a http requests as well. This is also where you would set up https, and I have not set it up on this server yet. So moving on to Grafana’s configuration. There are a bunch of good things in Grafana’s configuration file, and I would highly recommend you just check it out and read through everything that’s there because there are a ton of great ways to configure Grafana to make it more usable, but I’m just going to go through another-again, some quick configuration options to keep in mind when we’re setting up Grafana. Probably, the most useful one here is in the first server section. You can set up http by default. Grafana Listens are on port 3,000. Obviously, you can set this to 4043 or port 80 if you want a more traditional browser experience.
Gunnar Aasen 09:32.596 Another useful feature or useful configuration option is router logging, so if you’re having trouble debugging, whether requests are actually getting to Grafana or wandering, a bunch of different things around that, I recommend it. You can also enable the user to make your browser pages load faster. Moving on, there are a number of great sections that you should check out later, but the highlight here is the security section. Every Grafana instance when it comes-when it first comes up will have a default user and default password, default admin user, and default password. And that default username and the password for that user defaults to admin and admin. So I highly recommend if you are setting up Grafana on anything where anyone is able to get into for instances to set your own custom username or at least password to prevent people from logging in and have access. Moving on down, the other thing I would keep in mind if you were setting up Grafana for the first time is to-is that by default, Grafana will allow users to sign up and register and they’ll also allow non-admin users to create organizations. So I recommend setting as “false” these files so that you can prevent people from setting up users on organizations maybe if you didn’t want them to. Also, anonymous access to this is disabled by default, but an important point is you can enable anonymous access, which is very useful. Let’s say you have a public dashboard that you’d like to promote. Again, there are a bunch of authentication options you can set up, Google Auth and a bunch of other OAuth providers know all about that. Another important thing is that since most interaction with Grafana is-or almost all interaction with Grafana-is through the browser.
Gunnar Aasen 12:01.260 If you do a password reset or anything like that, that requires a user to receive an email, you’ll want to set up this SMTP section. Grafana also has a number of log levels, so if you’re trying to debug, definitely bump up the log level to debug. And finally, Grafana will expose metrics about itself using a Prometheus publishing format. And so you can either scrape that directly or-sorry, Telegraf has a Prometheus input built-in so you can direct it towards that and receive or collect internal metrics about Grafana and put them into InfluxDB and then graph them again in Grafana. And that about wraps it up for the configuration explanation. So I’m just going to go through a couple of quick common pitfalls around setting up InfluxDB and Grafana. And basically, the first common pitfall is deciding whether you want to set it up on the same instance as InfluxDB or not. I would pretty much always recommend just going with setting up Grafana and InfluxDB on the same instance. Typically, it ends up being operationally much easier to operate that way. However, if you find you have a very large installation of InfluxDB, or a number of Grafana users, or a certain security, or deployment profile within your org, then setting up InfluxDB and Grafana on separate servers is also perfectly acceptable. And then, again, like I mentioned when going through configuration, make sure you turn on authentication in both Grafana and InfluxDB, and that will save you a lot of headaches down the road. And that’s it for my section, so I’m going to pass the ball over to Carl to go over actually using the Grafana UI to set up graphs and using the InfluxDB query builder.
[silence]
Carl Bergqyist 14:50.633 Once you’re in config, let’s get a photo up. And once you’re logged in for the first time-oops, let’s switch over-you get this nice Home Dashboard telling you to add a data source. So either you can click this button, or you can go to the top left navigation and Data Sources. And then you add one, and you select the InfluxDB Type and give the sweet name. You might want to make it default data source for this Grafana instance. You pick InfluxDB URL, in this case, my machine. And then you have to enter the database, user, and credentials, which are super secret in my case. And then you add it. And once you add it, you get a notification regarding the success. So if we change it to some invalid setting, we get an error. So let’s go back Home. Next step is to add the dashboard. You can do that by this button, or you can click down here, or you can go up to the left menu. There’s a lot of ways to add a new dashboard. So dashboards are based off rows and panels and to add a panel to a row you just [inaudible] them and you can create new rows, and can also be just clicked on. And if you want to have multiple panels on a row, you just go to this, add panel, and you draw them out. To edit a panel, you click under title and edit. And to go back to dashboard, you click back to Dashboard. This is kind of frustrating when you have to click all the time, but if you hover over a panel and press E you go straight to edit mode. And if you press Escape, you go straight back to Dashboard. So this way you can navigate kind of quickly. While in edit mode, the first thing you see is the Metrics path, and here you specify your InfluxDB query. So here you can specify the InfluxDB policy that you want to use. In this case, I’m going to use the “default” one, and I’m going to choose “measurement”. And let’s take logging count. So that’s it. Now we have some metrics. And by default, we used the “value” as a field and “mean”, but you can use any aggregator that you would like. Maybe on some account. You can also select multiple fields, but in this case, in my test data, I only have value. But I can select the same field twice and just use different aggregators. So that’s how you select field. And if you want to remove them, you click on a field and remove. You can also add other kinds of transformation like moving_average, that’s kind of senseless on count, but let’s do “mean” again and then moving_average. We can see the difference. The other properties in the Metrics tab is how they’re grouped by. And by default GROUP BY, Grafana groups by this predefined interval value. And the interval value is calculated based on the screen and the panel width. So by default, the interval value is kind of senseful to have. But if you’re writing data at, let’s say 10-second interval, then it would not make sense to group by a lower value. So that’s when you can either specify 10 seconds, or you can specify down here any value below-value Grafana calculates cannot be lower than 10 seconds.
Carl Bergqyist 19:44.401 It can also specify how InfluxDB should handle null values. So if we specify null, it will not insert any null values. It can specify-you can allow nulls, or you can use the previous value. This can kind of modify the dashboard, if you use some strange-you get some strange GROUP BY’s. So in some cases, the values disappear. If you see that-yeah, like that-just make sure that your GROUP BY is not too small. You can also have select statements (WHERE statements) if you want to look at “hostname”, “server1”, you can filter all the values-time series-and you can also GROUP BY tags. So if you want to look at hostname differently, oh, that’s a lot of values. Let’s remove one. Now we’ve got one serial for each host for “datacenter”. We can also give the names a more beautiful name. Let’s say “moving avg”. And you can include tag value. So if you click here, you get all the Alias patterns that you can insert in the Alias part. Replacement that you can insert in the alias pattern. So if we do tag_hostname-source in this case, since we’re grouping by source-we get a more beautiful legend name. And you can also have multiple queries if you want them differently, instead of just multiple selects. So let’s remove that. In the Axes tab, you can specify units to make your graphs make more sense. So if you want to specify dollars, they will make much better sense. It can also specify mean values or “auto” if you want Grafana to calculate Min and Max. Yes. If you want to change how the legend is displayed, that’s what you do in this tab so you can render it as a table. You can render it to the right.
Carl Bergqyist 23:08.828 You can add current value to lost value in the time series. The total min, max, and average. Just remember, whenever you add min, max, or average, it will be the min, max, or average value from the time series returned from InfluxDB. So if the query uses min, average value, the min will be the minimum of the average value, not the minimum role value. That’s very important when you display this with the value’s name. You can also change how you want to display the graph; as lines, bars, points, and so on. There’s a lot of different settings you can play around with here. It’s kind of funny. And you can also do a series overwrite. So if I think the backend should not be aligned, I can disable that. But it should be a point. And a big radius. This way we can have different settings for a different series-which sometimes makes it much more obvious what you’re displaying. So when you feel ready with your Dashboard, you can click Save. And if the title is in Dashboard, we will ask you for a new title. So in this case, let’s use InfluxDB webinar. And save it. If you all have a name, you’ll just update that. So whenever you’re working with your Dashboard, please remember to save frequently. Otherwise, you might lose your changes. Now these Dashboards might not be super interesting and kind of basic. So let me show you another dashboard that I prepared as a good Swedish FIM. So in this dashboard, we have a lot of more powerful tooling. It allows us to update the query live with templates. So this will update all the queries to use these dynamic properties. In this case, we used the one panel. But if we had 10, all 10 would be updated. We have also inserted annotations, which is a way to draw line in the time where you might want to add stuff like deployments or alerts or stuff like that, which might give you information about why certain numbers change. So you can have a link to your deploy information or things like that. A new feature in Grafana 4 is also Adult Printers which makes it possible to have-to apply filters dynamically.
[silence]
Carl Bergqyist 26:45.650 Knowing upfront what filters you have because when you use specified data center, you have to add it to the metric panel. But if you use Ad hoc it will be added to all queries on this dashboard going to go in the factory. This is really powerful when you want to dig into certain errors or why circumvent servers are behaving in unexpected ways or something like that. So that’s all I have from Grafana. So I guess we could start with our Q and A unless Gunnar has something to add.
Gunnar Aasen 27:32.865 Yes. I’m actually going to go through and do a quick section on just how to think about making your InfluxDB and Grafana solution work better and more performant. So let me get behind the screen back up.
[silence]
Gunnar Aasen 28:07.590 So if you use InfluxDB-by the way, thank you, Carl, that’s a great introduction. So I just want to go through a few ways to improve the performance of InfluxDB and Grafana. Generally, you will find that InfluxDB and Grafana perform quite well together, but you can definitely hit some edge cases depending on if your installation of InfluxDB is fairly massive. So you’re collecting a lot of data, or you’re alternatively querying a lot of data and put some stress on InfluxDB. Typically, since InfluxDB is doing a lot of the server-side work, you will find that, or we find that InfluxDB is usually the first piece of the InfluxDB-Grafana chain to show issues or experience issues. So this is going to be a little bit about how to identify what might be going wrong, and then also some tips on how to avoid those things that go wrong. The first thing to explain is generally just Grafana graphs. Each Grafana graph is actually a query to InfluxDB. So the more graphs you have just equals more load on the InfluxDB server. So if you have a dashboard of 30 graphs, that’s 30 queries that you’re sending to InfluxDB and 30 queries that then need to get the results collated by InfluxDB, and then sent back through Grafana. Oh, am I-?
Chris Churilo 29:58.702 Just want to say that again?
Gunnar Aasen 29:59.077 Oh, sorry. Okay. So I guess my audio was down. So, I guess, going back over that, so every Grafana graph sends a query and sometimes multiple queries to InfluxDB on the backend. So if you think about it, every Grafana graph in your Dashboard is another query to InfluxDB. So as you increase the number of graphs in your Grafana dashboards, or at least the dashboards that are displayed, that will be an increased number of queries that end up going to InfluxDB. So if you have a dashboard of 30 graphs in Grafana, that’s equal to at least 30 queries that go to InfluxDB. InfluxDB has to collate the results, send them back to Grafana, and then the browser has to take those results and then display them on a graph. So if you have a lot of queries or a lot of graphs and you’re sending a lot of queries, we highly recommend, first, think about what you’re graphing and whether all the graphs that you have displayed are actually useful to you. The other thing you can do is-there is a very useful option, and this is just a InfluxDB instance monitoring a busy InfluxDB instance. You can set your rows. If you come over here and do the row options, you can set your rows to be collapsible. And having collapsible rows means that when you collapse those rows, just up here, it will not show that graph and will not render that graph, and therefore won’t produce the queries to InfluxDB. So if you do happen to have, say a dashboard that has a lot of useful graphs, but maybe they’re not all useful at once, a great way to reduce or speed up your Grafana experience with InfluxDB is to use collapsible rows.
Gunnar Aasen 32:10.556 There are also a bunch of different graph types and other, sort of, options you can add to graphs. So things like single stats, is up here in these things, these boxes, so you can have just sort of one result returned. So say for instance, in this case, we care only about the number of queries executed, we can just return the latest number of queries that was executed and not a graph over time. That can be a significant savings in processing power that InfluxDB needs handle. And then, you can also do things like templating and annotations. I don’t have templating set up here. But yeah, the templating, instead of having a graph per-say, if you have 10 instances-instead of having a graph per instance you can set up a template and have a just one graph showing for any instance at one time and then be able to easily switch to show other instances. Another thing to note is that a large number of series, in InfluxDB, so say you have a lot of tags and tag values, will eventually, once you get to a certain number, which typically, tends to be over 100,000 series, will slow down the query builder of drop downs in the Grafana, query editor. And this can sometimes be an annoying part of working with InfluxDB query builder when you have over a million series for instance. So at the moment, we don’t have much of a solution to that but it’s been an open issue for a long time, and we are planning to address it at some point soon. Some more performance tips. Again, every graph in Grafana equates to basically a query InfluxDB, and one thing that’s important to understand about InfluxDB is that it generally does not do, sort of, the query level caching. So, every time you send a query to InfluxDB, InfluxDB is going back to the data and collating all those results, and calculating say a min, or a max, or a percentile, and then returning those quantities. And every query in Grafana, that’s generated by Grafana graphs, typically has a GROUP BY option. And so, you’ll see a lot of that recalculation on each loading of the graph.
Gunnar Aasen 35:13.564 Generally, that’s not an issue. InfluxDB is pretty performant, so is Grafana. If you do a very expansive query to calculate and say, InfluxDB takes a while to respond and the query times out and you get an error in your Grafana graph, generally, we recommend waiting a little bit of time before you hit the refresh button. So don’t click it repeatedly because all you’re doing at that point is just sending InfluxDB more queries to query that, it already was having trouble responding to. I’d also recommend, when I’ve gone up the configurations, there are a bunch of logging options in both InfluxDB and Grafana. If Grafana’s displaying an error for a graph, I’d recommend going through and figuring out why it’s actually producing that error if it doesn’t show up after one, maybe two refreshes. Again, going along with the theme that InfluxDB doesn’t cache queries, so every time you send a query, it recalculates the results. Typically, if you’re zooming out, so for instance, if you’re zooming out or selecting larger time ranges, say 24 hours, it’ll take all that longer for InfluxDB to pull out all those results. If you say you’re going back seven days or a month, sometimes it takes a while for things to load. Just keep that in mind that the more data you accumulate, the more data that Grafana and InfluxDB have to go through to load. The way to fix this in InfluxDB is if you have a particular series where you have particularly high-fidelity data. Say, maybe you’re collecting data at a sub-one-second interval, and you’re displaying maybe, a day’s worth of that data, that can be a lot of data for InfluxDB and Grafana. Well, InfluxDB has to sort through to return just a query for say, an hourly average. One of the ways to fix that in InfluxDB is to use continuous query, which is just a periodic query that runs each specified query at a certain interval and then saves those results back into a new retention policy. And that can allow you, say, if you have one sub one second data and you want to display a day-long graphs at maybe a point every five minutes. It calculates that data upfront and store it back in InfluxDB instead of hitting the original high-fidelity data, you can query for a data created by the continuous query. This is a-We had a downsampling document on the InfluxDB docs that can explain how to use that along with retention policies.
Gunnar Aasen 38:37.508 That is it on my side. We will be taking questions in the WebEx Q8 channel. All right everyone, we have Sue Niel, asking a question on, “Should we create multiple measurements or measuring metrics in a single measurement? Would measuring metrics in a single measurement be better?” For InfluxDB specifically, this is different from if you’re coming from say, Graphite. In InfluxDB, we have a concept of both measurements and then tags. Tags are essentially the way that we recommend splitting up or discerning different time series within InfluxDB. We’d generally be-InfluxDB would recommend you have a measurement say, CPU. And then, we recommend using tags to say, discerning different CPU times series. You’d have maybe CPUCORE equal1 as one tag and then CPUCORE equal2 as another tag under the CPU measurement. Then, maybe you might also have HOST equals something as well. Senior also asks, “Could you please explain how to create global variables which can be used in any graph as arguments?” And I’m assuming or I think you’re asking about, whether you can set up global variables in Grafana that persists across Dashboards? And I do not think that is possible, although Carl maybe he’ll speak up and answer that.
Carl Bergqyist 40:35.135 So at the moment, that’s not possible. Template variables are always connected to one dashboard. We won’t add global template variables to our organization, but that’s not something that’s supported right now. It’s a good question.
Gunnar Aasen 40:58.529 And then Senior also asks, “Is Grafana always open source?” We’ve got a license. Grafana is an open source. So-
Carl Bergqyist 41:07.970 Grafana is a passage to an open source. It’s using SQLite as an internal database by default, for storing all dashboards, but you can use MySQL or Oscars as well engine. If you want to have multiple Grafana servers using the same the same dashboards.
Gunnar Aasen 41:34.154 Joel asks, “Any general advice on structuring data to better with Grafana?” So in InfluxDB, generally, as I was explaining, we recommend using sort of a top-level measurement and then separating out different time series under a measurement using tags. That’s certainly the recommended best practice, and that’s also what will work the best when using the Grafana InfluxDB query builder. And you can find more information on how to do a generally good structure, data structure, within InfluxDB on the InfluxDB DOCS. And what we recommend on the DOCS is generally aligned with, or pretty much aligned with what works well on Grafana as well.
Carl Bergqyist 42:26.893 Next question by Samalie, “Does Grafana use a caching layer?” It does not use a caching layer. I guess I’m not 100% sure if we can set up an annex or something, between Grafana and InfluxDB. I don’t think that’s the recommended solution. I will strongly advise you to minimize the amount of queries that you send, like Gunnar said, and don’t set refresh rate to five seconds if you have InfluxDB using a lot of resources. Increase those values to avoid unnecessary requests.
Gunnar Aasen 43:11.619 And the next question is how to fetch the underscore time field value in templating.
Carl Bergqyist 43:18.647 So I guess you’re referring to the GROUP BY TIME field. You can use that, if you want to do it for math calculations and so on, the dollar underline, underline interval is a reserved templater-build in Grafana 402, which is not in beta yet, but it will soon be. Otherwise, you can’t really change it. So if you want to modify the GROUP BY Time by using a template variable, you should create a new template variable called Interval something and we use that in the GROUP BY Statement. I’m not sure if that answers your question, but please ask again if something is not clear.
Gunnar Aasen 44:08.259 And the next question was, “What is the difference between Grafana and Kapacitor and which one is best?” I’m assuming you’re asking about Grafana versus Kapacitor. And I think they’re both sort of maybe different strategies. So if you’re using Grafana, you can use Kapacitor along with Grafana InfluxDB. It’s going to be a little bit more live work to use Kapacitor since the interacting with Kapacitor versus GLI. And so, Grafana alerts at that point is useful. Another feature I’d mentioned early about Grafana is that it supports multiple different data sources. So if you have data, and InfluxDB database but maybe also an open TSTB database or Prometheus, then Grafana alerts could also maybe give you some more flexibility since Kapacitor is only in InfluxDB. The alternative to that is Kapacitor is very tightly matched with InfluxDB, and they’re really only, or there’s a lot of development that basically they’re developed in parallel and made work very well together. So if you are using InfluxDB as your time series data store, then Kapacitor, of course, would be a pretty good choice.
Carl Bergqyist 45:41.379 Next question is by Sunil, “Can we integrate Grafana login with active director authentication?” And yes, that’s possible. Alert is supported by Grafana.
Gunnar Aasen 45:56.674 The next question from Sunil is, “Do we have update functionality available in an InfluxDB? Or InfluxDB can be used only for reading perspective?” I’m not 100% sure what you’re referring to by the update functionality. Yep. Sorry. We can move on to the next question then. Maybe you can send in a few with some additional clarification I can maybe answer that. Next question from Joel is, “For example,” I think he’s adding on to his previous question which is, “For example, tags versus having the values in the series name.” So we definitely recommend using tags, and I would recommend going through the InfluxDB docs. So when I say series, I actually mean InfluxDB parallax, that means both the measurement plus the whole tag set. So all the tags associated with the specific series. So when I say series, in InfluxDB land that also sort of encompasses the measurement and the tag. The next question from Novak is, “Hi, is it possible to connect Grafana to a JMX endpoint?” And I’ll let Carl answer this one.
Carl Bergqyist 47:29.486 No. Not that I’m aware. I think you want to collect data using Telegraf from the JMX endpoint?
Gunnar Aasen 47:40.331 Oh, I see. Okay. Yes, I would take a look at Telegraf, the collection agent that works very well with InfluxDB, has a plugin architecture, and I think there’s a way that you can-or one of the plugins allow you to collect JMX data but you’d have to do some research on the Telegraf docs to confirm how to do that exactly.
Carl Bergqyist 48:08.188 Brotheslov also asked about the Grafana and permit using integration about exposing metrics, I presume? The metrics sent by Grafana is actually not the Prometheus. It’s to Graphite. But you can use the Graphite converter to InfluxDB. The metrics set by Grafana is-Grafana does not-have made a way of sending metrics to InfluxDB yet. Eulos also asked about a way to set time ratio between Grafana and the link. Yes, that’s possible. If you, in Grafana, clicked the Share button then you can generate a link to that path port. And in that link, you would get the port value for four and two, so you can update those values in URL. And that would be one the dashboard below the Dashboard will be used.
Gunnar Aasen 49:15.822 Okay, and it looks like the last question from Sunil is, “What new features will be available in the upcoming releases in Grafana?” So Grafana has a change log that you can check as is InfluxDB. And I guess Carl, do you know Grafana posts roadmap?
Carl Bergqyist 49:35.783 Yes. So the major features we want to add in the next year I would say is Dashboard Folders. And permission on these folders. Organizations is a very strict way to organize different users and dashboards since they have no access to each other. So we want to add another layer, which will be Dashboard Folders. Let’s see. Something else we really want to add. We want to improve alerting. Grafana 4 was the first release for the alerting, and we still have a lot of things to add to that. We want to add plugins as well so we can have more support for alerting and various other things in the backend. We’re gonna keep moving more and more stuff to the backend because that enables us to do alerting much better.
Gunnar Aasen 50:42.386 Travis asks, “Can you restrict graphs within a dashboard based on these permissions?” You can currently restrict it using organizations. And so, every user within an organization will be able to see all the dashboards that are owned in that organization. And I do not think it’s possible to-I think what Carl was mentioning earlier is sort of what would look like to do the medium work or any other user permissions for a Dashboard.
Carl Bergqyist 51:15.816 Yeah.
Gunnar Aasen 51:17.619 Next question from Nathan is, “What values should I store in InfluxDB to represent errors if my normal data inflection values are numeric? E.g, zero to one. I’d like to be able to view the errors when they exist on a graph. Right now, storing error codes has negative numbers.” So, when you’re using Grafana, there’s a really useful feature called annotations. And you can actually store those errors and maybe a separate time series that just includes the points for those errors. And then you can set up the annotation and Grafana will display basically, a red line at each point where the annotation has a point and so you can then-it’s a nice way to show the ways, or the places, where exact errors actually occurred. That would be the way I would recommend setting it up. The other option is to maybe just have a separate series as well, say using tags, and tag all your errors as may be HB status equals 400, or something like that, and then set up two queries on the same graph to have one that displays everything and then the other one just displays the HB400 and make that HB400 display, at that time series display with a bright red or something like that. Next question is from Sunil. “When we click on a single stack can we pass values to another dashboard as argument?”
Carl Bergqyist 53:25.767 So-
Gunnar Aasen 53:26.802 Carl, yeah.
Carl Bergqyist 53:28.095 So that’s not possible by clicking the single stack. But every panel have a link setting. So if you go to the General tab, you can add drill down links from panels. And so, if you do that you get a small link icon next to the title. So that way you can link from one panel to other Dashboards. I hope that’s good enough.
Gunnar Aasen 53:58.147 Thanks, Carl. Chris just pointed out to me that there’s a number of questions being posted in the chat as well, so I’m going to go through those real quick. So Dave asks, “Are tags a bad idea for something like CI parent job number which would correlate a few measurements in the short term but would grow to a very large number of values over time?” So I guess at this point, I would recommend thinking about maybe how many CI jobs you’re actually going to be running. So if you’re only running maybe one CI job a day, or one CI an hour, that’s only going to add up to maybe a few thousand over time. And so, that case it’s perfectly fine. If maybe you’re running highly distributed Cooper90 system where you have a lot of sort of instance turn over or something that’s causing a lot of stuff to get spun up and shut down, then you might want to be a little bit more considerate about how many tag values you’re gonna be creating. In that case, the option are to maybe get a bigger box for InfluxDB to run on. Or you can store that as a field value which is not indexed and won’t contribute as much to memory use pretty much because it’ll be stored on disk instead of in memory. So definitely consider it that way and also-so depends on sort of what you’re monitoring in as well. So, in this case, CI, I’m guessing that you’d want to check each CI job individually. In that case, you’d probably want the CI to be a tag more than a field or anything else. And I also want to note that we’re going to be releasing very soon a preview of our new time series index which allows you to basically, theoretically store as many tags in your InfluxDB as you want, as many series, and InfluxDB as you have disk space. So it’ll cache or not cache, but it’ll save series to disk, so you don’t need the whole series loaded in at one time. And that case if you have say a CI tag, you’d assume that the most recent CI jobs are ones that people are probably going to be looking at, and so with the new time series index that will likely sort of be fully rolled out and InfluxDB version 1.3 which will come out, I think the target is May right now. Then you’ll be able to do that fine. So that’s in the future. You can think about it that way as well. We also got a question from Vera, which is “Does Grafana have any limitation concerning the number of metrics, displayed on one graph? And how does Grafana treat this situation with big numbers of metrics?” I don’t think there’s an actual explicit limitation, but it’s certainly limited by what your browser can handle. If you have a Dashboard with 30 graphs that creates a lot of query for InfluxDB, but then when you display a lot of data your browser has to pass through all that data and then render it. There’s sort of like a practical on the number of metrics and that would depend sort of what client you’re using. Where you’re writing your browser. So I don’t know, Carl you want to add any additional flavor to that?
Carl Bergqyist 57:58.279 No, not really. It really depends on your browser. There’s no limitation within Grafana. But I say that if you run more than 30 in graphs, you’re probably going to have a hard time to get any value from it. But maybe you could, but it’s mainly your browser that’s the limitation.
Gunnar Aasen 58:22.999 Yeah, and again also, if you’re thinking about how many metrics you want to display at one time you might also want to think about what that data is actually showing you and what you’re trying to accomplish by reading all that data. I guess maybe if you had a massive 60-inch TV screen or something like that it might make sense to display some stuff. But at the end of the day there’s only going to be so much you can look at at a single time and so again using Grafana’s collapsible rows, for instance, would be a good way to maybe limit the amount of data you’re actually displaying at any given time. We also have a question from Travis which is, “Can you access, edit CSS, and edit the styles used in the Grafana Dashboard?” I know there are certainly panels where you can use custom HTML. I’m not 100% sure on whether there’s a way to edit CSS, aside from maybe recompiling Grafana with changes.
Carl Bergqyist 59:37.928 Yeah, you can use the text panel to edit an HTML, but if you want to change the behavior of Grafana, you have to recompile it yourself. And there’s a guide in the key there for building Grafana from scratch. So if you follow that you should be up and running within minutes.
Chris Churilo 59:58.826 So thanks very much to our speakers today. I think they did a terrific job and I hope you guys enjoyed yourselves and we hope to see you guys again soon, bye-bye.
[/et_pb_toggle]
