Weather Station Monitoring with Raspberry PI & InfluxDB
In this session, Tim Raymond shares how he uses InfluxDB to gather metrics and events from his Raspberry Pi weather monitoring system.
Watch the Webinar
Watch the webinar on Weather Station monitoring with Raspberry PI & InfluxDB by clicking on the download button on the right. This will open the recording.
[et_pb_toggle _builder_version="3.17.6" title="Transcript" title_font_size="26" border_width_all="0px" border_width_bottom="1px" module_class="transcript-toggle" closed_toggle_background_color="rgba(255,255,255,0)"]
Here is an unedited transcript of the webinar “Weather Station monitoring with Raspberry PI & InfluxDB.” This is provided for those who prefer to read than watch the webinar. Please note that the transcript is raw. We apologize for any transcribing errors.
Speakers: Chris Churilo: Director Product Marketing, InfluxData Jack Zampolin: Evangelist, InfluxData Tim Raymond: Developer, InfluxData
Tim Raymond 00:01.084 Okay. Can everybody see everything all right?
Chris Churilo 00:07.483 It looks great. Thank you.
Tim Raymond 00:09.532 All righty. So I’m going to talk to you today a little bit about a little side project that I’ve been working on for the past six months, and that is building a weather station using the TICK stack. So I’ve been interested in the weather for a good number of years. And maybe you’re wondering why I wanted to do this. So I want to say about 10 years ago, I purchased a Davis Weather Instruments, the same weather instruments that’s preferred by a lot of meteorologists. And I live in a part of the Mount Washington Valley in New Hampshire which is known for a lot of kind of interesting microclimates. And it’s home to the Mount Washington weather observatory, which has a very long history of weather and it’s actually-they’ve recorded the highest wind speed on earth. It’s since been broken, but they held that record for a very long time. So it’s a very interesting area kind of from a climate perspective. And I also wanted to contribute my data to things like the Citizen Weather Observer Program. And this is a program that’s sponsored by the NLAA and it allows you to actually contribute your data to forecasts that are made by meteorologists all over the country and presumably all over the world.
Tim Raymond 01:49.893 And also it’s fun. It’s fun to work with time series data. And this is kind of something that’s personal to me and I wanted to play with the TICK stack with some fun data that I actually have. So this is the actual weather station. It has sensors for measuring barometric pressure, the outdoor temperature. There’s a rain collector on the top there for measuring precipitation. And it also has an anemometer for measuring wind speed. And then also it can measure the wind direction. And there’s a lot of good tools within the-it has a companion console so you can actually calibrate all of these instruments pretty accurately. And there’s some good guidelines that come with it on how to do that. So the data from the console is actually shipped wirelessly. So you don’t have to worry about running wires and drilling holes through your roof.
Tim Raymond 03:01.784 So the weather console looks something like this. And it’s great. You can view all of the data here. It even has a little rudimentary graph here, and you can view highs and lows. And there’s a simple alarm system for-like when certain temperatures are met. But I wanted a little bit more out of this. There’s not really anything actionable that I can take out of this when I’m not at the house. I really wanted to be able to see what the temperature was when I’m hundreds of miles away. And maybe the indoor temperature was getting too low. Maybe there’s a problem with the furnace, and we need to call some people to actually go and fix things, so. That was kind of the goal of this project. So you’ll notice down here there is a little USB connector. And so these consoles have a data collector which actually integrates with Davis’s software which is PC-only. Since it’s PC-only, you need a PC which is going to consume a lot of power. I wasn’t really interested in having that run all the time. So I wanted instead to use a Raspberry Pi. So what I did was I found a great piece of software called WeeWX or wee W-X, not exactly sure on the pronunciation there, but we’ll go with WeeWX. So it’s actually more of a framework for connecting to various weather instruments and then it has support for a lot of backends. So it’s all open-source. It’s written in Python, so it should be really easy to add your own integrations.
Tim Raymond 05:06.326 And importantly for me, the Davis Vantage Pro2 is one of the supported instruments. And it also has a plugin called WeeWX Influx which conveniently converts all of the data into the InfluxDB 1 protocol and writes it to InfluxDB. So it was a pretty easy setup. You basically install it. You add some configuration for where your InfluxDB host is, and with very little fuss, I had my weather data being written into InfluxDB. And also, since this is its own computer here, I wanted to monitor it and make sure that it’s continuing to ship my weather data. So I also installed Telegraf on it and set it up to collect CQ metrics and disk usage metrics. Because another problem with Raspberry Pi is, if you have a lot of churn on their SD card, you can wear it out. They’re not meant for that. So I wanted to watch that. So the InfluxDB instance that I have is just being hosted on a little T2 micro. It’s just a simple single-node instance and-so on there I also have-on the same instance I have Kapacitor running and Grafana and I don’t yet have Chronograf. I will soon. I actually have that set up locally just to experiment with and I will get into the reasons why I haven’t installed that just yet. But the main weather graphs that I’ve set up are all on Grafana at the moment.
Tim Raymond 07:04.952 And this here is the overall diagram of the way this is set up, so again, the roof-mounted weather instruments send data wirelessly to the Davis Weather Console. That is collected by the data logger and sent over USB which is just a-actually a serial protocol. And then it’s collected by WeeWx on the Raspberry Pi and shipped to AWS using WeeWx-Influx. So the next steps that I really would like to go with this, and I haven’t gotten them yet, is I would like to use Kapacitor to actually summarize a lot of the data that I have and then submit that to the Citizen Weather Observer Program. I haven’t done this yet because it requires me to actually encode that data in kind of a very old format. It has history with amateur radio. There’s a whole online version of that and so the documentation is kind of hard to come by. But it’s something that I’d like to do-excuse me. It’s something that I’d like to do because it would allow me to basically submit this data and have a nice separation between where the data is actually being collected and where it’s being submitted, which would allow me to actually have potentially multiple sensors in different locations, something like that. So something I’m playing with.
Tim Raymond 09:02.134 Again, I’d like to also use Kapacitor for low-temperature warnings indoors. So again, I want to have some message in Slack or something when the house is freezing or [laughter] various calamity situations like that. It’s, again, low-priority at this point since we’re getting into spring here in New England and we just had an 80-degree day yesterday. I can put it off for a little while, but it’s still definitely on the horizon. And then finally, I’d like to have some improved forecasting capabilities. InfluxDB has a function called Holt Winters which allows you to do some numerical forecasting. And I found it hasn’t been too bad for outdoor temperature, but it’s not quite as precise as some of the more advanced weather models. And it would take some time to actually develop some better models. But I don’t know. It’s kind of an aspirational thing. So we’ll see. So let’s kick it over to the demo time here. I’ll actually show you what I’ve put together for graphs. So this is Grafana. I’ve kind of organized these into the current conditions outside.
Tim Raymond 10:44.615 So it’s a pretty windy day. We’ve got 18-mile-an-hour winds gusting to 23 miles per hour. And then no rain, but if there was rain, you’d see this creep up as kind of the-excuse me [laughter]. That’s showing the current rain rate. And then down here, we have the temperature and pressure. The lighter green line to the left here is the historical temperature for over the past few hours. And then the darker green line is using Holt Winters to form a prediction from the past three days or so. So you can see there’s a little bit of a disparity here. It’s a little bit optimistic that we’re going to have another 80-degree day because we had one yesterday. And there’s a little bit of instability there because when it refreshes, it also re-runs the prediction. So I don’t know. Maybe this is the most appropriate prediction method for this kind of data. I might be able to apply it maybe to pressure. I haven’t tried that yet. So this is obviously the barometric pressure. And if we go down here, we can see this is my wind speed graphs which is showing both the current and the peak gust. And then also the direction here is- this is in units of degrees. Grafana doesn’t have a way for me to produce a radar plot, which is almost what I’ve-or a polar plot, I guess, would be more appropriate for this. But I don’t know. Maybe there’s some plugin that I haven’t found for it yet. And yeah.
Tim Raymond 12:44.525 I have graphs for rainfall, which there hasn’t been any in the past 24 hours. I’ll show you. Once I zoom out a little bit, you’ll be able to see some. And we also have the humidity in the upper viewpoint. So yeah. Let me zoom out a little bit. I can go out to a week here. And this is one of the great parts about doing this with Influx, is that I can just easily grab a week’s worth of data. It took no time at all. So you can see that I can immediately see things like the peak gust for the week has been 35 miles an hour. And also the highest temperature which we got for April was 84 degrees. It’s crazy. And then you can also see here that I’ve used Grafana to actually sum up the rainfall that we’ve had for the week. So I can tell you in a glance that we’ve had a quarter inch of rain, roughly speaking. And we can even go out further. We can say like, “Well, what has been the highest wind speed that we’ve recorded this year?” And we can see that we actually had a 60-mile-an-hour gust. And so this is some of the crazy weather that we see up in the valley. So if I can actually zoom in on that particular time, you can see it was actually beginning of March. And it corresponded with a little bit of rain as well.
Tim Raymond 14:31.670 So there are some interesting things you can pull out of this. With the temperature, you can sometimes-you can see fronts passing through, which that might be-no, it’s not quite as dramatic. You’ll see a very large drop in temperature in a short period of time if it’s a cold front. So yeah. Interesting stuff. And I also have the host metrics from Telegraf being shipped here as well. So this is the current conditions for the Raspberry Pi. And I’m looking at memory usage and CPU usage, and then also the disk that’s been used which actually looks like it’s-I need to work on it [laughter] because that is a lot. But so otherwise, it would have been pretty normal, but it looks like it’s still chugging away, so. But I would like to get Chronograf running on my AWS instance. I have it running locally connected to this cluster. And the reason that I want to do this is because to build this host metrics page, it took-I don’t know, it took me probably a good 45 minutes, an hour just to work to really tweak this to be perfect. But one of the benefits of Chronograf which would-our new version wasn’t available at the time that I created this back last August. So one of the things that we do is we have a host list here where we automatically detect what Telegraf plugins are running for each of your hosts.
Tim Raymond 16:34.236 So I can see at a glance that my CPU is 32% and then load, and then it’s up and running. And if I actually just click on that, I didn’t have to do anything to build any of these graphs. So with literally no work, I can see what the CPU usage is. I can see that this agrees that the disk use is 100%, and I will be looking at that probably after I get off this call. And yeah, I can immediately get an overview into the current health of my little Raspberry Pi. And I can also zoom out to the past seven days. And I can see that the CPU usage has been steadily increasing, which is kind of interesting. And maybe I’ll find something about that [laughter]. And I can adjust the refresh rate to be even quicker if I’d like. So I’ve waited until now because we recently added dashboarding features to Chronograf. So I’ve added a weather dashboard here which reflects a lot of what I’ve done with my existing weather graphs. So you can see here that I have a wind speed graph, which is very similar to the one that I had before. The purple line here is the wind speed in miles per hour and then this is the peak gust. And then also down here, I have the outdoor temperature and the pressure. And it’s very easy for me to add new cells to this. I can go in here, add a query. I can also show you a little bit of what the data that comes from WeeWX actually looks like. So I’m going to grab the weather database, and it writes to the record measurement-
Jack Zampolin 18:43.655 Hey Tim?
Tim Raymond 18:44.705 Yes?
Jack Zampolin 18:45.721 While you’re in the middle here, we have one question from Thomas Saltzer.
Tim Raymond 18:50.545 Oh, sure.
Jack Zampolin 18:51.208 Are you considering adding WeeWX support for dust monitoring sensors, for gathering PM25? So I think that’s different particulate-sized data. He says he’s using laser sensors to gather this data. And I’d like to know how polluted the air is in my area. I know you might not be working on WeeWX but maybe you ran across something that can help him out.
Tim Raymond 19:14.236 Yeah. Is the question actually posted? I might need to hear that again, but-
Jack Zampolin 19:20.670 It should be in the Q and A area down there at the bottom.
Tim Raymond 19:27.054 Okay. It looks like I can’t actually see that when I’m in presentation mode. I mean, that’s really interesting. I mean, WeeWX actually does have support for beyond standard commercial weather instruments. I know that in addition to the Vantage Pro2, there’s also more affordable weather instruments called the AcuRite. I’m blanking on the model number, but it retails for a little over $100. But you can also-it’s more of a framework for weather instruments in general. And I know that there have been a lot of kind of home brew setups that people have created using that. So I’m sure that it would be more than possible to add any kind of weather collecting device and certainly, measuring pollution is really interesting that way. I mean, I’m sure it would be a fabulous contribution. I mean, there’s a really active community. I’m pretty sure there’s a Google group which I don’t have available to me right now, but yeah, definitely worth a look, so.
Jack Zampolin 20:58.743 I’m going to go ahead and post that Google group in the chat here but-
Tim Raymond 21:03.000 Yeah, thanks, man.
Jack Zampolin 21:03.124 Yeah, thank you very much for answering that. And I believe that WeeWX is written in Python, so-
Tim Raymond 21:10.030 That’s correct, yeah.
Jack Zampolin 21:10.903 Yeah. So if you’re a Python user, maybe a good project for you.
Tim Raymond 21:16.903 Yeah. So continuing on, if I wanted to just add indoor humidities. And not quite as interesting, let’s just grab the outdoor humidity. Yeah, and so now we can see the kind of outdoor humidity. That’s not working. But we can see the graph and we can add it here. We can rename it to, let’s say outdoor humidity. And there. Let’s move this one back up, and we can resize things like so, and very easily I now have a dashboard with an additional graph on here. So it’s a little more flexible for dashboarding. We worked hard to really make it flexible to do that. There’s also a data explorer which makes it even easier to explore what kind of data that WeeWX is providing. So I can look at things like what is the-let’s see, a transmission-oh, so this shows me what the-no, it wasn’t that one. I’m looking for the console. Yeah. So if I wanted to find out what the battery voltage on the weather console is, I can just quickly look at that. I can also look at the dew point. So it just has a lot of features that make it easy to figure out what I can do with this data and then go from there to build dashboards like that that are saved. So that’s the future plans with that. So I think with that, yeah, does anybody else have any questions? Anything else I can show people?
Jack Zampolin 23:35.379 Thank you very much, Tim. That was awesome. Thank you for walking us through your home weather station there. And while we’ve got Tim on the line, I know you’d love to answer some questions, so any questions you guys have about this? Tim’s also one of our developers on Chronograf, so if you have any questions about our visualization side or other members of the TICK stack, I’m sure Tim would be happy to answer as well.
[silence]
Jack Zampolin 24:19.132 Stephen asks, “This project is really cool to see. I have a project using a few Raspberry Pis and Influx on VPS.”
Tim Raymond 24:31.180 Awesome.
[silence]
Jack Zampolin 24:48.537 Stephen, what are you monitoring with those Raspberry Pis, if you don’t mind me asking?
[silence]
Jack Zampolin 25:36.452 Okay, and then a couple more questions in here, but first one item from the chat. Yurgan asks, “Hey, there’s no sound. Is that normal?” No, it’s not. You did need to connect your audio. We will have a recording of this up on the website later. So if you missed the sound, you can go back and listen to it later. So a couple more questions, Charles asks, “Is there any way we can see his Grafana dashboard configurations?” Tim, do you have that in your repo for those?
Tim Raymond 26:06.653 I do not, actually. I can actually probably dub those out and you can possibly look at those. Do we have a way to share notes with this track?
Jack Zampolin 26:25.195 Yeah, there’s some text share down here. Do you have a project repo for this over on GitHub that’s public or is it kind of a private thing right now?
Tim Raymond 26:36.587 It’s kind of a private thing right now, but I mean, I can certainly drop the configs in a gist and share those.
Jack Zampolin 26:48.547 Okay. Charles, does that help you, hopefully? Gist would be great. Awesome. Thank you, Charles. Stephen, are you reducing the resolution of your data over time? I’m monitoring into your climate data humidity and temperature inside my own home and a friend’s.
Tim Raymond 27:12.612 Yeah. So it took me a little bit of time to tweak the queries, so that way I could zoom out to up to a year, particularly with things like peak wind gusts. Let me just pull that up again so I can see what specifically I did. Yeah, so it’s pretty common at least for me to use mean as an aggregate a lot, but in this case just using max for wind gusts was very performant and accurate. So that’s one thing that I did. The important thing to do is to just make sure that your group eyes are following what Grafana will interpolate in there. And then also it isn’t kind of like stepping over a boundary that-like if you’re collecting data every 10 seconds, you want to make sure that-I found that that isn’t a-there are certain situations where you can end up in a strange boundary, I think. I might be able to share some more information later on that because I need to actually look at my queries again.
Jack Zampolin 28:53.755 No problem. Thank you very much, Tim. Hopefully that answers your question, Stephen. All right, well, we’ll be sticking around here for another 10 minutes or so. If anyone’s got any more questions, we’ll get to them as they come in. But again, we will be posting the video for this soon so that you all can come back and check it out, and yeah, thank you all very much.
[/et_pb_toggle]
