Metrics as a Service (MaaS) for Real-Time Monitoring in Modern Web Applications

Session date: Oct 23, 2024 08:00am (Pacific Time)

This webinar explores how developers use InfluxDB as a Metrics as a Service (MaaS) platform to set up real-time monitoring for their modern web applications built with Node.js and React.js. It also covers the technical aspects of integrating cutting-edge open source monitoring tools to enhance the performance and responsiveness of your web services.

In this webinar, you’ll learn:

• How to use MaaS with InfluxDB for real-time data monitoring of modern web applications
• How to manage application processes with PM2, an advanced process manager for production-level Node.js applications
• Strategies for visualizing time series data to detect trends and anomalies crucial for maintaining optimal performance and security
• Practical examples of building a comprehensive monitoring stack tailored to modern development environments

Watch the Webinar

Watch the webinar “Metrics as a Service (MaaS) for Real-Time Monitoring in Modern Web Applications” by filling out the form and clicking on the Watch Webinar button on the right. This will open the recording.

[et_pb_toggle _builder_version=”3.17.6” title=”Transcript” title_font_size=”26” border_width_all=”0px” border_width_bottom=”1px” module_class=”transcript-toggle” closed_toggle_background_color=”rgba(255,255,255,0)”]

Here is an unedited transcript of the webinar “Metrics as a Service (MaaS) for Real-Time Monitoring in Modern Web Applications.” This is provided for those who prefer to read than watch the webinar. Please note that the transcript is raw. We apologize for any transcribing errors. Speakers:

• Suyash Joshi: Senior Development Advocate, InfluxData

SUYASH JOSHI: 00:14
Hello and welcome, everyone. We’re just waiting for one more minute for everybody to join in. Welcome wherever you are coming from. Feel free to type it in the chat. It’s nice to have you here on a Wednesday. So, we’d start in one minute.

SUYASH JOSHI: 01:14
Okay. So, I guess we can start now. I will share my screen. Awesome. So, let’s go full screen. Welcome to this webinar on metrics as a service for real-time monitoring of your modern web applications. My name is Suyash Joshi. I work at InfluxData. This webinar is being recorded, and recording will be available after the webinar. You’re also welcome to ask questions in the chat. I’ll answer it after the presentation. And you can also email us with any questions afterwards, and we will get back to you. So, I’m glad to see you all here. And let’s learn about how to monitor web applications, what is metrics as a service, and how does InfluxDB help in that regard? This is my email. You can also directly contact me. Or if you feel like tweeting something, below are the Twitter or X handles as well.

SUYASH JOSHI: 02:24
So, I am a Senior Developer Advocate at InfluxData, the company behind InfluxDB, and I’ve been working for over a decade as a software engineer, developer advocate at different companies. Web application has always been part of my job. And before that, I obtained my undergrad degree in computer science. And actually, recently, I finished my master’s degree here in London, where I live right now. And I also have a hobby, so I love doing magic tricks when I’m not coding. So today, the agenda is as follows. We’ll start by learning about time-series data and what do I mean when I talk about metrics. Then we’ll dive into the context of modern web applications. We’ll see some things in action, and I’ll show you some resources where you could further dive into it, start incorporating what you learned today, and then, of course, we’ll take questions.

SUYASH JOSHI: 03:35
So, time-series data and metrics 101. If we were doing this live, I would love to know how many people are already familiar with the InfluxDB, how many people use it, but there are always new people. So, time-series data is essentially any data where one of the key attributes of the data is the time. Now, the time in this example here, we are seeing different types of data. On the left, we have a stock chart showing Apple’s stock price at a certain time. And as you can see in the graph, you can go back depending on what duration of time. So, time property can also change. You can go from seconds to minutes, days, months, etc. And time is critical in this data, right? If you want to buy a stock and then you want to sell it, you want to sell it at a time when the price is ideally high, right? You want to make a profit. And if you want to do an analysis on that, you want to see what date the stock was the lowest or highest. So, time is a critical factor in financial data.

SUYASH JOSHI: 04:50
Also, IoT data. For example, if you have a temperature monitor, you want to know what time it gets hot, what time it gets cold. Biometrics data. If you are measuring, let’s say, your heart rate, you want to know, “What time did my heart rate spike up?” So, time is a critical element. And whenever it is a critical element in the data, that type of data is considered a time-series data. So, time moves linearly, and we can measure time in different scale, different granularity. And that generally is represented on the x-axis, and on the y-axis can be some other value that you’re measuring. So, for stock, it could be the stock price. Further examples of this, especially in our context today, would be web performance. So, you are measuring performance of your web applications. Time is essential, right?

SUYASH JOSHI: 05:51
I mean, there is a term, very popular, called latency. What that means basically, it means, “What is the time duration something takes for an information to be sent from A to B? What is the network latency? What is the latency when you make a database call?” So InfluxDB, you can do real-time performance monitoring, and we’ll talk about that. Why? Because it just removes the latencies. The queries are really, really fast. So here on the right side, all of this information is stored somewhere. Well, first, you have to gather these metrics, then you have to store them, and the third step is you want to visualize them because it’s easier to understand and see. So here, we’re seeing time-series data visualized in these different types of graphs, and I’ll talk about these in a few moments.

SUYASH JOSHI: 06:51
There are two essential elements when we talk about metrics. So, metrics, the way we like to define it is you can differentiate between metrics and events. Metrics would be something that is gathered at a regular interval. In case of, let’s say, heart rate or biometrics, you can capture your heart rate throughout your day from your smartwatch. But you want to be notified when your heart rate spiked above a certain threshold. Now, when that anomaly happens, we would call that as an interesting event. It’s something irregular, and it is something important. Now, you also want to track metrics which are just some kinds of—essentially, it’s another type of event, you can say, but it’s happening at a regular time interval. It might not be that interesting, but it may be important for archiving purpose—for long-term analysis if you want to do events—would be more interesting. Maybe in the immediate term, you want to know what happened, but you also want to know, “How was my heart rate for this whole week or month?” etc.

SUYASH JOSHI: 08:14
If you have any questions around these, just drop them in the chat, and I’ll come to it towards the end. So, when we have these metrics, well, we can collect them various ways, and we can analyze them in different ways. If you want to do everything under an umbrella, we would call that metrics as a service. You probably must have heard various things as a service. Functions as a service, very popular, right, such as Amazon’s Lambda, where you can run a function as a service. Metrics as a service would be a cloud service where you can collect, store metrics, and then you can perform some kind of analysis on that. And that’s what InfluxData offers. So, we offer our platform that can be used for metrics as a service platform. Here, metrics are measured over time. They’re structured, very common—we talked about network monitoring metrics, so that would be a type of metric that is also called telemetry data. And you can do various operations on it.

SUYASH JOSHI:
09:18 So, collecting is one. You can probably process on it because you may not need to store everything you collect. You may want to process it. You may want to clean up the data and store only relevant data, or you want to do some kind of a mathematical computation on it like aggregate the data or find the mean, max, etc. And then you want to visualize this. So, if you store it, then you want to visualize it. And finally, you also want to set up some kind of an alerting system. So that would be monitoring in real time, and then you can also set up alerts. And InfluxData, the nice thing is that we provide a lot of these services, and we integrate with other third-party providers who offer additional services that helps with metrics as a service end-to-end process. So, we’ll talk about, in our context today, a couple of type of monitoring, how you collect these metrics. Web app performance is our topic, but also related to it is application performance, slightly different; network monitoring, slightly different; security monitoring.

SUYASH JOSHI: 10:36
All of these are slightly different variations and equally important depending on your job, depending on the context, and all of these basically emit some kind of data, which is our metric that we want to store and analyze and monitor. So, some use cases of time-series data, some other use cases beside web app monitoring. Well, industrial IoT factories, they have a lot of sensors. InfluxDB is used heavily in that. Robotics. Robot has many sensors. In real-time, you want to know what it’s doing. And cloud metrics. If you’re running services on the cloud, you want to know—because you’re paying for it, you want to know in real time—if there is something happened, you want to allocate new resources, etc. Even if it does it automatically, you’re interested in knowing how your cloud services are being consumed.

SUYASH JOSHI: 11:38
Couple of things that come to mind when we are collecting this type of data is ingestion. So generally, this data is emitted in high-scale, high volume. And you want a database that can handle high-volume ingestion of data. Time-series data is perfect for that. Also, because this data is almost emitting all the time, depending on your use case, it can pile up and become huge. And what becomes important—because if you are storing the data, there is some kind of—generally—some cost involved. You want to reduce your cost. So, you want to be mindful if the data can be stored in a compressed manner, in an optimal manner, so you’re not paying for something that you don’t want to. Cardinality is another aspect of this type of data. Essentially, you can think of it as different attributes of the data or different columns of the data. And if the data has multiple values— if you’re familiar with InfluxDB, you probably already know what I’m talking about. But let’s say a robotic sensor that has various type of setting. It’s giving accelerometer data in X, Y, Z axis. It’s giving other type of data that can have unlimited cardinality. Because as data scales, these values are also increasing. So, you want to be able to store all of that. So, with time-series database, it’s important that it can scale or offers almost unlimited cardinality.

SUYASH JOSHI: 13:23
And then last thing is that we want to store the data, but then we want to perform query on it. We want to analyze the data. And that should happen in almost real time, so the database query performance needs to be optimized. And the nice thing is that InfluxDB, if you look at the right side, it is ranked the fastest and the best performing time-series database. And it’s not me, but it’s a website you can go called DB-Engines. It does a comparison analysis of different type of database. If you search for time series, you’ll see InfluxDB is regarded as the leader by far. But there are other types of databases as well. Very common ones are relational database like MySQL, Postgres, and generally, people get confused. They think, “Okay. The data, it has an attribute of time, which is important. What if I just store that in a relational database?” That is not the best way to do things. One, because the relational database is not optimized for all of these things on the right side that I talked about. These things are specially optimized in a time-series database. Fast ingestion, querying on time, cardinality, compression, a lot of these things, relational databases lack because they’re designed for things such as your Amazon. You want to store customer information. You want to store a catalog of products that you sell, or you want to sell user information. Relational databases are better suited for that type of use case.

SUYASH JOSHI: 15:08
Document storage or document type of database, such as MongoDB, are useful for a database that is unstructured, such as email or chat, etc., where the schema of a relational database doesn’t lend itself to it. So, there are different— the point I’m trying to make is there are different use cases, and those use cases determine what type of database is best suited. Elastic is another product. It’s an open-source company. Also, they have their own service which is best suited to store something if you want to do or perform a search. If you have a search on your website, you can use something like Elasticsearch for that. So different databases have different purposes. 20, 30 years ago, we didn’t have this. But now as our industry is evolving, we’re realizing for the best job, you want to use the best product because that serves the customer of the company. It provides the best experience.

SUYASH JOSHI: 16:14
So just a little peek inside InfluxDB’s current 3.0, which is our latest version of our database. It’s built on top of open-source technology. And we offer various at the high-level control for the users or developers through our Cloud UI portal or our APIs. So, you can do all kind of management if you need to, or you let us manage in the cloud, but you can administer it still. And then you can perform all kind of operations on the database. So, you can do that through our CLI, through our APIs, or through the Cloud portal. And then the layer below that is underneath what is happening. So, we have ingestor, like I talked about for high ingestion of data, we have a query engine, and a lot of these are using open-source tools such as Apache Flight, which is used for high performance of network transfer, Apache DataFusion— this is an open-source project. All of these are Apache project that we use in our product, which is used to provide the SQL query engine.

SUYASH JOSHI: 17:30
Apache Arrow. It’s very important for doing in-memory, storing information in the RAM for columnar data, which is our database. So, it’s a type of database that stores information in columns. Apache Parquet, which is important to store the data in an optimized manner. The file format is optimized for a low footprint, so it ends up saving costs for the customers. So, all of these are latest and greatest open-source technologies that we incorporate, and it comes to you just by using InfluxDB. It offers developer productivity. Because they’re based on open standard, you can integrate with various third-party services, and there’s plenty of documentation information about these online if you’d like to deep dive in it.

SUYASH JOSHI: 18:29
So, we’re jumping into the modern web application monitoring metrics. What are the metrics that we are interested in? So, let’s talk from an application level. So, if you’re running a web application, ca couple of metrics that might be important are average request time it took for a web request. So, for example, somebody opened your website, how long did it take for them to load the page? Number of HTTP request that were served by your applications in a day, in an hour, whatever time horizon. You determine that. Or what about how many errors? How many errors were given by the application, and what was the cause of the errors? Well, first, you need to know what happened, which would be the metric, and then you dive into how it happened. And then that would be the analysis of that.

SUYASH JOSHI: 19:30
So, this is just few examples. We’ll dive further into this. Other type of metrics are cloud metrics. So, if you’re running operations on the cloud, that would include server networks, so such as CPU usage, disk, free space, memory, how it’s being used, how much is free, total number of instances of whatever services you’re running such as, let’s say, in a Kubernetes environment, the status of these services, how many are up, down, etc. Metrics related to the browser. So that would be page views. These can also be business metrics. So, on your page, if there is, let’s say, a button that says, let’s say for InfluxDB, “Sign up for InfluxDB,” or whatever, Amazon, “Buy,” how many people are clicking on that button? Well, that’s a metric that you want to store. So that would be a call-to-action metric. For business reasons, very important.

SUYASH JOSHI: 20:34
Average session duration. How many people,or, how long did they stay logged in to your web application? Very important business metric. These ones, you can get by using JavaScript because they are related to the browser. Similar metrics, but more that are user-focused or UI and UX-focused. So, one would be— in InfluxDB, we define something called, “Time to awesome.” In this context, I’m using that phrasing. When somebody, again, comes to your website, how long does it take for them to get something important they came there to do? Let’s say you come to InfluxDB. How long did it take you to sign up, store the data, and maybe make a query? What is that you determined? What is that your key task, and how long did that take? That’s a very important UX metric, right, for user experience. Again, I talked about page load time. We’ll dive further deeper into this.

SUYASH JOSHI: 21:42
It’s also something called perceived performance. One thing is actual performance; something else is called perceived performance, which sometimes, you will see on an app or a website. If it is loading slowly, they will put a spinner or they would start showing not the actual data, but some kind of boxes and things that seems like something is about to load or a loading bar. What that creates is a perceived performance that something is about to load, it’s working fast, versus just giving a blank page. So, it’s very interesting, these things. And I encourage you— I’ll share some links where you can dive further. So, let’s double-click on our context. So, for web application, there are various metrics that you can gather from different tools but also from web standard performance APIs. So, the web standard has been evolving for the last many decades, and there are many JavaScript APIs. They’re part of W3C, which is the web body that organizes and develops APIs for various web-related things, standards such as HTML5, JavaScript, etc. So, there are many APIs that are provided-you can say by the browser-that gives you information about your web application performance.

SUYASH JOSHI: 23:13
And then there is also a tool that is very popular, which is the third column, which is called Lighthouse. It’s a tool that you can say Google develop, but it’s kind of like open source that you can use. And that also gives critical metrics. And so, there are various metrics on the first column. And on the third column, I’m showing which one would be related to the Lighthouse tool. In the fourth column, we’re talking about what type of category of metric this covers. So, in this case, they’re all about performance. First four are about load time. So, they give you the timestamp when the navigation starts, when the DOM content loaded, when the load event ended—now, these are very fine-tuned metrics. You might not need all of these, so that is why I would start—if you’re a beginner, I would start with the Lighthouse metrics because that just gives you the DOM content loaded. That’s probably a higher-level metric that might be most relevant. “When did all of the DOM of the HTML page loaded?” You want to know that time.

SUYASH JOSHI: 24:29
Then there are about resource. So detailed timing information about resources on the page that the browser requested: images, scripts, video, whatever it may be. Timing of the painting of the pages. Again, it’s kind of like when information is being displayed. So that can be broken down into when the first paint happened, which would be when the first pixel was rendered, or when the browser is rendered, the first content in full that could be text or an image. Lighthouse called that FCP or First Contentful Paint. Largest Contentful Paint is also very critical. So, if you have a huge image, that could block the UI because that is being loaded. And so, it depends also how you structure your HTML and JavaScript. How are you writing your code? And we’ll dive deeper into this. So Largest Contentful Paint is another key metric that is also provided by Lighthouse.

SUYASH JOSHI: 25:40
Some other further ones are cumulative layout shift. So, this is also very common with async applications. So, for example, if you click on something and it changes the page, well, now the page has to be rendered again by the browser. A lot of these things, we don’t pay attention because they’re just part of the life. They happen dynamically, and we are used to it. But all of these operations are critical, and they can be measured. And when you measure something, then you know how long everything takes, and then you can optimize these things. So, all of these, if you use the tool called Lighthouse-I’ll show you in the demo-it gives you a score. So, Google has a scoring parameter. So that would be, “This is the best score.” You can say grade A, B, and C. So, you can compare your website performance against the standard they have created. So furthermore, there are metrics around, “First input was displayed.” This is useful for interactivity. So, somebody has to type something, well, they cannot until it’s rendered and it’s ready for the user to interact.

SUYASH JOSHI: 26:52
Blocking time. If something was blocked, how long did that take before the page became responsive again? Generally, it’s the main thread in the browser or browser is single-threaded. It’s that thread. How long was that blocked? Time to interactive, when page became fully interactive, ready for users to interact with, and then performance-related metrics— further performance-related metrics. So that would be, for cumulative layout shift, based on viewport changes. So, if I make the browser smaller or larger, it laid out again, we can get that metric. And then you can also create your own custom marks if you want to measure when certain event happens. “Give me the time for that.” So, you can create your own custom metrics. Memory. Again, this is about how much memory it is consuming, the web page, because if it consumes too much, the browser can crash. Network information metric. So, this is about network performance.

SUYASH JOSHI: 27:59
And this is what a lot of these things look like. A lot of these are available to you. Well, we’re interested in our context getting these through API so that we can analyze them, and we can store them. Because this is time-series data, right? But you can also see them through the browser tools if you just want to have a quick peek at it. If you want to do analysis, if you want to store this for various purpose, if you’re going to log these, you want to use the API. And then you can also set up alerts for certain operations that are critical, like we talked earlier. So here, this is in the developer tools of your browser. If you go to the Network tab, it will show you all the requests that were made, how long did it take—and here you can see— and then you can analyze, “Okay. What took long?” and then you can dive deeper into why it took long.

SUYASH JOSHI: 29:04
So, this is for measuring UI. So, we were talking about the time something took for painting the UI. Here, we are looking at—In this screenshot, we are doing— So you can do an audit of a performance. You can record the web page performance, you can capture, and then you can replay. And then you can see, “Okay. How long did it take?” You can make change in your code, and you can run that audit again. So, you can also do a performance audit and record those. Furthermore, or I would say, steps during, let’s just say a web page load. So, this is going beyond the browser. If we look at what the request might be going through, so if we start it on the right side, some kind of a load event that happens, that could be the page load event, and we can break that down further. So, when the DOM was loaded, etc.

SUYASH JOSHI: 30:14
And then before that, we are talking about when there was processing, right? So, before the DOM was loaded, JavaScript was being executed. There was some processing that was happening. How long that took? Before that, we made a request to the server. When was that request made, and what was the response? How long did that response take us? Now, one thing I want to point out here, you don’t just have to—you might not just be interested in time. Time is critical. Timestamp is critical. But here, especially with the request and response, we’re also interested in counters. How many requests were made, and how many responses? What type of response. All of this is a perfect use case for time-series database. So, in InfluxDB, all of this, you can think of it like a table, which would be your measurement, and then there would be fields. So, it would be how many requests, and the values would be 200, or the response is 200 or 400 or whatever it might be, right? So, you can store that. And that would have a timestamp as well. And timestamp, you can add the timestamp, but anything you store in InfluxDB automatically gets a timestamp also.

SUYASH JOSHI: 31:40
So other values that might be interested in is, “How long did it take for the TCP connection to be made? Or the DNS resolution?” And then you can store: “What was the DNS? What was the domain name? If there was caching, did we use caching or not?” So etc., etc. So, all of this comes—if you’re doing an audit of your network, you would look at all of these things. And on the very end— In this chart, it doesn’t show. But beyond caching would be your server metrics, which would be server application metrics. And then if you have a database, so let’s say you have a relational database, I mean, that’s a big factor as well because those databases must do some kind of query or transaction. How long did that take? So, all of that would come under your network audit.

SUYASH JOSHI: 32:41
So here, if we’re going to use InfluxDB in our use case, the way we want to use this is we want to use the Node.js client SDK. And for our Cloud V3, the best client SDK is our V3 SDK. It’s a lightweight SDK. And one of the applications that I use for server-side monitoring—especially if you’re building a node application—is called PM2. It’s a free tool, and it gives you all kind of application analytics: disk usage, memory usage of the node server that is running, whatever, your Next.js application or your Express Web application, etc., on the back end. On the front end, we would use something like I showed you, the Lighthouse tool. Now, you can look at it through the browser, or better, we can use it programmatically. And again, it has a NPM module. And then like I showed the W3C APIs for monitoring our performance, various aspects of the performance.

SUYASH JOSHI: 33:50
Finally, we want to see all this data, analyze it, and we can use various tools. If you already have a web application, if you use something like React, there are various libraries to create charts and graph. So, one use case would be you take all this data, you store that in InfluxDB, and you query that, and you see all of that in your charts and graph. Or you can hook up a third-party tool like Grafana or Power BI, and you can see the visualization through Grafana dashboard or Power BI or whatever your favorite visualization tool is. Next step would be if you want to set up alerts. You can also then set up alerts. So, something that alerts you. For example, you want to get a Slack notification or SMS when the request— There was an error or whatever event that is interesting to you.

SUYASH JOSHI: 34:53
So, there is a lot of data that you can collect, but I always like to show this slide when we talk about– because not everything is important or may not be relevant. So, there are a couple of questions I would like you to ask when you are setting up monitoring service, essentially, to collect various metrics. What is the data that you are collecting? A lot of these APIs I showed you, they can give you a lot of data. Do you want to collect all the data? What is the granularity or the precision of the data that you want to capture? InfluxDB, you can store data as precise as a nanosecond, or you can store it for a day. Now, what precision do you want to store this data? What granularity are you interested in capturing and storing this data? Or by the minute or seconds? How frequently do you want to collect this data? At a certain interval? It could be a couple of seconds, almost real time—like I said, you can store it in nanoseconds or every subsecond or by minute. So, you would ask these questions before you set up your monitoring system.

SUYASH JOSHI: 36:12
How long do you want to store this data? Again, we talked about storing. Storage cost money. Nice thing with InfluxDB, it’s optimized for storing. It compresses. So, you are actually paying much, much less than if you would be storing it otherwise. Data analysis. Do you need to do real-time analysis, or do you want to save it for archival reason, historical purposes? That is another question you want to ask. And then finally, you have all this data. Now, how do you make sense of it? Is this good or bad? So, there are benchmark for all these things. Like I talked about, Google has a thing called— There is something called Google Page Score, Google Web Vital Score. There are also things called service legal agreements, SLAs. Your company might promise that our service: “This is the uptime, and this is— That we promised in our agreement that becomes legal that, at this time rate,” or whatever it might be, “This is our service. We offer the service at this level of agreement.” So that would be critical because you want to make sure that you’re fulfilling that criteria. Right?

SUYASH JOSHI: 37:33
So, we talked about this, visualization and alerts that you can set up after answering all these questions, storing this data, then you want to set up some kind of visualization alerts because this is part of the larger monitoring process. Now, there is something I want to show you, but unfortunately, it’s giving me an error right now. I was just trying to sort it out, but I will show you couple of things. So, the app is here. Let’s just go through this way. So, I’ve created a Next.js app with a React front end, and it connects to InfluxDB Cloud. What it is, it’s an app that stores stock trading data. Real-time, you can get stock data through it. Almost real-time. I wouldn’t say real-time. It queries at every few seconds. And I’m using a third-party API called Alpha Vantage. And if you want to run this app, well, you would have to use your own API. But the nice thing is that they give you a free API that you can grab. And you want to store it in your InfluxDB, so you can create a free account in InfluxDB.

SUYASH JOSHI: 39:03
So, what we can do is, at least, we can log in here. And if we go, “Query data”— So, this is InfluxDB Cloud Portal. And here, first, if you are new to it, you would sign up. It’s free to sign up without credit card. The one I’m using is called Cloud Serverless, our offering, and we’re adding data to it through API. So, I’m using Node.js, our Node.js SDK, to store the information. When you clone and download the app, all you have to do is run npm install, and it will install all the dependencies on the backend. Then you just run npm run dev. But before you do that, you want to open this file and store your credentials. So let me, at least, show you. So, this is the application, and it has a little ticker display. It has a chart graph. So unfortunately, you can see some logs here. I’m making SQL queries. But sadly, it’s giving me an error right now from the Alpha Vantage API. It looks like I’m running over their limit or something. So, I need to replace this one because I’m using their free plan.

SUYASH JOSHI: 40:30
So essentially, if we look at the package.json, we have here InfluxDB client that you would install via NPM and then other dependencies. So, I’m storing my credential in a ENV file. This is a Next.js application. It uses React, and it uses this library, Recharts and, of course, ReactDOM. So here, when you would clone this, you can see my information. I will delete it after this. But here is what you would have to do. So, all of this would be blank. In this, if we look at—okay. It’s not here, but it should be at the root. Okay. I need to add this file. I will add this file because I think I’ve ignored it because it’s my private file. But what you would need is— I would put a sample .ENV file here, and you would clone this repo, and you would have this without any of this information. So, I’ll have all of the values blank. The keys are, we need our URL from the InfluxDB, which is essentially this part of the URL. So, you would type that in, and then you need a token, which when you have an account, you can get a free token.

SUYASH JOSHI: 42:07
So, if you go— Couple of things here, this is your URL. You can find your organization, which is right here, which you would put the name of the organization. Bucket name is your database. So, in my case, I call it stock data. You would put that, and you would put your token. When you create your bucket, then you can— Or you can create a token anytime. So, you can go and—there are various ways to create tokens. So, we can navigate here, and we can just say, “Generate API token.” You can generally recommend it to create a custom API token. You don’t want to get all access, security reasons, and then you can say which bucket I want to use this for. So, in my case, it would be the stock data. I want to read, and I also want to write. Because the application writes data to InfluxDB, getting the data from this service that gives you real-time stock information. It stores that in InfluxDB, and then it shows that using the React front end.

SUYASH JOSHI: 43:17
And then we are doing analytics of the web page. So, it’s showing you all the web page metrics as well. And I use something called PM2 on the back end, and I’m also using Web Performance APIs on the front end. So, the link is here for this app. You can take a screenshot, but it’s also going to be sent to you in the recording. And you can download and clone it, and I’ll update this with the environment variable files so you can just run this project. So that is our application. I mean, I can walk you through this, but I guess—the code, you can look into it. I’ll walk you through some key parts of it. So here, we are using Alpha Vantage API. Here, I’m using the key, but that is actually coming from our environment variable, which, again, you’re going to define it here. This variable. And then it’s querying that it’s getting the data every few seconds. And after it gets the data, it uses another service, which is our InfluxDB utility, and it stores the data. So, I’m storing the data for seven days. I’m retaining it for seven days because I’m not interested beyond that right now.

SUYASH JOSHI: 44:54
And when you create an InfluxDB bucket, you can also determine how long the bucket needs to store the data. So, we provide different retention period that you can set for how long you want to store it, depending on— And then after that, beyond that, the data would be automatically deleted. Or you can have an option of never deleting the data, and that also depends on what type of plan you have. So certain plans have certain limitations. I’ll show you this part. So, we are storing the data in InfluxDB. We are writing the data, and I use something called line protocol. So, I’m storing the stock price, the symbol of it, the actual price, and the timestamp. Timestamp here, I’m converting it to nanosecond, and I’m storing that. And the other thing that this does, it queries the data. And the way I query it, just by writing a SQL query. Select from my measurement, stock price, where I’m getting the current, minus the current time interval for that particular symbol that you would enter in the web application. And it executes the query, it shows you the result, and I’m logging various things here.

SUYASH JOSHI: 46:14
So that is basically—sorry, the demo is giving me problem, but you can run this project, and I’ll update the instructions here. But just to go back, few other resources I want to share with you. This is a tool. For example, I don’t know in the UK. I watch BBC for news, so why don’t we open bbc.com? Here, we can do a performance audit, various metrics we were talking earlier, and then we can see what it looks like on web or mobile and it’s giving a score. BBC is actually doing quite well. So, this is the benchmark. If it is under green, means you’re doing quite well. Largest content paint, it took 1.1 second. If you want to look at BBC website, has a lot of information, large images, etc., etc., but it is well optimized here. We can see in-page interaction. Time to first byte is also critical. So, these are the measurements.

SUYASH JOSHI: 47:32
Well, oops. Sorry. I’m going to make this as a URL. Oops. Okay. Yeah. This should be a URL, so you can click on it when you look at the slides. This is called the Lighthouse tool by Google. Now, this one, you can just open it. If you’re on Chrome, you just go to your tab, and you can open Lighthouse just by clicking on, “Lighthouse.” And now, you can generate a report, you can analyze page speed, etc., etc. So right now, it’s asking me to close other tabs in the same origin to open this tab. Okay. If I do this, I click, “Analyze Page Speed,” and you can see, right now, it’s analyzing. It’s doing it for mobile view, but we can do it for desktop view, and it gives me a score. Well, Google’s own webpage, I guess it’s not all green, so there’s scope for improvement. So here, we can see the performance. Now, this is the benchmark score. 90 to 100 is grade A, grade B, and grade C. In this case, Google scored 61, and it tells you LCP, largest content paint, is one of the key issues and other things. And you can further deep dive into it, and you can see how long something took. And then it even shows you frames at each level, and further, you can diagnose.

SUYASH JOSHI: 49:14
So, it’s a really helpful tool, and I would recommend you use that. Now, we just used it in the browser. Here, Google tells you that you can use it. You can use it through—Let’s see. You can use it—and these are the key metrics. These are the key metrics we looked at that it tracks. You can also create your own custom metrics. Underlying, it’s using a lot of the web APIs for that. And the way to use it is you can use it through the browser, but you can also use it through NPM. So NPM, if we type Lighthouse, we go to the NPM package, and you can see the project on GitHub and you can install it and it’s showing you the same thing. But you can programmatically get it. That’s what the application gets. That’s what I would encourage you to have within your application. But be careful what you’re measuring and what you want to measure and monitor. So that is that.

SUYASH JOSHI: 50:20
We go back here. There are other things. I also wrote an article about metrics as a service, and there are more articles coming where I talk deeper diving into that, including the web application one that is soon going to be published. Good article by Mozilla about monitoring web performance using web APIs. So, they talked about all these browser-based tools, what they are, how to use them. A lot of my information came from here. I really like her web knowledge, web development. Reading up on Mozilla’s documentation, encourage that. And then further, if you want to learn more about InfluxDB, you can go to our documentation, how to use our APIs. You can also go to our university where we have free courses, video courses for various things, and you can follow them. It’s free. And join our community. You can join our flag forum. Ask questions. I’d love to hear from you, answer you. And here’s a QR code and the link to try our database in the cloud. If you haven’t already, you can also grab it from any cloud marketplace if you are already using these cloud services.

SUYASH JOSHI: 51:38
With that, let me move on to questions. Thank you so much. And a lot of questions here. Let me take them one by one. So, I’m going to look at the questions, and I’ll just read them out loud. “Is there a limit of measurement size for InfluxDB in the number of points?” Unlimited. Almost unlimited. So, you can store billion cardinalities. You can store a lot of data, cardinality, and a lot of points as well. So almost unlimited. We have users. They have huge petabytes of data. They’re using InfluxDB, so I don’t think that is a problem. If that is, then we’ll help you. We’ll figure it out. We have our ways around that as well. “Can InfluxDB store events and metrics as a service or just metrics?” Yes. We can store events and metrics both. So, it’s optimized, like I talked to you about, storing regular events, but also metrics.

SUYASH JOSHI: 52:42
“Does InfluxDB handle logs and traces well?” Great question. So, you can store logs and traces in InfluxDB. Currently, it’s not optimized for that. So, you can store it. Yes, it’s a time-series database, but in near future, we’ll have it optimized, fine-tuned for storing logs and traces. Right now, yeah. So, I would encourage you to try it out, see if it meets your need, then you just use InfluxDB for events, metrics, log traces. But right now, I would say it’s not fine-tuned for that. “Is it already possible to integrate in CI/CD jobs?” Yes. So, when you install these tools and you set up monitoring, well, you can create a report as part of your CI/CD. And you can create—just like how you would do automation tests, and you would have a report, how many tests failed, and whatnot—you can set up your own benchmark. “Okay. I want to know about five things. The page load, the number of requests, response time, and something else.” And you can generate that report as part of your CI/CD. So, you would have to write that custom logic for that, but all the tools are there programmatically. They have APIs, but you would have to write the logic, what you want to do, as part of your CI/CD jobs.

SUYASH JOSHI: 54:11
“What is the supported deployment of this product?” Yes. So, our product, you can run on-prem. So InfluxDB comes in three flavors. What I showed you, the free one is Cloud Serverless. Without putting a credit card, you can use that. Whereas we have a cloud dedicated, and we have a clustered offering. So, if you want to run on-prem, you will use something called Clustered, which is our cloud dedicated single-tenant offering. You can run on-prem. “Can we get this code repo?” Yes, it’s on GitHub. And the presentation will be sent to you. It has the link. “Thank you.” Okay. Thank you very much. “Do you have documentation details, integration with Power BI?” Soon, we will have documentation. There are some issues with Power BI, but I’ll try to find information for you. And if you ask this in our forum, I would be able to answer it directly to you. Just drop us a Slack or in our community forum this question and more. Thanks. Yes, the recording will be there. Appreciate it. Thank you very much. I hope you learned something today.

SUYASH JOSHI: 55:23
There is one more question. Okay. I guess we answered this one live. And if there are any other questions, yeah, you can just email me or just go to InfluxDB forum or you can even put it in Stack Overflow. I monitor our Stack Overflow as well. I can try to answer you there, or the community might answer it there. But yeah. Monitor your applications, use metrics that are critical to you, store them in InfluxDB, store logs, try it, and see. The best way to do these is just set up an afternoon, try these things, and then you would know the best answer for yourself. So, thank you very much again, and hope you have a rest of a great day. Bye-bye. We will stop the recording now, but the recording will be available and will be sent to you.

[/et_pb_toggle]