Boston Time Series Meetup at Wayfair

By Ed Bernier / Use Cases
May 18, 2018

Last night was our second Boston Time Series Meetup hosted by Wayfair.

We had a turnout of around 45 people even with the severe thunderstorms that hit the Boston area a little before the event. Ryan Betts, Yaacov Ankori and I attended from InfluxData and a number of people from Wayfair attended as well. Wayfair provided a great space with 3 large screens and lots of seating room.

The meetup started with Jim Hagan from Wayfair giving a talk entitled “The Four R’s of Metrics Delivery” which describes the Wayfair environment and the data pipeline they’ve built so far for metric collection, delivery, and use. Wayfair currently uses both Graphite and InfluxDB as a Time Series Platform and sends a diverse set of event trackers, timers, and other system metrics from over 2,000 VMs running hundreds of applications. Their systems are spread across three major data centers and the data is used by their developers, business stakeholders, and by our internal alerting engine. Most importantly their 24x7 Ops Monitoring Center is using this data to constantly analyze the vital signs of Wayfair’s IT infrastructure and storefront operations.

At the 3 data centers, they use Kafka Mirror Maker to replicate the data to all 3 locations and they have (3) 6 node InfluxDB Enterprise clusters, which are dedicated to different workloads:

• Store Front metrics
• General monitoring of things like Kafka queues, containers, etc.
• All other application monitoring

Here is a copy of his presentation on SlideShare.

Their InfluxData implementation is still a work-in-progress, and they are working towards building their next-generation pipeline which will take advantage of Kafka and the Telegraf streaming service to create a more robust data topology. Here is the blog post where Jim Hagan wrote in more detail on their environment. He spoke for about an hour, and the audience was really engaged and asked a lot of questions.

Next, we had Ryan Betts from the InfluxDB team do a talk on InfluxDB Internals and how Time Series Databases are unique. It was an update to the talk that he gave at InfluxDays NYC 2018, and you can take a listen to the recording or download the slides. Again lots of great feedback.

Overall a good event. The next meetup will be held on July 17, 2018 at Wayfair with a talk from Ben Bianchi of Wayfair who will discuss an exciting new application using time series data to provide real-time intelligence to mission-critical algorithms. Also joining the meetup will be Jacob Lisi from Grafana who will discuss how to most effectively monitor and visualize your Kubernetes cluster using the Grafana Kubernetes plugin and PromQL.

Hopefully, for the next meetup, the weather will cooperate. Hope to see you there!