The ITOA SaaS Platform for Capital Market Data Visualization

By Chris Churilo / Product, Developer
Jan 06, 2020

If money makes the world go round, then technology is becoming its engine. Financial services, in particular, have widely adopted big data technologies and analytics to inform better investment decisions. The past decade has seen the rise of digital investment platforms. Financial institutions —  facing increasing competition, regulatory constraints and customer needs — are seeking ways to leverage technology to gain efficiency and competitive advantage. The speed that market data is distributed can also become critical when trading systems compete on how fast they are able to analyze data. And algorithmic trading uses, in conjunction with big data, vast historical data with complex mathematical models to maximize portfolio returns.

The only way these technologies can keep up this level of execution is to collect and act in real-time on key system performance metrics and events. The volume of high-frequency trade is high, so it should come as no surprise that the volume of time series data required to track performance is large. In addition, to drive latency down, data needs to be collected in small increments and be queried in real-time in order to take action.

A need for real-time capital market data visualization and dashboarding

The increasing volume of market data poses a major challenge for financial institutions. Along with vast historical data, banking and capital markets need to actively manage ticker data. Similarly, investment banks and asset management firms use voluminous data to make sound investment decisions. Financial institutions can differentiate themselves from the competition by focusing on efficiently and quickly processing trades and performing market data analyses to generate better insights and forecasts.

But achieving differentiation in the financial services industry requires real-time visibility, at the desired level of detail, into capital market data infrastructure across data centers globally. This need is driving financial industry demand for IT Operations Analytics (ITOA) platforms — platforms that retrieve, analyze and report data for IT operations. As cost optimization is a primary concern for IT leaders in the financial management space, access to ITOA helps provide visibility into system infrastructure performance, accelerate decision-making and improve planning.

IT Operations Analytics (ITOA) platforms and why they matter

The capital markets and financial services industry has a particular need for ITOA platforms because its products and services largely depend on up-to-date capital market data and related insights. ITOA technologies are used to discover complex patterns in high volumes of often “noisy” IT system availability and performance data. ITOA platform goals include improving the accuracy of application performance monitoring (APM) services, facilitating better integration with the data, and enhancing predictive analytics capabilities. Analyzing data via multiple sources enables proactive and data-driven operations with clear, contextualized operational intelligence.

Capital market data comes from exchanges (such as the New York Stock Exchange and London Stock Exchange) which typically send their own data rounds straight to the clients, and also comes from well-known market data vendors such as Thomson Reuters and Bloomberg, who have millions of instruments all updating at very high rates per second. A huge number of industries and individuals around the world need that data at very low latency all at the same time.

Responding to that need is CJC — a global UK-based company specialized in managed services, consultancy and products for global capital markets firms. CJC is meeting its clients’ demand for real-time visualization of their IT system health through Mosaic Open Analytics (mosaicOA): A monitoring tool, based on InfluxDB, focused on the global financial services sector and its requirement to import and interrogate large volumes of data, displayed in one place via easy-to-create IT operational performance dashboards.

<figcaption> Delivering on-demand visibility and advanced analytics while driving time and cost savings</figcaption>

An ITOA big data visualization platform for capital markets and financial firms

Monitoring complex IT infrastructures for capital market and financial industry firms is a major challenge since existing solutions can be expensive and often not fit for purpose. With so many moving parts, market data and transaction messaging systems are prone to system overloads, bottlenecks and other potential pitfalls that can disrupt data flows and bring trading to a halt. mosaicOA enables firms to visualize and analyze any significant data set, and thereby benefit from managing both the risk and opportunity that this data can provide.

mosaicOA supports real-time data infrastructures across data centers globally and brings large data sets together in one place to provide invaluable insight into financial and capital market firms’ data infrastructure.

Market data dashboard visualization and analytics data made simple

CJC has provided its clients with ITOA since 2012. Yet to support the visualization capabilities in demand by their clients, CJC realized that old-fashioned databases weren’t fit to purpose and that they needed a purpose-built time series database (since capital market data is time series data) in order to:

1. Cut storage cost - data storage represents one of the highest costs in the capital markets business.
2. Avoid the need to continually purge their relational database - relational databases are not designed for time series data.
3. Set flexible data retention policies - different volumes of data preserved can generate radically different trend line perspectives.
4. Enable advanced visualization and dashboarding capabilities - they needed an open source time series database that makes it easy to visualize data through various dashboarding tools.

InfluxDB Cloud met CJC’s requirements. Using InfluxDB, they set out to build mosaicOA from scratch.

<figcaption> InfluxDB deployment in mosaicOA</figcaption>

Here are some mosaicOA design highlights:

• mosaicOA uses InfluxDB to store metrics from hundreds of specialist servers, networking and middleware systems, with some firms doing over 1 million writes per minute.
• As different volumes of data preserved can generate radically different trend line perspectives, InfluxDB retention policies are key to how mosaicOA works.
• CJC designed mosaicOA to provide views at different degrees of granularity.

<figcaption> mosaicOA enabling end data projections</figcaption>

InfluxDB open source time series database vital to business

mosaicOA, as part of the CJC Cloud Platform, provides a private and secure cloud-based infrastructure offering elastic scaling, very high-performance storage, and real-time notifications of system health and activities. Using InfluxDB, mosaicOA registered a major win for the CJC private cloud platform and enabled CJC to go head-to-head on price with major public cloud vendors, deliver a service from a secure private location, and provide specialist add-on services outside of server orchestration. InfluxDB has proven vital to CJC’s business by enabling real-time market data visibility for one of the fastest-paced, most data-driven and highly regulated global industries. With so much to track and process, seeing is competing.

Learn more by reading the full case study.

If you’re interested in sharing your InfluxDB story, click here.