Lately, the two words “Big Data” seem to be in the headlines of every business magazine and at the very top of every finance executive’s worry list. There’s good reason to be concerned: We know that our data is growing at a tremendous rate, and when it comes to taming, organizing and making sense of this spiralling mass of data, the stakes have never been higher.
But I’m going to let you in on a secret - Big Data isn’t a new problem and you don’t need to fear it. Capital markets firms have been managing Big Data for the better part of 20 years. What’s changed is technology: Recent advances have made our best practices for managing Big Data a whole lot more effective.
In short, Big Data is an old problem with some exciting new solutions. It’s time to drop the trepidation and look at Big Data for exactly what it is: an opportunity with ample rewards for financial services firms of every size.
Grappling with Granularity
Big Data’s bad reputation is primarily a function of the industry’s new requirement for fine-grained data analysis at real-time speed. The market collapse of 2008 gave birth to an era of regulatory micro-management that rendered our then-current state of analytical prowess almost instantly trifling. Whereas it was once enough to run profit and loss calculations at the end of the trading day, today’s organizations are required to know their exact collateral position and overall market exposure at virtually every moment throughout the trading day.
While many firms maintain that they’ve been doing intraday data analysis for years, the truth is that it took most capital markets firms two to three days to discover the extent of their exposure to Lehman Brothers once it collapsed in September 2008. Since those dark days, the bar has been raised precipitously.
Social media is also extending the profile of Big Data. New data types and characteristics—such as audio, voice and text — are helping financial organizations understand customer demographics. In the retail banking sector, social media data can help banks develop behavior profiles to improve upsell, marketing and customer service interactions.
The fact that today’s data sets are much larger than they used to be is merely a mitigating factor in the primary struggle to answer explicit questions at a moment’s notice. It raises the complexity but changes little about the problems we need to solve.
Big Data for All
Even the size of an organization has little effect on Big Data requirements. Small firms want and need access to Big Data insights just as badly as large organizations, but they have unique challenges that the big organizations don’t have. Namely, they lack the time and resources to develop Big Data solutions on their own.
What could be seen as a natural disadvantage for smaller firms is, in fact, a perfect opportunity for market symbiosis. Larger organizations, which will already be investing in solutions to enable Big Data analysis, will wisely offer the fruit of those investments to smaller firms, who will be happy to buy Big Data analytics as a service from a larger partner. Buy-side firms get what they need at a price they can afford; sell-side firms maximally monetize the analytics investments they need to survive.
The revenue from such value added services is potentially significant. For example, exchange providers that offer metrics as an upsell atop a traditional market data subscription can expect to earn 15 to 20% more per subscriber—with relatively few new sales or support costs. Paying for Performance Higher revenue will come in many forms. The most direct form will be improved market performance enabled by robust pretrade limits and other algorithmic functions in trading systems. Imbuing those systems with simple mechanisms to help authorized users directly manipulate algorithms without IT involvement is another.
Arguably the best way to move the needle with Big Data will be at the enterprise level. With better visibility across the organization at a granular level, risk managers can make better business decisions more quickly and keep regulators at bay.
The solutions that can help you bank that extra revenue exist today. But they aren’t all equal; in fact, they aren’t even terribly similar. From multi million dollar supercomputing solutions to highly parallelized desktop solutions that do analytics in the cloud, your cost-to-performance ratio will vary widely depending on the solution you choose. The money that you spend—or save—when solving your Big Data challenge is the biggest hammer in your toolbox—and the only thing about Big Data that should be keeping you up at night.
David Parker is Director of Global Financial Services Solutions at SAP.