Book Extract: The truth about big data

Big data is no longer just a shiny new management tool. It is an evolutionary set of capability that will have unanticipated uses over time

When I talked to big-company executives about big data, they all agreed that they considered big data to be an evolutionary set of capabilities that would have new and sometimes unanticipated uses over time. But every one of these executives conceded that they couldn't afford to make big data a mere academic exercise. It needed to drive value, and better sooner than later. For some, like Citigroup, big data technologies had to pay their way through rapid cost reduction. These firms had little use for big data simply as a shiny new management object.

Big data at Citigroup
Citigroup has been a data-driven financial services firm since before many of its competitors were in business. Founded in 1812 as City Bank of New York, the financial services conglomerate has evolved to serve 200 million consumer and institutional customers across 160 countries. The wider the company's reach, the greater the role of big data in its strategy.

The subject of corporate information-its integration, its quality, and its growing volumes-was a natural by-product of executive-level conversations around regulatory and competitive demands. In 2010, the company established a Chief Data Office. Shortly thereafter, the company downloaded Hadoop and began reengineering computation-heavy data transformations using the big data environment. A major focus of the Hadoop implementation is cost reduction.

The firm's plans include expanding that environment to refine its understanding of customer relationships and behaviors. On the consumer side, Citi is establishing relationships between so-called "white label" cards and commercial cards to detect potential increases in credit risk. And on the business side, the firm can view high-value transaction data and create custom supply chains or fine-tune financing structures for counterparties in commercial transactions. The low-cost, high-performance nature of its Hadoop infrastructure also gives Citi the ability to dynamically deploy targeted digital offers to customers' mobile devices when they step over a "geographic gate."

The lack of structure is the key element in big data: Thomas H Davenport

The skills necessary to convert unstructured data such as text, voice,  into structured data for analysis are quite substantial, Thomas H Davenport tells Ankita Rai

Is there any similarity between traditional analytics and big data?
Initially suspected that big data was simply a new term for business intelligence and analytics, but after doing some research I changed my mind. The activities necessary to convert unstructured data — text, voice, video, etc — into structured data for analysis are substantial. The lack of structure is the key element in big data. The analysis tools and approaches are similar to those we use with traditional small data.

Which are the key business functions that can benefit from big data the most?
Marketing is the function that has the highest level of activity now. Managing the many forms of customer data and touchpoints is important if you want to retain customers, promote loyalty, and deliver an integrated customer experience. Supply chain management will benefit from big data as companies put sensors in products, pallets, and containers. Human resource management is probably the fastest-growing function in its use of analytics and big data.

With smartphones, for example, we can know exactly where our employees are, whom they are communicating with, and what they are producing. We have to be careful not to invade employee privacy.

Can big data help companies to build competitive advantage?
The most successful firms will either find some source of proprietary data — on their customers or products — or be an early adopter of technologies and analytics that other firms in their industry haven't discovered or harnessed. In any case, however, you need to keep innovating. Airlines were early adopters of analytics, for example, but they stopped innovating for the most part, and now their analytics provide little or no competitive advantage.

Thomas H Davenport
President’s Distinguished Professor in Management and Information Technology, Babson College

Automating existing processes
Whether they need to do a proof-of-concept, explore preliminary data, or convince executives to invest, many companies have to prove the value of big data technologies as a first step to broader big data delivery. This often means delivering cost efficiencies or economies of scale within existing business paradigms.

Most of the executives we interviewed introduced big data technologies through an initial proof-of-concept approach to illustrate the high performance, lower cost of ownership, scale, and advanced business capabilities of big data solutions by applying them to current, often cumbersome, business processes. In some cases the proofs of concept showed the need for changes in other processes. At a major US airline, for example, analysis of call center speech-to-text data showed that customer interactions with call centers could be useful in predicting customer behavior, but also that call center processes needed some basic improvements that were more important than finer-grained predictive models.

Other companies see the promise of big data to bring together disparate platform and processing functions that were previously fragmented and siloed. The executives I interviewed spoke aspirationally of the ability to combine data reporting, analytics, exploration, protection, and recovery functions on a single big data platform, thereby eliminating the need for complicated programming and specialized skills to tie legacy systems together. They don't expect this to happen anytime soon, however. Some, like Sears Holdings are making major investments in supplying "data as a service" to make big data analysis a pervasive phenomenon throughout their organizations.

The good news about applying new technology to existing problems is that opportunities for improvement are already well understood, and thus consensus is more easily achieved. One banking vice-president explained, "Fixing what's known to be slow or broken gets more support from my CEO than promoting new technologies out of the box. He doesn't care whether our competitors are using big data. He cares that they could be gaining market share by making faster decisions."

Big data at Sears Holdings
When it comes to the adoption of information technology, Sears was years ahead of most retailers, implementing an enterprise data warehouse in the 1980s while most retailers were still relying on manually updated spreadsheets to examine their sales numbers. These days, the company is using big data technologies to accelerate the integration of petabytes of customer, product, sales, and campaign data in order to understand how to increase marketing returns and bring customers back into its stores. The retailer uses Hadoop to not only store but also process data transformations and integrate heterogeneous data more quickly and efficiently than ever.

"We're investing in real-time data acquisition as it happens," says Oliver Ratzesberger, (at the time of the interview) Vice President of Information Analytics and Innovation at Sears Holdings. "No more ETL Big data technologies make it easy to eliminate sources of latency that have built up over a period of time." The company is now leveraging open-source projects Apache Kafka and Storm to enable real-time processing. "Our goal is to be able to measure what's just happened."

Reprinted by permission of Harvard Business Review Press. Copyright 2014 Harvard Business School Publishing. All rights reserved.

BIG DATA @ WORK: dISPELLING THE MYTHS, UNCOVERING THE OPPORTUNITIES
AUTHOR: Thomas H Davenport
PUBLISHER: Harvard Business Review Press
Price: Rs 1,250