Industrial companies of all sizes are confronting massive volumes of data - financial transactions, logistics stats, RFID and bar-code data, images, web analytics, social media streams, machine data, sensor readouts - all streaming in at high velocity and in a dizzying variety of formats. This "big data" concept has emerged in recent years as an important new decision-making tool for manufacturers.
The term "big data" has come to represent the efforts of businesses to manage extremely large and complex sets of information and turn them into usable and actionable business intelligence.
U.S. manufacturers are getting very good "when it comes to capturing and analyzing data on everything from customer behavior to production-line efficiencies," and this is one of the factors setting the stage for "a U.S. manufacturing revival," Sebastian Mallaby, of the Council on Foreign Relations, writes in the Financial Times.
While recognizing big data as an issue and an opportunity, executive decision makers are understandably cautious about purchasing and implementing new technology. All you have to do is think back to boondoggles that arose during previous decades as organizations tried to install systems for CRM, ERP, supply chain management, electronic medical records (EMR) and other technology overlays that often fell flat. These systems can all be very useful, but only if they are accompanied by sound business processes and decision making.
As Shvetank Shah, who leads the IT practice at Corporate Executive Board, put it, "big data - no matter how comprehensive or well analyzed - needs to be complemented by big judgment."
Kimberly Knickle, who directs the IT strategy practice for research firm IDC, said her research reveals that companies will be spending $16.9 billion on big data technology and services by 2015. The market is growing seven times as fast as the overall market for information and communications technology (ICT). Knickle warned that "spending those funds wisely is going to be very important to manufacturers, starting with identifying the right use cases for those investments."
Knickle noted that big data is defined by four principal characteristics: volume, variety, velocity, and value. She also identified one of the key big data challenges for manufacturers; companies now confront "multiple types of data, from structured to unstructured or semi-structured data, in combination." As a result, manufacturing firms could end up analyzing "a mix of data from web logs with customer information stored in a database with sensor data that provides real-time information on inventory or shipments," which would prove to be a waste of resources.
The good news revolves around the opportunity for "big value." Open source software and falling prices for hardware are making data analysis systems more affordable, which means that managing and employing big data processes can pay off sooner.
In the case of consumer products manufacturers, for example, Knickle found that companies are using big data to understand the consumer better and to deliver better customer service.
"Specific examples include trade promotion optimization, SKU optimization, and modeling marketing campaigns," she noted. "More use cases in this industry segment include sentiment analysis based on behavior in e-commerce websites or comments on social media forums."
Other manufacturing segments are interested in extracting value from the supply chain. For example, "knowing when product shipments could be delayed based on ocean carrier data or other information sources for inventory in-transit," could deliver value "from knowing in advance if the current supply chain performance would impact customer orders."
A 2012 report from research firm Gartner found that big data processing challenges don't arise necessarily from large single databases, but from the interplay between information assets when they are combined. "When combining large assets and new asset types, how they relate to each other becomes more complex," the report explains. "As more assets are combined, the tempo of record creation and the qualification of the data within use cases becomes more complex."
Moreover, simply buying big-data technology won't deliver value to an organization. "Investments in analytics can be useless, even harmful, unless employees can incorporate that data into complex decision making," Harvard Business Review notes. At this very moment, "there's an odds-on chance that someone in your organization is making a poor decision on the basis of information that was enormously expensive to collect."
Research at Corporate Executive Board has identified the best decision makers as those who "effectively balance judgment and analysis, possess strong analytic skills, and listen to others' opinions but are willing to dissent." These data-savvy knowledge workers are "informed skeptics," and organizations need to invest in development and training programs that help more employees to become like them.
For instance, at Tiffany & Company, "nearly all knowledge workers participate in ongoing data education." As a result, employees are better equipped to leverage information, and the IT team spends more time helping them derive value from the company's data and less time answering simple data support questions.