The role of big data in a digital world

It's a concept that gets people excited but, at the same time, it sounds a little daunting. What exactly is big data? Do I need it? How do I use it?

The role of big data in a digital world

It's a concept that gets people excited but, at the same time, it sounds a little daunting. What exactly is big data? Do I need it? How do I use it?

What is Big Data?

It's essentially exactly what it says - a really, really huge collection of information drawn from a myriad of internal and external sources. According to Gartner information becomes big data when the volume can no longer be managed with normal database tools.

One of the difficulties many airlines face is staying abreast of these disruptive technologies and establishing themselves as retail outlets for their services.

Collecting, storing and analysing big data provides operational intelligence insights that can help make a company more productive, profitable, competitive and secure.

On a practical level, it helps you better target your audience, implement personalisation strategies, speed up supply chain processes, and even change the way your run your business entirely as you open new avenues of competitive advantage.

Sources of big data are all around us and can roughly be divided into business data, human data, and machine data from the internet of things.

Typically these are: social media, archives, public web, data storage, media, sensor data (such as from cars, road cameras or cell towers, etc), web server log files, documents, apps, etc. It could be flight path information, weather changes over a decade, the number of people attending a concert, medical records or podcasts - the list is endless and growing by the minute.

IBM data scientists break big data into four dimensions: volume, variety, velocity and veracity. This infographic explains and gives examples of each.

Do I need it?

The short answer is: yes. To stay competitive a business needs to know as much as it can about people, the environment it's operating in, and who and where the competitors are. Without it, everything is very much guess work.

Big data analytics helps businesses:

  • identify more efficient ways of doing business
  • find new opportunities
  • streamline operations
  • increase profits
  • find ways to reduce costs
  • stay ahead of the competition
  • enable faster decision making
  • increase customer base
  • learn how to retain existing customers

Big data future uses and applications. Graphic: grasundsterne GmbH. Source:

How do I put big data to use?

Big data is not only about storing and extracting charts of information, but using the proper mining technologies to find the relevant information within each type. These include Hadoop, MapReduce, YARN, Pig, Apache, Flume, Hive, Cassandra and many more.

There are so many big data technology options and vendors, it can be difficult to find the right answer to what you need. It's no wonder data science is becoming the hot new career of the year and as a first step employing a data analyst (or scientist) could be the best move to steer your big data mining and analysis endeavor in the right direction.

The three steps of using big data

Data sources: Decide what data are relevant to your business. Forbes has listed 35 free big data sources that can give you a great start.

Data platforms: This is where data is captured and managed and then converted into customer insights. From Hadoop to SASInstitute, listed these 16 top big data analytics platforms.

Big data analytics tools and apps: this is the 'front end' that business executives, analysts, managers and others in your organisation access to deliver better products and services to customers.You can go for big data analytics software, often bundled with platforms, or put your newly employed data analyst to work. lists 40 platforms and analytics software tools to get you on your way.

Examples of big data in action

  • Car manufactureres can combine telemetry (automated communications) with warranty claim data to predict when parts would fail or when vehicles need servicing
  • A retailer can analyze various suppliers in terms of price, quality and on-time delivery, allowing them to compare suppliers and negotiate better contracts
  • Hospitals may use doctor and nurses' notes, patient family history, laboratory results, diagnoses, medications, social media and patient surveys to improve their diagnostics and interventions and so save patients' lives.
  • Crowdfunding businesses can help entrepreneurs gather data by opening their business idea to a wider audience and get market feedback
  • A clothing manufacturer can use photographic data collected from social networks to identify popular designs and colours in real time and then brings these designs to market while they are still popular, outsmarting the competition whose operations aren't as agile and is trying to figure out next season's trends.

The bottom line is businesses intelligence is changing and both large and small companies have everything to gain and nothing to lose by taking all that data that's just sitting there and putting it to work for them.

Navigate back