Every business needs data to drive decisions, but how do leaders create an information system able to keep up with their changing objectives?

They used to call it business intelligence

Businesses have always needed data to make decisions. As far back as World War II, it could be argued that the power of data-driven insights delivered unquestionable advantages. Prior to the Battle of Midway, the plans of the Japanese Fleet were signposted by US Intelligence teams. And Churchill was convinced that the code breakers of Bletchley Park were his biggest secret weapon.
Jump forward seventy years or so, and the world has shifted from thinking about data as something that gets reported at the end of a long analytical process, towards real-time data insights interpreted as they are created—Business Intelligence.

employee looking at financial data on the wall

From reporting past events to living with data

The fundamental change we’ve seen in digital businesses is the opportunity to capture data from all points of the compass as part of work tasks, largely because more and more software robots and machine-to-machine interfaces exist. Instead of waiting for documents to be scanned, a human to write things down, or a data entry screen to be key filled, today business systems can capture data through robots and present insights instantly.

I don’t mean to say that business people don’t report on anything today—that certainly isn’t the case. There are still millions of organizations around the world that continue to use manpower and spreadsheets as a workhorse to find out things. For some at least, the promised digital summer never happened after the analog report writing winter.

Where spreadsheets and self-service business intelligence tools fall down as reporting tools

Spreadsheets (like Microsoft Excel) create huge productivity wastage owing to the demands they place on humans to perform highly repetitive tasks that machines tend to be better at. But, push the issue of productivity to one side for a moment, and there remains the issue that spreadsheets are a poor solution for making sense of business data for a number of reasons:

1. They are prone to formulaic inaccuracies (creators making fundamental mistakes)
2. They suffer from version control (people referencing old reports by mistake)
3. They shroud original data as the purest evidence of data integrity (when you create data, or republish it in a spreadsheet, it’s impossible to drill down to where the original sourced data came from in the first place)
4. They don’t cope well with large data files (the data processing capacity of spreadsheets is often limited to your desktop processing power)
5. They are easy to manipulate

employee analysing data graphs pointing at the monitor in the office

BI is popular but not pervasive

With all the hyperbole about digital business, and the need to be data-driven, one might imagine that every business has an active strategy for embracing decisioning, but that isn’t the case. While Gartner predicted the Business Intelligence would hit $16.9 Billion in 2016, even now, the figure is only just hovering around the $17.7 billion mark according to Select Hub. So, why hasn’t the growth appeared as industry pundits expected?

Speak to people in the ‘enterprise data industry’ and they think it’s more down to people than technology. Nick Lawrie is Managing Director of the data science consultancy, NDMC Ltd; a company with a 20-year track record in tackling some of the world’s most challenging data applications. Lawrie believes it’s ‘people issues’ that are holding innovation back. “Our experience working with management teams is that almost without exception, departmental managers are hungry to harvest and use data more readily, but they face resistance from business units and individuals who seek to retain control over their spreadsheeting empires. While executives can see arguments for cloud-driven data science solutions, the path to achieve this panacea is spanned by political and emotional barricades in many organizations.”

The shift from ‘known-decisions reporting’ to the powering of curious minds

It’s not all bad news for the business intelligence industry. This year, Information Week published research suggesting a third of large-scale organizations plan to adopt decision intelligence by 2023. They argue, one of the main reasons for this significant growth comes from businesses that see a role for business intelligence beyond reporting of outcomes.

Since the dawn of the business intelligence hype curve, the primary driver for investing in tooling has been to analyze data in response to known questions. What organizations are today increasingly wanting to do is equip their people with tools to start asking new questions of what they do, and how they do it; to equip the curious with the answers they seek by harnessing the value of known and new data.

Lawrie argues that, while the tools of the BI trade are similar to those that came before, their application is very different. “Management teams want their people to be proactive in improving customer experience and business outcomes, but they can’t do that without benchmarking performance today, and asking new questions about processes, customers, use of resources and assets, etc. The bigger challenge for technologists and data scientists, becomes less about the presentation of data, more about how to harvest data from multiple system silos, data quality, and integrity.”

The way forward

Finding solutions to powering the curious begins not with business intelligence then, but in data abstraction, organization, aggregation, and quality enrichment—what enterprise IT architects would label the extract, transform, and load (or ‘ETL”) layer. Cloud technology is coming to market in the form of Platform-as-a-Service (PaaS) solutions like Mendix, Encanvas, Boomi, and Talend to put all of your data to work by providing the ability for decision-makers to harvest and re-use their data for other purposes, such as asking new questions. These new digital platforms bring the power of artificial intelligence and cloud infrastructure to bear in taking your data, helping to overcome the ETL and data integration challenges that have hampered business intelligence projects in the past. Only time will tell, whether PaaS technology solutions ultimately create the data-driven business world that industry commentators speak of.

Ian Tomlin

Ian Tomlin


Ian Tomlin is a marketer, entrepreneur, business leader and management consultant. His passion is to help make great ideas happen. Relentlessly optimistic about the potential of technology for good, Ian’s 30+ year career has focused around the intersect of strategy, technology and marketing. He writes on subjects including enterprise computing and organizational design. He also works as a consultant and advisor to the executive teams of PrinSIX Technologies, Answer Pay and INTNT.AI, helping to rethink their marketing in order to tell their brand story.

Ian has founded a series of successful businesses including NDMC Ltd (2003), Encanvas (2006), and Newton Day Ltd (2019). He has written books, articles and guides on brand, digital transformation, enterprise applications, data science, workforce management, and organizational design. He can be reached via LinkedIn or Twitter.

Blog template

Blog template

Let’s examine how a no-code cloud application fabric can transform how your business extracts value from data and help overcome data silos.

How to Avoid Data Spaghetti

How to Avoid Data Spaghetti

Most businesses today operate more than 80 SaaS apps, making ‘data sense’ harder. What can be done to tame the data spaghetti?