Data is still the king, but the focus is now on actionable and automated insights

Data is the new oil – we have heard this many times in the last few years. While many organizations have ramped up investments in the data space to have control and access to their data, there is still a big gap when it comes to driving insights from this data, and more so in driving actionable insights. 

In these two-part articles, I would look back on how Data and BI space has evolved and where it might be headed in first part, and then take a deep dive into some of the upcoming trends in this space in the second part.

MIS and reporting era

Organizations globally started taking note of this new currency – data, with the advancements in the automation of MIS and reporting, that was earlier a largely manual exercise by various departments within the organization. Technologies such as erstwhile Business Objects, Cognos, etc made this easier in those days. This was the time when pixel-perfect reporting made headlines and organizations poured in money to acquire these technologies and adopt at scale. Still, the basis of all this was data that could be centralized and consumed more easily. The focus was more on factual insights and decision making was still largely based on that instinct which had developed from years of experience.

While organizations made great strides in front end technologies, there was a huge gap when it came to performance and scalability. The focus shifted to the centralization of entire data in a data warehouse which was agile, fast, and scalable. Innovations in data engineering space resulted in a plethora of new-age DW and ETL tools that made this quick and easy. Data engineering technologies such as Datastage, Oracle Warehouse Builder, etc made creating data marts and warehouses much easier which were the basis for all the reporting. Organizations spent a huge amount of time and money on acquiring these tools and started getting their data game correct. While this worked for a little while, it could not yield complete results as the requirements of end business users kept on changing. While MIS and reports still had its place, business users now wanted more – a way in which they can explore data on their own

Disruption in BI space

Self-service BI started gaining popularity in the late 2000s and caught up like wildfire in the first half of 2010s. Technologies such as Tableau, Qlik, and Power BI disrupted the entire BI market and forced even the erstwhile powerhouses such as Business Objects and Cognos to relook at their value proposition and the offerings. Business users started loving them mainly because of ease of use and less dependence on IT. They could now create their dashboards and play with their data. A kind of power that they had never experienced before. This coupled with excellent technologies such as Alteryx, Informatica, etc in the data engineering space, organizations accelerated investments in this combination. Soon we had everyone in the organization playing around with their data and generating their reports and dashboards. This has probably been one of the largest disruption seen in the software market after the cloud. 

What the future holds

As things have evolved in the last few years, the need for factual reports and dashboards is slowly getting down the priority charts of most organizations. While this is still important and I wouldn't discount it yet, it is not the key question in front of CXOs today. Decision-makers are now asking more from their BI investments and would like to have actionable insights in an automated way. Gone are the days when getting factual insights more easily was a winning proposition. With very little time on hand, business decision-makers are asking a simple question – why has something happened and what actions they need to take in the context of their KPIs. And yes, they do not want to browse through different dashboards or create few of their own to get this. They want all of this information in simple natural language in an automated way for the periods they define, and the context in which they are looking at it. New technologies and solutions are evolving and the older ones are ramping up to win this race. No one has cracked the holy grail 100% yet and it is to be seen how things evolve from here. Data might still be the king, but the way it is consumed has changed, and any BI investment which provides automated, actionable insights contextually is going to be the winner.

We at Course5 are working with our customer organizations in helping them enable this at scale with Course5 Discovery (https://www.course5i.com/course5-discovery/). Please reach out to me if you would like to know more. 

To be continued