Capitalisation and Business Intelligence Ventures

This is my first post, and I thought I would articulate some of my experience in the burgeoning Business Intelligence (BI) markets and how a company can benefit from BI.

My forage into these markets started way back 2009 with a system to interpret twitter data (called fullbe). A hiatus followed those initial failures and in 2012 I returned to the issues with a renewed vigour: this time focusing more on Big Data technologies such as HDFS, Hive, Pig, and the emerging deep learning methodologies (such as utilizing deep neural networks).

My interests have morphed into what is really data intelligence, a bedrock form of intelligence underpinning business intelligence (the patriarch in the intelligence market). As I examined data, I concluded data patterns revealed user behaviour.

I talked to experts in the field and I felt drawn into their world of bits and bytes;  graphs; and relationships between data. The challenges faced by the engineer miscommunicating with the investor are well known, and I certainly felt the palpability of that hackneyed conflict.

By analogy, I mused over the development of the locomotive engine, which revolutionised all transport, and how colliery capitalists were driven by a quest to reduce their horse fodder costs: engineers were driven by the leaps in engineering after the boiler was invented. If their fodder costs had not soared due to the Napoleonic Wars, innovation could have been delayed for decades.

Hedley discusses how the locomotive engine developed in Who Invented the Locomotive Engine?: By Oswald Dodd Hedley; and for the Chapman engine in 1813:

This experiment cost the owners of the colliery nearly £3,000.

These historical developments underline that change is driven by economic pressure burdening the capitalist.

Today, I work with understanding e-commerce data which involves scrapping web data, analysing patterns, and converting the data into structured format. From my enquiries, I have concluded a lot of companies are performing these tasks manually using templates, and BI specialists tend to avoid data mapping from unstructured to structured data completely – they display and manipulate data only.

This shortfall in potential market demand speaks to the issues of under-capitalisation of emerging ventures. The Center on Capitalism and Society discusses the role of capitalism and innovation:

Yet innovation is not the only aspect of capitalism on which there is not yet much fundamental understanding.

The problem of solving unstructured data mapping to structured is very real and very solvable but until capitalists start to support innovators the problems will not be solved within our lifetimes.

Larry Page feels passionately on the subject of innovation and capitalisation, and I am proud to quote him:

Invention is not enough. Tesla invented the electric power we use, but he struggled to get it out to people. You have to combine both things: invention and innovation focus, plus the company that can commercialize things and get them to people.