The Business Intelligence Discipline
If you want to deliver Business Intelligence (BI), you need to understand what it actually is. Unfortunately the industry is ridden with people that have a very poor understanding on the subject. Even worse, there are those who think they know what it is, but fail at the most fundamental test. In this post I introduce my view on the Business Intelligence discipline, which I have promoted as the MedModus CTO. Since we adopted this model in MedModus, it became the backbone of our work ethos and the foundation of our competitive advantage.
Let’s start by looking at the following definition of business intelligence, by Gartner:
A broad category of applications and technologies for gathering, storing, analysing, sharing and providing access to data to help enterprise users make better business decisions.
I see a serious flaw on this definition. BI is not software — but a business strategy. Of course there are BI software out there, but confusing BI systems (i.e. applications and technologies) with the broad concept of BI is a fundamental flaw in the delivery of BI projects. In fact, the term Business Intelligence first came about over 150 years ago, way before we ever dreamed of computers. And understanding this diference between BI and BI systems is the fundamental test I was referring to.
I often joke that equating BI to software is like an original sin for the industry, perpetuated by those coming from IT with little business background. And I state that with the conviction of one who was once also a sinner.
During the last stages of my dissertation I came up with the draft of what would became the Business Intelligence Discipline presented below. There wasn’t much time to incorporate it on my dissertation, so I decided to revisit it at a later stage. Right at the first weeks as CTO of MedModus, I revisited the model with our CEO, and it remained untouched until January 2018 when we did a small yet crucial amendment to it.
The Business Intelligence Discipline Diagram
The diagram outlines five elements that I consider to be crucial in the pursuit of a good BI discipline, which are industry know-how, methodology, architecture, software and technology, and data science. At the centre of the diagram, there is improvement delivery, which is the objective of any BI project.
Industry Know-How
Knowing the vertical in which you want to deliver business intelligence for is a key element for understanding business needs and goals. It is about making sense of the data we get, and to be able to come up with the key performance indicators. Understanding the industry is fundamental for driving continuous improvement in a BI programme and for providing industry-specific services
Quite often I see IT BI consultants going into customer sites with little understanding of their customer’s industry. This often result in them having to ask the business what they have to report into, which often rubs management the wrong way. Such reactive approach takes people’s time, since one wouldn’t be able to come up with new reports and dashboards without asking around. Also it quite often results in poor report creation, as some users might not fully understand the data and its KPIs, and some might suffer from biases and informal fallacies such as illusory correlation and questionable causes.
Methodology
Methodology has to do with how we pursue business intelligence from a resources and processes point of view. This includes project management methodologies such as the Kimball’s DW/BI Lifecycle or Moss’ Extreme Scoping. It also includes other fields of interest such as enterprise data/information management.
NOTE: Some methodologies (e.g. Kimball) also encompass the way we architect a DW, while others (e.g. Moss) focus solely on the project and resource management aspects.
Architecture
Architecture has to do with how we pursue business intelligence from a technical point of view. Sure, this includes how we architect the data warehouse (and/or data marts) as well as data models (e.g. normalised, relational data warehouse vs dimensional model). But it also includes security aspects such as access control and row-based security, and infrastructure considerations like business continuity and capacity planning.
Software and Technology
I made a point at the start that business intelligence isn’t software. While software and technology accounts only for 1/5 of the BI discipline, the knowledge and experience we have on the software and technology being used to deliver BI can dictate the success or failure of a project. Sure we can come up with a great methodology and what we think is an outstanding architecture, but how we deliver it using the software and technology at our disposal? How do we configure row-based security in SQL Server Analysis Services or Qlik? How do we partition tables in Oracle? Can we create the measure we came up with in Tableau? Do we have the knowledge to write code in SQL and to design the ETL process in SQL Server Integration Services? Can we deliver this architecture in the cloud using Microsoft Azure or Amazon Web Services?
As you can see, there is a back-and-forth between software and technology (i.e. the tools) and architecture. We need to understand if the tools we have at our disposal will allow us to deliver what we came up in our architecture. If not, is it because we don’t have he knowledge on how to do it using the tools we have, or is it because the tools we have do not support what we came up with? Can we find an alternative with the tools we have, or should we obtain another tool?
Data Science
Being a guru in Oracle databases or a Microsoft Most Valued Professional for SQL Server means nothing if you do not understand data science to a degree. Understanding data science dictates our ability to make proper sense of data and deliver a valid analysis, and at the very least one must understand some fundamental principles of statistics. This is crucial to avoid amateurish mistakes and misinterpretation of data, such as the Simpson’s paradox. It is also important for understanding patterns in data and how to respond to them.
With the countless amount of free tutorials and courses on data science and statistics available all over the web (e.g. edx.org, Coursera and Khan Academy) there is no reason to shy away from the subject. Again, a fundamental understanding of statistics is the bare minimum, but those looking into more advanced subjects like predictive and prescriptive analytics, data mining and machine learning must pursue data science with much more enthusiasm than just the ‘bare minimum’. My thoughts on the matter are further explained in this post.
Improvement Delivery
Central to the business intelligence discipline is the goal of improvement delivery. This element was the last element incorporated into the diagram, added in January 2018. It summarises the overall objective of business intelligence. Why are we pursuing BI? Why are we doing all of this? The answer to these questions must always converge to improvement delivery, which shall equate with measurable benefits. In other words: business value. Commercially, we can often pair measurable benefits with cost reductions and/or increase in profits.
This element serves as a guiding start across all other elements in the discipline. When choosing the right KPIs, when deciding the scope of the next development iteration, when evaluating different architectures, when selecting the right software to address the requirements, and when creating analytical models.
What about enterprise data management?
Those familiar with the concept of enterprise data management might be wondering where it fits in this business intelligence discipline diagram. The diagram shows the elements that are crucial in the pursuit of BI from a higher level. The topics covered under enterprise data management are, in fact, distributed across different elements of the BI discipline diagram — mostly under methodology and architecture.
Final thoughts
The pursuit of business intelligence as a strategy goes beyond software and technology. Any organisation that is serious about BI must understand that BI is a discipline consisting of several elements. Each element carrying the same weight in importance, towards delivering the ultimate benefit of delivering improvement.
This work (Business Intelligence Discipline) is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This hits the nail on the head. I never understood when applying for BI consultant positions how they shy away from industry focus. If you don't know the business well, it is so much more difficult to add the desired business value. Thanks for sharing.