≡ Menu

Moving to fact-based decisioning

Creating a culture of fact-based decision-making is critical to long-term success with Digital Decisioning. If an organization believes that making decisions “with your gut” is the best way to work, then it will tend to resist Digital Decisioning no matter what. Over time you need to create a fact-based culture.

I am purely empirical. I am not attached to any romantic notion of how the business should be run.

Gary Loveman CEO of Harrah’s Casino

Fact-based decisioning requires a focus on the decision-making approach as well as the results. It requires a broader statistical awareness and a policy of presenting data and decisions as a set. It does not dismiss the value of experts but rather complements it and it builds on a data strategy foundation.

Decision-making approach not just results

Tom Davenport, author of “Competing on Analytics” and “Analytics at Work,” once asked a rhetorical question. Suppose you had two executives, and each had to make one decision that really mattered to your business results. One had a really thorough and coherent approach to making the decision that involved analyzing data, considering options, and carefully thinking through consequences. The other liked to go with his or her gut—essentially guessing which option to pick. Which one would you rather promote and employ? It seems pretty obvious. But what if the first executive was unlucky, and their choice did not pan out, while the second one did get lucky and was successful. Which one would your performance assessment and reward process actually promote and reward? The answer for most organizations is that the guesser would do well, and the better decision-maker would do poorly.

To become an organization that values fact-based decisions, you have to value the decision-making approach, not just the results. Ideally, this reaches from top to bottom with executives being evaluated this way, as well as managers, supervisors, and front-line staff. This kind of culture will make the adoption of Digital Decisioning easier and will be strengthened by those same systems. It also creates the environment necessary for experimentation to flourish, as it establishes the value of improving decision-making approaches—not just making good decisions right now.

Statistical awareness

Most people don’t have a good sense of statistical concepts. They don’t understand what “statistically significant” means, and they don’t have the skills or experience to look at what’s happening to see if there are material trends that should be considered. They are also poor at differentiating between correlation and causation. Organizations that are attempting to become fact-based need to worry at least as much about this broad problem as they do about recruiting those with deep mathematical and statistical skills. The latter are important for building predictive analytic and optimization models, but the broader base of statistical awareness is critical if those models are to be adopted and effectively used.

One of the biggest issues in this regard is the tendency of people to take personal experience, stories from friends and colleagues, and dramatic examples to heart even when they are not statistically significant. In law, there is even an expression for this: “Hard cases make bad law.” In other words, the more dramatic and unpleasant a case is, the less likely it is to be a good basis for making a new law. This problem comes up with designing Digital Decisioning also, as in when specific remembered examples of unhappy customers are given greater weight than the overall pattern of outcomes.

The inability of many to differentiate between correlation and causation is also problematic. People are good at spotting patterns but can easily mistake a simple correlation (customers who have this product are the most profitable) for causation (customers who buy this product become more profitable). The correlation can be useful as a shorthand way to tell profitable customers apart—more or less. If there is no causation, however, the selling of this product to a new customer may not make them more profitable. It might be closer to the truth, for instance, that the organization tends to only offer this product to those who are already profitable because it is an expensive product that is good at generating loyalty. Selling this to a new customer may make them loyal but unprofitable.

Present data with decisions together

Many systems just present data, expecting users the system to make the right decisions based on that data. Others might present only the decision, with no supporting or explanatory data. In general, a fact-based decisioning culture is best served when the two are presented together.

This means building decision support systems that are explicit about the decisions they are supporting so that data is presented in that context. It means identifying which actions might be recommended or unavailable, even if the decision cannot be automated completely using Digital Decisioning. It means prioritizing predictive analytic models that are explicable and ensuring that the reasons models make predictions are presented as part of the decision itself. Combining data and decisions will make it clearer as to why a decision is being recommended and will improve the ability of those consuming data to make the right decision.

To support this approach, organizations need to invest some time and energy in managing data lineage and ensuring transparency in data transformations. Presenting summary data without any explanation of where it came from or using model results that rely on data that is not well understood can undermine the best intentions. Predictive analytic models can be considered new data elements in this context, and their lineage and explainability can be part of the data presentation.

Role of experts

In his book Blink (Gladwell, 2005), Malcolm Gladwell describes the way in which a group of experts rapidly identified an ancient Greek Kouros statue purchased by the Getty Museum as a fake. Their reaction to the statue turned out to be more accurate than the initial set of scientific tests. Essentially, their know-how and years of experience enabled them to perform a kind of complex pattern matching. They took a pattern they had in their minds about real Kouros statues and matched it to the actual example in front of them. Thousands of hours of experience trying, failing, learning, and trying again had developed their ability to make accurate snap judgments.

At first sight, this would seem to contradict a focus on fact-based decisioning. After all, if these experts can make snap judgments that are this good, then perhaps “gut” decisions are going to be more effective. But there are several reasons to doubt this. Most obviously, there is a limited supply of experts. Even if your most experienced customer service representative makes great judgments, they are not typical of your call center. Embedding their judgment and experience into Digital Decisioning helps everyone else make better decisions. More than that, though, a focus on fact-based decisioning can also help make the best use of the power of experts.

The judgment of experts is often mysterious[md]the experts have no idea why they make the decisions they make. This reliance on the unconscious makes it hard to explain decisions; this can be a real problem both in terms of sharing what works and in terms of explaining decisions to regulators or in legal cases. Digital Decisioning can help by exposing and making transparent decisions so that the knowledge can be taught and shared. “Snap” judgments are also prone to corruption and bias. An emotional investment in a particular decision can seriously undermine judgment, and bias exists and distorts judgment in many areas. Data-based approaches can show where bias is undermining the quality of decisions. Data-oriented systems can also improve the quality of feedback given to experts and so increase the value of experience in building judgment.

Predictive analytic models are particularly useful also in supporting expert judgment. Human judgment tends to work best where there is just a little bit of data. For instance, insisting on doctors asking just a few, pertinent questions (four, in fact) when someone is admitted to an emergency room for a suspected heart attack improves their decision-making dramatically. Many decision-making failures can be traced to too much information. Using predictive analytic models to simplify large volumes of data into a probability clarifies and focuses decision-making on the information that matters. Digital Decisioning can create an environment where the allowed options and most relevant data are presented to focus experience and expertise effectively.

To get the best decisions from your decision makers, you must build tools to nurture and protect those decision makers.

Have an information strategy

Last, but by no means least, you need to build this approach to data-driven decision making on a coherent information strategy. Understanding the information you have, use, and accumulate is critical, as is having a plan to bring that information together so it can be integrated and consumed. A multi-step approach, where an information platform is developed incrementally and using specific projects to build an increasingly sophisticated environment, is key. Organizations should resist the temptation to build a whole information strategy and platform first and then think about analytics and Digital Decisioning. Building specific solutions, expanding the platform, and implementing an information strategy should happen in parallel.