Pay Close Attention to What Everybody Tells You to Ignore

Organizations develop blind spots. The drill is: people decide something isn't important, so all of the reporting ignores it, so nobody thinks about it, so nobody gets it put on their goals, and the cycle reinforces itself. Opportunities develop that everyone ignores.

A good example is involuntary attrition (attrition due to bad debt) at LTC. People concentrated on voluntary attrition and ignored involuntary attrition, and forgot that involuntary attrition was very much the result of a voluntary choice on the customers part. As a result, there were actually more opportunities for helping LTC with respect to involuntary attrition than voluntary attrition.

Daily Churn: The Project was a Complete Success and the Client Hated Us

The story eventually had a less than desirable ending. After producing accurate daily forecasts for months our work was replaced by another group's work, with the predictions that were much higher than ours. It turned out that having attrition sometimes higher than predictions and sometimes lower was very stressful to upper management and what they really wanted to be told wasn't an accurate prediction of attrition but that they were beating the forecast.

Ultimately the problem was a large difference between what management wanted and what they said they wanted. What management said they wanted was an attrition forecast at a daily level that was very accurate. To this end my group was constantly refining and testing models using the most recent data we could get. What this meant was that all the most recent attrition programs were already baked into the forecasts.

What management really wanted to be told was the effect of their attrition programs, and by the design of the forecasts there was no way they could see any effect. It must have been very disheartening to look at the attrition forecasts month after month and being told in essence your programs were having no effect.

What my group should have done is to go back roughly a year, before all of the new attrition programs started, and to build our forecasts using older data. Then we could make the comparison between actual and forecasts and hopefully see an effect of programs.

Surprisingly, I've met other forecasters that found themselves with this same problem: their forecasts were accurate and they got the project taken away and given to a group that just made sure management was beating the forecast.

Daily Churn Prediction

The next project gone off I want to talk about is when my group created daily attrition forecasts for a company.

Attrition is when a customer leaves a company. I was charged with producing daily attrition forecasts that had to be within 5% of the actual values over a month. The forecast vs. actual numbers would be feed up to upper management to understand the attrition issues of the company and the effect new company programs were having on attrition.

Because my group had been working at the company for a few years we were able to break the attrition down by line of business, into voluntary and involuntary (when customers don't pay their bills), we were able to build day-of-week factors (more people call to leave the company on a Monday) and system processing factors (delays from the time a person calls to have their service canceled and when the service is actually canceled). Our forecasts performed within 3% of actual attrition. Often we were asked to explain individual day's deviations from predictions which we were always able to do – invariably major deviations were the result of processing issues, such as the person that processed a certain type of attrition taking a vacation and doubling up their processing the next week.

We were able to break down the problem like this because we knew the structure of the information that the company data contained and we were able to build a system that respected that information.

The analysis was a complete success but the project died. Why tommorrow.

Building an Attrition System

We're talking about setting up an attrition intervention system.

This is all about information: how to get customer care reps the exact information they need to help out our customers.

The first big step is getting commitment to build a system and do it right. A well-done simple policy is a lot better than a badly done sophisticated policy. The next step is getting commitment to test the system at every level. Customers are fickle creatures and we don't understand how they will react to our best efforts. I'll have to say something about how to measure campaigns soon, but right now let's just say that we need to do it.

Let's start with the intervention. The obvious thing is to try to throw money at customers, but buying customers can get very expense quickly. What will often work better is to talk with them and just solve their problems. But here you need a good understanding of what their problems are. We can do this by a combination of data analysis, focus groups, surveys, and talking to customer reps. There are a couple of dangers here. 1) Trying to do this by simply building an attrition model. Attrition models will typically tell us the symptoms of attrition , but not the root causes. 2) Relying on the intuitions of executive management. Executives often have some ideas about attrition but rarely have a comprehensive understanding of why customers actually leave.

The next step is trying to get an understanding of the finances involved. What are the financial implications of, say, reversing a charge the customer didn't understand? It's going to be different for one customer that has done this once and another customer that habitually tries to take advantage of the system.

Everything, everything, everything needs to be checked against hard numbers. We have experiences and form opinions on these experiences but until be check we don't know what's really going on.

The last step is what people usually start with: building an attrition model to tell when customer are likely to leave. A standard attrition model won't really give us the information we need. We don't just need the chance someone is going to leave. We need to match customer with intervention; that's a much more specific type of information.