Monday, March 31, 2008

The Secret Laws of Analytic Projects

The First Certainty Principle: C~ 1/K ; Certainty is inversely proportional to knowledge.
A person who really understands data and analysis will understand all the pitfalls and limitations, and hence be constantly caveating what they say. Somebody who is simple, straightforward, and 100% certain usually has no idea what they are talking about.

The Second Certainty Principle: A ~ C ; The attractiveness of results is directly proportional to the certainty of the presenters.
Decision-makers are attracted to certainty. Decision-makers usually have no understanding of the intricacies of data mining. What they often need is simply someone to tell them what they should do.

Note that #1 and #2 together cause a lot of problems.

The Time-Value Law: V ~ 1/P ; The value of analysis is inversely
proportional to the time-pressure to produce it.

If somebody want something right away, that means they want it on a whim not real need. The request that comes in at 4:00 for a meeting at 5:00 will be forgotten by 6:00. The analysis that can really effect a business has been identified through careful thought, and people are willing to wait for it. (A cheery thought for those late-night fire drills.)

The First Bad Analysis Law: Bad analysis drives out good analysis.
Bad analysis invariably conforms to people's pre-conceived notions, so they like hearing it. It's also 100% certain in it's results, no caveats, nothing hard to understand, and usually gets produced first. This means the good analysis always has an uphill fight.

The Second Bad Analysis Law: Bad Analysis is worse than no analysis.
If there is no analysis, people muddle along by common sense which usually works out OK. To really mess things up requires a common direction which requires persuasive analysis pointing in that direction. If that direction happens to be into a swamp, it doesn't help much.

2 comments:

Scott said...

Ed,
Very funny and too true. I often find myself walking a fine line between providing the certainty middle and upper management wants, and caveatting with risks, identified assumptions and unknowns. Too far in either direction and the project is bound for failure or cancellation more often than not.

-Scott Daley
Daley Insight

Anonymous said...

truth is usually counter intuitive :)

http://acc6.its.brooklyn.cuny.edu/~phalsall/texts/taote-v3.html#45