The Weibull Training Wheels

My

old post, Inside a Lead Time Distribution, remains popular. I wrote it more than five years ago based on what I learned through practice and research 5-7 years ago. People continue to read it, reference it, and ask me questions about interpreting lead time distribution charts. And sometimes they ask me specifically about the Weibull distribution.

The post largely stood the test of time, but some clarifications are in order.

We need to acknowledge the special role Weibull distribution played in the attempts to understand the nature of lead time (time-in-process) in professional services and knowledge work. A good number of empirical lead time data sets matched Weibull reasonably well. And of course there were data sets that didn’t. A simplifying assumption, that the lead time data from yet another service would likely match Weibull, lead us to some discoveries and insights. Then it turned out that we can remove the “Weibull assumption”, but the insights and the practical advice we can derive from are still valid.

Thus Weibull distribution served us as some kind of intellectual crutch or training wheels. As the Kanban method matured through years of worldwide practical application, its guidance related to lead time became proven and could stand on its own. The training wheels became unnecessary.

Weibull helped me understand the practical meaning of statistical hazard functions in knowledge work. Hazard is imply the ratio of two probabilities: the probability that a previously unsolved problem will get solved in the next instant and the probability that the problem will stay unsolved from the beginning until that instant. As a reminder, we’re in business of solving problems of some type collaboratively and delivering solutions to customers who want to know the lead time it will take.

Several possibilities:

• If our hazard function is constant, the lead time will have the exponential distribution
• If our hazard function is decreasing (which can happen due to poor prioritization, implicit classes of service, poorly managed dependencies on unpredictable services — sounds like problems that can actually occur in professional services), the lead time distribution will be sub-exponential. This domain is colloquially known as Extremistan. Nassim Taleb calls this the Lindy effect. A problem that stayed unsolved for so long is likely to stay unsolved even longer. Services with lead time in this domain are wildly unpredictable and likely not fit for their customer’s purpose. The practical advice here is: apply risk reduction and mitigation to “trim the tail” and make the service fitter for purpose.
• If our hazard function is increasing faster than linearly, the lead time distribution will have little variability. All risks in our service are thin-tailed. That’s a signal we’re about to be disrupted. Can you think of something your business should do about it?
• If our hazard function is increasing, but slower than linearly (which can happen thanks to our incorporating customer feedback into our problem-solving activities and thanks to good management of delays and dependencies, keeping them out of Extremistan), then we land in what I called the Domain of Well-Managed Knowledge Work. Geeks can call it Borderline Mediocristan.

Of all distributions in the Weibull family, those with shape parameter 1<k<2 are in this happy well-managed knowledge work middle domain. Weibull k=1.5 provides an example of a distribution from this domain. Weibull with k<1 are in Extremistan and in the “fix-this-shit” domain of unpredictable services. Weibull with k>2 are Mainland Mediocristan, the domain of “you aren’t really doing knowledge work, ripe for disruption.” Weibull k=1 and k=2 mark the boundaries.

Interestingly, we can now remove the “Weibull assumption” (real-world lead time distributions may or may not be Weibull) and the three domains are still there. Any distribution fits in one of the three. And we can now ask, what domain does the signature of your process fit in? And then we say, here’s pragmatic actionable guidance for you, three different ways.

Assuming for the moment (not entirely correctly as you know now) that a lead time distribution is kind of Weibull allowed us to think in simple categories of shape and scale. The shape (with Weibull this literally means the shape parameter) encapsulated all information about the pattern of risks and delays in the service. The scale parameter tells us what time units we got under the horizontal axis.

Now remove the Weibull assumption. The pattern of risks and sources of delay still determines the shape, only now it’s not some number, but the outline of the lead time distribution chart the manager is bringing to the Service Delivery Review. And you still need to know the time scale under the x-axis: how fast does your service deliver in a typical, medium-happy scenario? Is it hours? days? weeks? months? If your primary source of this understanding is your “empirical” data, looking at the 60th to 70th percentile range is a healthy habit. Mathematicians out there can figure out the upper bound of the confidence interval for the estimation of the median lead time. Given the typical sizes of lead time data sets managers in professional services collect and deal with, this upper bound is likely to fall east of the 60th percentile. This is pure coincidence with the 63th-percentile shape-invariant point on the Weibull curve. And this applies to any distribution, Weibull or not.

As you can see, Weibull distribution was training wheels for us several years ago, but these training wheels have by now fallen off. The Kanban method practical advice on managing lead time of professional services stands on its own. We can also understand sources of delay and model them realistically. We can use such models to see the impact of improvement actions. We can then prioritize those actions that make the greatest impact for our customers.