Learning from Mistakes

I attended a brown bag conference call hosted by the Industrial Research Institute on the topic of "Learn from New Product Failures". Our speaker was Jim Hlavacek, one of the authors of the paper by that name published in Research-Technology-Management. They have had experience in reviewing failed projects at manufacturing companies based on techniques used in the medical profession. In many teaching hospitals, when there is an adverse patient outcome to treatment, the clinical team undergoes a Mortality and Morbidity Conference, where the course of treatment is reviewed by an objective team and mistakes identified.

Hlavacek reported experiences at companies like Intuit, Toyota, and 3M where a regular process of review is built into the engineering/marketing culture.

Under Hlavacek's approch the Failed Product Review (FPR) would consist of the following:

Background

Name of the failed venture

Dates project began and was terminated or shelved

New Venture leader and cross-functional team members

Objective and qualified principal investigators

Inputs

Face-to-face interviews with people who were particpants on the project

Face-to-face interviews with OEM and end-use customers who were involved

Face-to-face interviews with distributors/dealers and/or key suppliers

Obtain all e-mails, business plans, documents, trials, and project presentations

Methodology

Develop timelines and milestones of critical events or decisions

Doucment the unfavorable outcomes with data

Develop fishbone diagrams for the project and processes

Develop root-cause analysis of the fishbone diagrams

Recommendations

What went well for the project

What went wrong for the project
Lessions learned and corrective actions

This approach to learning from mistakes reminded me of some similar approaches I am familiar with. Those of you with a military background may have participated in an After Action Review. This is used primarily during training exercises to understand what happened, evaluate everyones performance, and discuss what could be done better. From the USArmy manual comes a similar outline for an AAR:

Introduction and rules

Review of objectives and intent

Training objectives

Commanders mission/intent

OPFOR commander's mission/intent

Relevant doctrine, tactics, techniques, and procedures

Summary of recent events (what happened)

Discussion of key issues

Chronological order of events

Battlefield operating system

Key events/themes/issues

Discussion of optional issues

Soldier/Leader skills

Tasks to sustain/improve

Statistics

Others

Discussion of force protection (safety)

Closing Comments

The final example is from my experience as a Certified Scrum Master. Under SCRUM a small team produces a product release using a succession of short time boxed miniprojects called Sprints. Sprints typically take anywhere from 2-4 weeks for the team to produce a functional version of the product. After every Sprint the team gets together for a Restrospective Meeting. In this meeting the team discusses the following:

What worked well last Sprint that we should continue doing?

The practices that worked well during the previous Sprint should be identified and continued in the coming Sprint.

What didn’t work well last Sprint that we should stop doing?

The team or customers should identify practices that worked against the team during the last Sprint and focus on stopping those things during the next Sprint.

What should we start doing?

The team identifies practices that should be implemented during the coming Sprint that will help them work better together.

Out of this discussion comes a list of actions that the Scrum Master captures and and is responsible for implementing during the next Sprint.

So lets compare/contrast these three approaches:

1. The FPR is conducted when things go wrong. The AAR and Retrospective occur for all outcomes.
2. Both the FPR and AAR require the participation of one or more objective reviewers. The Retrospective depends on the team and, sometimes, invited guests.
3. The FPR and AAR both use detailed analysis to find root causes. The Retrospectice is more adhoc.
4. The AAR assumes that particpants will change behavior based on issues being surfaced, the FPR has a deliverable of lessons learned but no clear followup for change, and the Retrospective has a set of actions with the Scrum Master responsible for seeing they are implemented immediately.

In my opinion, the key to making any of these techniques work is to have a culture of trust surounding the proceedings where everyone understands that people make mistakes and that for the majority of participants mistakes that are surfaced will not be used to punish them. I say majority, because if the same individual is constantly exposed as making repeated mistakes and not correcting behavior then it may result in termination.

Over my career, I have participated in and led a lot of project/product reviews. Almost always, people are defensive and guarded about what happened. To set the right tone in establishing a review program executives should lead by example, being willing to have their actions reviewed and also demonstrating with their subordinates that mistakes they make are not used to influence annual appraisals.