Building the Plane as You Fly It -- How to Implement Pilot Testing in your Improvement Project

By April Resnick

Did you know that birds flying into airplanes cause $1.2b in damage to commercial aircrafts each year? And that, in order to evaluate the results of such a "bird strike" on an aircraft, engine manufacturers employ a test involving throwing dead birds into test engines on the ground with a so-called “chicken gun”– a large-diameter, compressed-air cannon? It's not great news for the chickens, but it should bring some comfort to air travelers, knowing that airlines are going to great lengths to understand what works and what doesn't.

This is just one (pretty gross) example of an element of the Pilot Testing process -- trying to "break" the process in a safe environment and collecting information to understand how to mitigate that risk. In this final post in our Journey Mapping the Personnel Vetting Process blog series, we will cover the key elements of bringing your improvement project to a successful launch...and then what happens next:

Step 6: Design and test prototype solutions

Step 7:  Build an implementation plan and conduct continuous monitoring 

In our last blog post, we focused on identifying and prioritizing the root cause(s) of problems with the process and then began to generate ideas for possible solutions to address those root causes. Using the list of solutions as a starting point, the next step is to match solutions with the highest prioritized root causes. Here are some key considerations as you work through this exercise:

1. Are there any high-priority root causes that do not have any associated solutions? 
2. Is there significant pushback from any group of stakeholders around a particular solution?
3. Are any of the solutions really "umbrella" solutions that may need to be broken down into smaller, easier to implement pieces?
4. Does the list of solutions encompass all "types" of solutions that may be implemented? Some examples include Standard Operating Procedures, Policy, Programming, and Process.
5. You may want to plot the refined list of solutions on a Benefit/Effort matrix to prioritize implementation.

Once your team has a good sense of which solution(s) to launch in what order, it's time to design and implement a Pilot Plan to test the improvements. Important considerations for running an effective Pilot program:

  • To prevent bias, both real or perceived, all actions should be performed by employees that normally perform operations -- not the process improvement team. 
  • The improvement team should always stay close to the Pilot during the scheduled performance period to support operators and receive feedback. 
  • Try to run the Pilot beyond the identified schedule if possible, to provide immediate implementation of some solutions and to deliver longer term changes at the end of the project.

As your Pilot program runs, the improvement team will be able to continuously capture data, both quantitative and qualitative, to understand how the solution is working (and not working!), and this information can be used to tweak the solution "in flight" as needed, refining the process until it consistently delivers the desired result.

Interested in how your team can use Customer Journey Mapping to solve problems and improve your customer experience? Contact a Big Sky Associate today.  

Contact a Big Sky Expert