Posts Tagged ‘Understanding Physics’

Seeing Things as They Can’t Be

When there’s a big problem, the first step is to define what’s causing it. To do that, based on an understanding of the physics, a sequence of events is proposed and then tested to see if it replicates the problem. In that way, the team must understand the system as it is before the problem can be solved.

Seeing things as they are. The same logic applies when it’s time to improve an existing product or service. The first thing to do is to see the system as it is. But seeing things as they are is difficult. We have a tendency to see things as we want them or to see them in ways that make us look good (or smart). Or, we see them in a way that justifies the improvements we already know we want to make.

To battle our biases and see things as they are, we use tools such as block diagrams to define the system as it is. The most important element of the block diagram is clarity.  The first revision will be incorrect, but it must be clear and explicit. It must describe things in a way that creates a singular understanding of the system. The best block diagrams can be interpreted only one way.  More strongly, if there’s ambiguity or lack of clarity, the thing has not yet risen to the level of a block diagram.

The block diagram evolves as the team converges on a single understanding of things as they are. And with a diagram of things as they are, a solution is readily defined and validated. If when tested the proposed solution makes the problem go away, it’s inferred that the team sees things as they are and the solution takes advantage of that understanding to make the problem go away.

Seeing things as they may be. Even whey the solution fixes the problem, the team really doesn’t know if they see things as they are. Really, all they know is they see things as they may be. Sure, the solution makes the problem go away, but it’s impossible to really know if the solution captures the physics of failure.  When the system is large and has a lot of moving parts, the team cannot see things as they are, rather, they can only see the system as it may be. This is especially true if the system involves people, as people behave differently based on how they feel and what happened to them yesterday.

There’s inherent uncertainty when working with larger systems and systems that involve people.  It’s not insurmountable, but you’ve got to acknowledge that your understanding of the system is less than perfect. If your company is used to solving small problems within small systems, there will be little tolerance for the inherent uncertainty and associated unpredictability (in time) of a solution.  To help your company make the transition, replace the language of “seeing things as they are” with “seeing things as they may be.”  The same diagnostic process applies, but since the understanding of the system is incomplete or wrong, the proposed solutions cannot not be pre-judged as “this will work” and “that won’t work.”  You’ve got to be open to all potential solutions that don’t contradict the system as it may be. And you’ve got to be tolerant of the inherent unpredictability of the effort as a whole.

Seeing things as they could be. To create something that doesn’t yet exist, something does things like never before, something altogether new, you’ve got to stand on top of your understanding of the system and jump off.  Whether you see things as they are or as they may be, the new system will be different. It’s not about diagnosing the existing system; it’s about imagining the system as it could be. And there’s a paradox here. The better you understand the existing system, the more difficulty you’ll have imagining the new one. And, the more success the company has had with the system as it is, the more resistance you’ll feel when you try to make the system something it could be.

Seeing things as they could be takes courage – courage to obsolete your best work and courage to divest from success. The first one must be overcome first. Your body creates stress around the notion of making yourself look bad. If you can create something altogether better, why didn’t you do it last time? There’s a hit to the ego around making your best work look like it’s not all that good. But once you get over all that, you’ve earned the right to go to battle with your organization who is afraid to move away from the recipe responsible for all the profits generated over the last decade.

But don’t look at those fears as bad. Rather, look at them as indicators you’re working on something that could make a real difference.  Your ego recognizes you’re working on something better and it sends fear into your veins. The organization recognizes you’re working on something that threatens the status quo and it does what it can to make you stop. You’re onto something. Keep going.

Seeing things as they can’t be. This is rarified air. In this domain you must violate first principles. In this domain you’ve got to run experiments that everyone thinks are unreasonable, if not ill-informed. You must do the opposite. If your product is fast, your prototype must be the slowest. If the existing one is the heaviest, you must make the lightest. If your reputation is based on the highest functioning products, the new offering must do far less.  If your offering requires trained operators, the new one must prevent operator involvement.

If your most seasoned Principal Engineer thinks it’s a good idea, you’re doing it wrong. You’ve got to propose an idea that makes the most experienced people throw something at you. You’ve got to suggest something so crazy they start foaming at the mouth. Your concepts must rip out their fillings. Where “seeing things as they could be” creates some organizational stress, “seeing things as they can’t be” creates earthquakes. If you’re not prepared to be fired, this is not the domain for you.

All four of these domains are valuable and have merit. And we need them all. If there’s one message it’s be clear which domain you’re working in. And if there’s a second message it’s explain to company leadership which domain you’re working in and set expectations on the level of uncertainty and unpredictability of that domain.

Image credit – David Blackwell.

Wanting things to be different

Wanting things to be different is a good start, but it’s not enough. To create conditions for things to move in a new direction, you’ve got to change your behavior. But with systems that involve people, this is not a straightforward process.
To create conditions for the system to change, you must understand the system”s disposition – the lines along which it prefers to change.. And to do that, you’ve got to push on the system and watch its response. With people systems, the response is not knowable before the experiment.
If you expect to be able to predict how the system will respond, working with people systems can be frustrating.  I offer some guidance here. With this work, you are not responsible for the system’s response, you are only responsible for how you respond to the system’s response.
If the system responds in a way you like, turn that experiment into a project to amplify the change.  If the system responds in a way you dislike, unwind the experiment.  Here’s a simple mantra – do more of what works and less of what doesn’t. (Thanks to Dave Snowden for this.)
If you don’t like how things are going, you have only one lever to pull.  You can only change.your response to what you see and experience. You can respond by pushing on the system and responding to what you see or you can respond by changing what you think and feel about the system.
But keep in mind that you are part of the system. And maybe the system is running an experiment on you. Either way, your only choice is to choose how to respond.

Read the rest of this entry »

The Courage To Speak Up

If you see things differently than others, congratulations.  You’re thinking for yourself.

If you find yourself pressured into thinking like everyone else, that’s a sign your opinion threatens. It’s too powerful to be dismissed out-of-hand, and that’s why they want to shut you up.

If the status quo is angered by your theory, you’re likely onto something.  Stick to your guns.

If your boss doesn’t want to hear your contrarian opinion, that’s because it cannot be easily dismissed. That’s reason enough to say it.

If you disagree in a meeting and your sentiment is actively dismissed, dismiss the dismisser. And say it again.

If you’re an active member of the project and you are not invited to the meeting, take it as a compliment. Your opinion is too powerful to defend against. The only way for the group-think to survive is to keep you away from it. Well done.

If your opinion is actively and repeatedly ignored, it’s too powerful to be acknowledged.  Send a note to someone higher up in the organization.  And if that doesn’t work, send it up a level higher still. Don’t back down.

If you look into the future and see a train wreck, set up a meeting with the conductor and tell them what you see.

When you see things differently, others will try to silence you and tell you you’re wrong. Don’t believe them.  The world needs people like you who see things as they are and have the courage to speak the truth as they see it.

Thank you for your courage.

Image credit – Cristian V.

Ask for the right work product and the rest will take care of itself.

We think we have more control than we really have.  We imagine an idealized future state and try desperately to push the organization in the direction of our imagination.  Add emotional energy, define a rational approach, provide the supporting rationale and everyone will see the light. Pure hubris.

What if we took a different approach? What if we believed people want to do the right thing but there’s something in the way?  What if like a log jam in a fast-moving river, we remove the one log blocking them all? What if like a river there’s a fast-moving current of company culture that wants to push through the emotional log jam that is the status quo?  What if it’s not a log at all but, rather, a Peter Principled executive that’s threatened by the very thing that will save the company?

The Peter Principled executive is a tough nut to crack. Deeply entrenched in the powerful goings on of the mundane and enabled by the protective badge of seniority, these sticks-in-the-mud need to be helped out of the way without threatening their no-longer-deserved status. Tricky business.

 

Rule 1: If you get into an argument with a Peter Principled executive, you’ll lose.

Rule 2: Don’t argue with Peter Principled executive.

 

If we want to make it easy for the right work to happen, we’ve got to learn how to make it easy for the Peter Principled executive to get out of the way.  First, ask yourself why the executive is in the way. Why are they blocking progress? What’s keeping them from doing the right thing? Usually it comes down to the fear of change or the fear of losing control.  Now it’s time to think of a work product that will help make the case there’s a a better way. Think of a small experiment to demonstrate a new way is possible and then run the experiment. Don’t ask, just run it. But the experiment isn’t the work product. The work product is a short report that makes it clear the new paradigm has been demonstrated, at least at small scale.  The report must be clear and dense and provide objective evidence the right work happened by the right people in the right way.  It must be written in a way that preempts argument – this is what happened, this is who did it, this is what it looks like and this is the benefit.

It’s critical to choose the right people to run the experiment and create the work product. The work must be done by someone in the chain of command of the in-the-way executive.  Once the work product is created, it must be shared with an executive of equal status who is by definition outside the chain of command.  From there, that executive must send a gracious email back into the chain of command that praises the work, praises the people who did it and praises the leader within the chain of command who had the foresight to sponsor such wonderful work.

As this public positivity filters through the organization, more people will add their praise of the work and the leaders that sponsored it.  And by the time it makes it up the food chain to the executive of interest, the spider web of positivity is anchored across the organization and can’t be unwound by argument. And there you have it. You created the causes and conditions for the log jam to unjam itself.  It’s now easy for the executive to get out of the way because they and their organization have already been praised for demonstrating the new paradigm.  You’ve built a bridge across the emotional divide and made it easy for the executive and the status quo to cross it.

Asking for the right work product is a powerful skill. Most error on the side of complication and complexity, but the right work product is just the opposite – simple and tight. Think sledgehammer to the forehead in the form of and Excel chart where the approach is beyond reproach; where the chart can be interpreted just one way; where the axes are labeled; and it’s clear the status quo is long dead.

Business model is dead and we’ve got to stop trying to keep it alive. It’s time to break the log jam. Don’t be afraid. Create the right work product that is the dynamite that blows up the status quo and the executives clinging to it.

Image credit – Emilio Küffer

Is your tank empty?

Sometimes your energy level runs low.  That’s not a bad thing, it’s just how things go. Just like a car’s gas tank runs low, our gas tanks, both physical and emotional, also need filling.  Again, not a bad thing. That’s what gas tanks are for – they hold the fuel.

We’re pretty good at remembering that a car’s tank is finite.  At the start of the morning commute, the car’s fuel gauge gives a clear reading of the fuel level and we do the calculation to determine if we can make it or we need to stop for fuel.  And we do the same thing in the evening – look at the gauge, determine if we need fuel and act accordingly.  Rarely we run the car out of fuel because the car continuously monitors and displays the fuel level and we know there are consequences if we run out of fuel.

We’re not so good at remembering our personal tanks are finite. At the start of the day, there are no objective fuel gauges to display our internal fuel levels.  The only calculation we make – if we can make it out of bed we have enough fuel for the day. We need to do better than that.

Our bodies do have fuel gages of sorts.  When our fuel is low we can be irritable, we can have poor concentration, we can be easily distracted.  Though these gages are challenging to see and difficult to interpret, they can be used effectively if we slow down and be in our bodies.  The most troubling part has nothing to do with our internal fuel gages.  Most troubling is we fail to respect their low fuel warnings even when we do recognize them.  It’s like we don’t acknowledge our tanks are finite.

We don’t think our cars are flawed because their fuel tanks run low as we drive.  Yet, we see the finite nature of our internal fuel tanks as a sign of weakness. Why is that? Rationally, we know all fuel tanks are finite and their fuel level drops with activity. But, in the moment, when are tanks are low, we think something is wrong with us, we think we’re not whole, we think less of ourselves.

When your tank is low, don’t curse, don’t blame, don’t feel sorry and don’t judge.  It’s okay.  That’s what tanks do.

A simple rule for all empty tanks – put fuel in them.

Image credit – Hamed Saber

How to Choose the Best Idea

We have too many ideas, but too few great ones.  We don’t need more ideas, we need a way to choose the best one or two ideas and run them to ground.

Before creating more ideas, make a list of the ones you already have.  Put them in two boxes.  In Box 1, list the ideas without a video of a functional prototype in action.  In Box 2, list the ideas that have a video showing a functional prototype demonstrating the idea in action.  For those ideas with a functional prototype and no video, put them in Box 1.

Next, throw away Box 1. If it’s not important enough to make a crude physical prototype and create a simple video, the idea isn’t worth a damn.  If someone isn’t willing to carve out the time to make a physical prototype, there’s no emotional energy behind the idea and it should be left to die.  And when people complain that it’s unfair to throw away all those good ideas in Box 1, tell them it’s unfair to spend valuable resources talking about ideas that aren’t worthy.  And suggest, if they want to have a discussion about an idea, they should build a physical prototype and send you the video.  Box 2, or bust.

Next, get the band together and watch the short videos in Box 2, and, as a group, put them in two boxes.  In Box 3, put the videos without customers actively using the functional prototype.  In Box 4, put the videos with customers actively using the functional prototype.

Next, throw way Box 3.  If it’s not important enough to make a trip to an important customer and create a short video, the idea isn’t worth a damn.  If you’re not willing to put yourself out there and take the idea to an important customer, the idea is all fizzle and no sizzle.  Meaningful ideas take immense personal energy to run through the gauntlet, and without a video of a customer using the functional prototype, there’s not enough energy behind it.  And when everyone argues that Box 3 ideas are worth pursuing, tell them to pursue a video showing a most important customer demonstrating the functional prototype.

Next, get the band back together to watch the Box 4 videos.  Again, put the videos in two boxes. In Box 5 put the videos where the customer didn’t say what they liked and how they’d use it.  In Box 6, put the videos where the customer enthusiastically said what they liked and how they’ll use it.

Next, throw away Box 5.  If the customer doesn’t think enough about the prototype to tell you how they’ll use it, it’s because they don’t think much of the idea.  And when the group says the customer is wrong or the customer doesn’t understand what the prototype is all about, suggest they create a video where a customer enthusiastically explains how they’d use it.

Next, get the band back in the room and watch the Box 6 videos.  Put them in two boxes.  In Box 7, put the videos that won’t radically grow the top line.  In Box 8, put the videos that will radically grow the top line.  Throw away Box 7.

For the videos in Box 8, rank them by the amount of top line growth they will create.  Put all the videos back into Box 8, except the video that will create the most top line growth.  Do NOT throw away Box 8.

The video in your hand IS your company’s best idea.  Immediately charter a project to commercialize the idea.  Staff it fully.  Add resources until adding resources doesn’t no longer pulls in the launch.  Only after the project is fully staffed do you put your hand back into Box 8 to select the next best idea.

Continually evaluate Boxes 1 through 8.  Continually throw out the boxes without the right videos.  Continually choose the best idea from Box 8.  And continually staff the projects fully, or don’t start them.

Image credit – joiseyshowaa

How to Avoid a Cliff

Much like living organisms continually evolve to secure their place in the future, technological systems can be thought to display similar evolutionary behavior.  Viruses mutate so some of them can defeat the countermeasures of their host and live to fight another day. Technological systems, as an expression of a company’s desire to survive, evolve to defeat the competition and live to pay another dividend.

There are natural limits to evolutionary success in any single direction.  When one trait is improved it pushes on the natural limits imposed by the environment.  For example, a bacterium let loose in a friendly Petri dish will replicate until it eats all the food in the dish. Or, on a longer timescale, if the mass of a bird increases over generations when its food source is plentiful, the bird will get larger but will also get less agile. The predators who couldn’t catch the fast, little bird of old can easily catch and eat the sluggish heavyweight. In that way, there’s an edge condition created by the environmental Petri dishes and predators.  And it’s the same with technological systems.

Companies and their technological systems evolve within their competitive environment by scanning the fitness landscape and deciding where to try to improve.  The idea is to see preferential lines of improvement and create new technologies to take advantage of them.  Like their smaller biological counterparts, companies are minimum energy creatures and want to maximize reward (profit) with minimum effort (expense) and will continue to leverage successful lines of evolution until it senses diminishing returns.

The diminishing returns are a warning sign that the company is approaching an edge condition (a Petri dish of a finite size). In landscape lingo, there’s a cliff on the horizon. In technology lingo, the rate of improvement of the technology is slowing.  In either language, the edge is near and it’s time to evolve in a new direction because this current one is out of gas.

Like the bird whose mass increases over the generations when food is readily available, companies also get fat and slow when they successfully evolve in a single direction for too long.  And like the bird, they get eaten by a more agile competitor/predator. And just as the replication rate of the bacterium accelerates as the food in the Petri dish approaches zero, a company that doesn’t react to a slowing rate of technological improvement is sure to outlive its business model.

Biology and technology are similar in that they try new things (create variants of themselves) in order to live another day.  But there’s a big difference – where biology is blind (it doesn’t know what will work and what won’t), technology is sighted (people that create use their understanding to choose the variants they think will work best).  And another difference is that biological evolution can build only on viable variants where technology can use mental models as scaffolds to skip non-viable embodiments to cross a chasm.

There’s no need to fall off the cliff.  As a leading indicator, monitor the rate of improvement of your technology.  If its rate of improvement is still accelerating, it’s time to develop the next line of evolution. If its rate is declining, you waited too long. It’s time to double down on two new lines of evolution because you’re behind the curve. And remember, like with the population of bacteria in the Petri dish, sales will keep growing right up until the business model runs out of food or a competitor eats you.

Image credit — Amanda

With novelty, less can be more.

When it’s time to create something new, most people try to imagine the future and then put a plan together to make it happen.  There’s lots of talk about the idealize future state, cries for a clean slate design or an edict for a greenfield solution.  Truth is, that’s a recipe for disaster. Truth is, there is no such thing as a clean slate or green field. And because there are an infinite number of future states, it’s highly improbable your idealized future state is the one the universe will choose to make real.

To create something new, don’t look to the future. Instead, sit in the present and understand the system as it is. Define the major elements and what they do.  Define connections among the elements.  Create a functional diagram using blocks for the major elements, using a noun to name each block, and use arrows to define the interactions between the elements, using a verb to label each arrow. This sounds like a complete waste of time because it’s assumed that everyone knows how the current state system behaves.  The system has been the backbone of our success, of course everyone knows the inputs, the outputs, who does what and why they do it.

I have created countless functional models of as-is systems and never has everyone agreed on how it works.  More strongly, most of the time the group of experts can’t even create a complete model of the as-is system without doing some digging. And even after three iterations of the model, some think it’s complete, some think it’s incomplete and others think it’s wrong. And, sometimes, the team must run experiments to determine how things work.  How can you imagine an idealized future state when you don’t understand the system as it is?  The short answer – you can’t.

And once there’s a common understanding of the system as it is, if there’s a call for a clean sheet design, run away.  A call for a clean sheet design is sure fire sign that company leadership doesn’t know what they’re doing.  When creating something new it’s best to inject the minimum level of novelty and reuse the rest (of the system as it is).  If you can get away with 1% novelty and 99% reuse, do it.  Novelty, by definition, hasn’t been done before. And things that have never been done before don’t happen quickly, if they happen at all. There’s no extra credit for maximizing novelty. Think of novelty like ghost pepper sauce – a little goes a long way.  If you want to know how to handle novelty, imagine a clean sheet design and do the opposite.

Greenfield designs should be avoided like the plague. The existing system has coevolved with its end users so that the system satisfies the right needs, the users know how to use the system and they know what to expect from it.  In a hand-in-glove way, the as-is system is comfortable for end users because it fits them.  And that’s a big deal.  Any deviation from baseline design (novelty) will create discomfort and stress for end users, even if that novelty is responsible for the enhancement you’re trying to deliver.  Novelty violates customer expectations and violating customer expectations is a dangerous game. Again, when you think novelty, think ghost peppers. If you want to know how to handle novelty, imagine a green field and do the opposite.

This approach is not incrementalism.  Where you need novelty, inject it.  And where you don’t need it, reuse. Design the system to maximize new value but do it with minimum novelty.  Or, better still, offer less with far less. Think 90% of the value with 10% of the cost.

Image credit – Laurie Rantala

What’s your problem?

If you don’t have a problem, you’ve got a big problem.

It’s important to know where a problem happens, but also when it happens.

Solutions are 90% defining and the other half is solving.

To solve a problem, you’ve got to understand things as they are.

Before you start solving a new problem, solve the one you have now.

It’s good to solve your problems, but it’s better to solve you customers’ problems.

Opportunities are problems in sheep’s clothing.

There’s nothing worse than solving the wrong problem – all the cost with none of the solution.

When you’re stumped by a problem, make it worse then do the opposite.

With problem definition, error on the side of clarity.

All problems are business problems, unless you care about society’s problems.

Odds are, your problem has been solved by someone else.  Your real problem is to find them.

Define your problem as narrowly as possible, but no narrower.

Problems are not a sign of weakness.

Before adding something to solve the problem, try removing something.

If your problem involves more than two things, you have more than one problem.

The problem you think you have is never the problem you actually have.

Problems can be solved before, during or after they happen and the solutions are different.

Start with the biggest problem, otherwise you’re only getting ready to solve the biggest problem.

If you can’t draw a closeup sketch of the problem, you don’t understand it well enough.

If you have an itchy backside and you scratch you head, you still have an itch. And it’s the same with problems.

If innovation is all about problem solving and problem solving is all about problem definition, well, there you have it.

Image credit – peasap

If you want to learn, define a learning objective.

Innovation is all about learning.  And if the objective of innovation is learning, why not start with learning objectives?

Here’s a recipe for learning: define what you want to learn, figure how you want to learn, define what you’ll measure, work the learning plan, define what you learned and repeat.

With innovation, the learning is usually around what customers/users want, what new things (or processes) must be created to satisfy their needs and how to deliver the useful novelty to them.  Seems pretty straightforward, until you realize the three elements interact vigorously.  Customers’ wants change after you show them the new things you created. The constraints around how you can deliver the useful novelty (new product or service) limit the novelty you can create.  And if the customers don’t like the novelty you can create, well, don’t bother delivering it because they won’t buy it.

And that’s why it’s almost impossible to develop a formal innovation process with a firm sequence of operations.  Turns out, in reality the actual process looks more like a fur ball than a flow chart. With incomplete knowledge of the customer, you’ve got to define the target customer, knowing full-well you don’t have it right. And at the same time, and, again, with incomplete knowledge, you’ve got to assume you understand their problems and figure out how to solve them.  And at the same time, you’ve got to understand the limitations of the commercialization engine and decide which parts can be reused and which parts must be blown up and replaced with something new.  All three explore their domains like the proverbial drunken sailor, bumping into lampposts, tripping over curbs and stumbling over each other.  And with each iteration, they become less drunk.

If you create an innovation process that defines all the if-then statements, it’s too complicated to be useful.  And, because the if-thens are rearward-looking, they don’t apply the current project because every innovation project is different. (If it’s the same as last time, it’s not innovation.)  And if you step up the ladder of abstraction and write the process at a high level, the process steps are vague, poorly-defined and less than useful.  What’s a drunken sailor to do?

Define the learning objectives, define the learning plan, define what you’ll measure, execute the learning plan, define what you learned and repeat.

When the objective is learning, start with the learning objectives.

Image credit Jean L.

Don’t trust your gut, run the test.

At first glance, it seems easy to run a good test, but nothing can be further from the truth.

The first step is to define the idea/concept you want to validate or invalidate.  The best way is to complete one of these two sentences: I want to learn that [type your idea here] is true. Or, I want to learn that [enter your idea here] is false.

Next, ask yourself this question: What information do I need to validate (or invalidate) [type your idea here]?  Write down the information you need.  In the engineering domain, this is straightforward: I need the temperature of this, the pressure of that, the force generated on part xyz or the time (in seconds) before the system catches fire.  But for people-related ideas, things aren’t so straightforward. Some things you may want to know are: how much will you pay for this new thing, how many will you buy, on a scale of 1 to 5 how much do you like it?

Now the tough part – how will you judge pass or fail?  What is the maximum acceptable temperature? What is the minimum pressure? What is the maximum force that can be tolerated? How many seconds must the system survive before catching fire?  And for people: What is the minimum price that can support a viable business? How many must they buy before the company can prosper? And if they like it at level 3, it’s a go. And here’s the most importance sentence of the entire post:

The decision criteria must be defined BEFORE running the test.

If you wait to define the go/no-go criteria until after you run the test and review the data, you’ll adjust the decision criteria so you make the decision you wanted to make before running the test. If you’re not going to define the decision criteria before running the test, don’t bother running the test and follow your gut.  Your decision will be a bad one, but at least you’ll save the time and money associated with the test.

And before running the test, define the test protocol.  Think recipe in a cookbook: a pinch of this, a quart of that, mix it together and bake at 350 degrees Fahrenheit for 40 minutes.  The best protocols are simple and clear and result in the same sequence of events regardless of who runs the test.  And make sure the measurement method is part of the protocol – use this thermocouple, use that pressure gauge, use the script to ask the questions about price and the number they’d buy.

And even with all this rigor, good judgement is still part of the equation.  But the judgment is limited to questions like: did we follow the protocol? Did the measurement system function properly? Do the initial assumptions still hold? Did anything change since we defined the learning objective and defined the test protocol?

To create formal learning objectives, to write well-defined test protocols and to formalize the decision criteria before running the test require rigor, discipline, time and money.  But, because the cost of making a bad decision is so high, the cost of running good tests is a bargain at twice the price.

Image credit – NASA Goddard Flight Center

Mike Shipulski Mike Shipulski
Subscribe via Email

Enter your email address:

Delivered by FeedBurner

Archives