Solid DNA blog

Blog about stuff on Solid Edge CAD software

Making Estimates in an Uncertain World – Part 2 of 2

This material is derived from Making Robust Decisions by David G. Ullman, 2006 Book

In last month’s newsletter, we explored estimation uncertainty and anchoring. In this newsletter, we will explore: how estimation needs are often in conflict with uncertainty, how higher uncertainty is often seen to indicate less knowledge, how optimistic hindsight can cloud estimates, the concepts of common and special cause uncertainty, and the Law of Designed Behavior.

· Estimation needs are often in conflict with estimation accuracy. You can always improve your accuracy by estimating wider intervals. For example, an estimate that claims “I am 90 percent sure I can do the dishes in the interval between 5 and 50 minutes” has a high probability of being correct. However, this kind of overly wide interval is so broad that it conveys nothing useful and is in conflict with management’s desire to have when planning. Management, ideally, would like a single value. The following incident illustrates both uncertainty and our human unwillingness to accept it. Some military officers were assigned the task of forecasting the weather a month ahead, but they found that their long range forecasts were of low fidelity; in fact, they were no better than numbers pulled out of a hat. Since their estimates were useless, the forecasters asked their superiors to be relieved of this duty. The reply was: “The Commanding General is well aware that the forecasts are no good. However, he needs them for planning purposes.” There are proven methods to improve the accuracy and specificity of estimates…read on.

· Another result of uncertainty is that since higher uncertainty is seen to indicate less knowledge, it is often not shared with others. We are not willing to show our ignorance because it implies that we don’t know enough to do our jobs and we often hide uncertainty instead of disclosing it for further discussion and investigation. But hiding uncertainty often results in problems later. In order to make a robust decision, uncertainty must be made visible in a non-threatening environment.

· Hindsight usually paints a picture that is rosier than reality. We tend to forget our bad experiences or assume that they won’t occur again. This leads to overconfidence in our ability to complete projects within the time and cost we estimate and with the features we think we can achieve.

Beyond being optimistic, most hindsight is just plain wrong. In one experiment, a group was asked two weeks after a meeting to recall specific details of it. In recounting the meeting they:

  • Omitted 90 percent of the specific points that were discussed.
  • Recalled half of what they did remember incorrectly.
  • Remembered comments that were not made.
  • Transformed casual remarks into lengthy orations.
  • Converted implicit meanings into explicit comments.

Capturing an accurate log (or audit trail) of how a decision was reached can ensure a better outcome in the decision process.

· The terms common cause” and “special cause” differentiate between uncertainties that can be controlled and those that can’t. This clarification was developed by W. Edwards Deming, one of the fathers of modern product quality. Common cause uncertainty results from chance causes in the system. Common causes are the noises that cannot easily be controlled. For example, it might take you five minutes to process certain paperwork one day, and six minutes the next. Or, the car you design might sometimes be driven by a teenager and a little old lady at other times. Resources put into managing common cause uncertainty are generally wasted. Instead, you should make decisions should be made that are as insensitive as possible to common causes.

Special cause uncertainty results from an assignable cause. Special causes are generally controllable—if you know about them. There are really two types of special causes, known-unknowns and unknown-unknowns (commonly called “unk-unks”). The known-unknowns, such as the possibility of bad weather causing shipping delays, you can plan for. The unk-unks—a change in direction by upper management, or a competitor developing a new technology—are more difficult to foresee and are directly tied to a lack of knowledge about the future.

Each type of uncertainty may have both common and special parts. The challenge is to tease these apart. In general, you can’t control common cause, and attempting to control it is a mistake. Since common cause is just chance effects in the system, it will be expensive or impossible to control, and not possible to eliminate. You can control special causes to some degree if you expend enough resources on doing so and build in enough buffers to reduce their impact. You must determine the cost/benefit of doing so before you commit resources to try to control special causes.

· Finally, many organizations repeat the same pattern over and over, which leads to an important point: If an organization repeatedly shows the same cost/scope behavior, it is designed to create that behavior. Put another way, most organizational structures are unintentionally designed to produce missed estimates. At first read this may sound very harsh and perhaps even untrue, but the conclusion, while difficult to admit, is inevitable. We call this the Law of Designed Behavior.

So, what can you do to get the best possible estimates? Here are some guidelines (numbering starts at 6 as the first five were in last month’s newsletter):

6. While it is true that uncertainty implies low knowledge, this is not bad, but a signal that works needs to be done or relationships need to be built. Don’t kill the messenger.

7. Base estimates on documented historical information whenever possible to offset optimistic hindsight. Documentation should include scope, initial estimates, actual results and what caused deviation. Very few organizations do this in any formal way.

8. Estimation methods need to become part of the organization culture and be considered a core competency.

9. Collect the reasons (causes) behind the special cause uncertainty and base planning on these reasons. Put no effort into overcoming common cause uncertainty.

10. Work to understand your organization’s behavior toward estimates. Only by understanding can change begin.

This material is derived from Making Robust Decisions by David G. Ullman, 2006 Book

Second part of this newsletter shows that wider estimate to narrow down to a single number (factor) $$$ to hide uncertainty will result in problem later.

When it comes time to choose CAD software, if noise is introduce to make Solid Edge with Synchronous Technology irrelevant to force you in their 3D path, wipe out the noise with

Evolve to 3D

Hope this introduction will help you make Robust decision . I also hope that I fulfill part of my responsibility in regard of the CAD community event if I am pro-Solid Edge.

Stay tune for some video, next one will be about auto dimension.

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: