Robust Design Prof. Dan Frey Mechanical Engineering and Engineering Systems 16.888 – Multidisciplinary System Design Optimization Control Factor Response Plan for the Session ? Basic concepts in probability and statistics ? Review design of experiments ? Basics of Robust Design ? Research topics – Model-based assessment of RD methods – Faster computer-based robust design – Robust invention Ball and Ramp Ball Ramp Funnel Response = the time the ball remains in the funnel Causes of experimental error = ? Probability Measure ? Axioms – For any event A, – P(U)=1 – If the intersection of A and B=I, then P(A+B)=P(A)+P(B) 0)( tAP Continuous Random Variables ? Can take values anywhere within continuous ranges ? Probability density function – – – xxf x allfor)(0 d 1d)( 3 f f xxf x ^`xxfbxaP b a x d)( 3 d x f x (x) a b Histograms ? A graph of continuous data ? Approximates a pdf in the limit of large n 0 5 Histogram of Crankpin Diameters Diameter, Pin #1 Frequency Measures of Central Tendency ? Expected value ? Mean P= E(x) ? Arithmetic average 3 S x xxfxgxgE d)()())(( | n i i x n 1 1 Measures of Dispersion ?Variance ? Standard deviation ? Sample variance ? n th central moment ? n th moment about m )))((( 2 xExE  V )))((()( 22 xExExVAR  V |    n i i xx n S 1 22 )( 1 1 )))((( n xExE  ))(( n mxE  Sums of Random Variables ? Average of the sum is the sum of the average (regardless of distribution and independence) ? Variance also sums iff independent ? This is the origin of the RSS rule – Beware of the independence restriction! )()()( yExEyxE   222 )()()( yxyx VVV   Concept Test ? A bracket holds a component as shown. The dimensions are independent random variables with standard deviations as noted. Approximately what is the standard deviation of the gap? A) 0.011” B) 0.01” C) 0.001” "001.0 V "01.0 V gap Expectation Shift x y(x) E(x) y(E(x)) E(y(x)) S f x (x)f y (y(x)) S=E(y(x))- y(E(x)) Under utility theory, S is the only difference between probabilistic and deterministic design Probability Distribution of Sums ? If z is the sum of two random variables x and y ? Then the probability density function of z can be computed by convolution yxz  3 f  z z yzxzp ]]] d)()()( Convolution 3 f  z z yzxzp ]]] d)()()( Convolution 3 f  z z yzxzp ]]] d)()()( Central Limit Theorem The mean of a sequence of n iid random variables with – Finite P – approximates a normal distribution in the limit of a large n. 0<)( 2 !f  G G ii xExE Normal Distribution 2 2 2 )( 2 1 )( V P SV   x x exf P 99.7% 68.3% 1-2ppb +6V-6V +3V +1V -1V-3V Engineering Tolerances ? Tolerance --The total amount by which a specified dimension is permitted to vary (ANSI Y14.5M) ? Every component within spec adds to the yield (Y) q p(q) L U Y y y 18 Process Capability Indices ? Process Capability Index ? Bias factor ? Performance Index C UL p {  /2 3V CC k pk p {()1 k UL UL {    P 2 2()/ q p(q) L U UL 2 UL 2 Concept Test ? Motorola’s “6 sigma” programs suggest that we should strive for a C p of 2.0. If this is achieved but the mean is off target so that k=0.5, estimate the process yield. Plan for the Session ? Basic concepts in probability and statistics ? Review design of experiments ? Basics of Robust Design ? Research topics – Model-based assessment of RD methods – Faster computer-based robust design – Robust invention Pop Quiz ? Assume we wish to estimate the effect of ball position on the ramp on swirl time. The experimental error causes V = 1 sec in the response. We run the experiment 4 times. What is the error our estimate of swirl time? A) V = 1 sec B) V = 1/2 sec C) V = 1/4 sec Ball Ramp Funnel History of DoE ? 1926 – R. A. Fisher introduced the idea of factorial design ? 1950-70 – Response surface methods ? 1987 – G. Taguchi, System of Experimental Design Full Factorial Design ? This is the 2 4 ? All main effects and interactions can be resolved ? Scales very poorly with number of factors -1-1-1-1 +1-1-1-1 -1+1-1-1 +1+1-1-1 -1-1+1-1 +1-1+1-1 -1+1+1-1 +1+1+1-1 -1-1-1+1 +1-1-1+1 -1+1-1+1 +1+1-1+1 -1-1+1+1 +1-1+1+1 -1+1+1+1 +1+1+1+1 ResponseDCBA -1-1-1-1 +1-1-1-1 -1+1-1-1 +1+1-1-1 -1-1+1-1 +1-1+1-1 -1+1+1-1 +1+1+1-1 -1-1-1+1 +1-1-1+1 -1+1-1+1 +1+1-1+1 -1-1+1+1 +1-1+1+1 -1+1+1+1 +1+1+1+1 ResponseDCBA Replication and Precision “the same precision as if the whole … had been devoted to one single component” – Fisher The average of trials 1 through 8 has a V  of 1/8 that of each trial Resolution and Aliasing Trial A B C D E F G 1 -1 -1 -1 -1 -1 -1 -1 2 -1 -1 -1 +1 +1 +1 +1 3 -1 +1 +1 -1 -1 +1 +1 4 -1 +1 +1 +1 +1 -1 -1 5 +1 -1 +1 -1 +1 -1 +1 6 +1 -1 +1 +1 -1 +1 -1 7 +1 +1 -1 -1 +1 +1 -1 8 +1 +1 -1 +1 -1 -1 +1 2 7-4 Design (aka “orthogonal array L8”) Resolution III. FG=-A +1 +1 +1 +1 -1 -1 -1 -1 Projective Property A B C + - + + - - Considered important for exploiting sparsity of effects. DOE – Key Assumptions ? Pure experimental error error in observations is random & independent ? Hierarchy lower order effects are more likely to be significant than higher order effects ? Sparsity of effects there are few important effects ? Effect heredity for an interaction to be significant, at least one parent should be significant Sparsity of Effects ? An experimenter may list several factors ? They usually affect the response to greatly varying degrees ? The drop off is surprisingly steep (~1/n 2 ) ? Not sparse if prior knowledge is used or if factors are screened 0 0.2 0.4 0.6 0.8 1 1.2 1234567 Pareto ordered factors Factor effects Hierarchy ? Main effects are usually more important than two- factor interactions ? Two-way interactions are usually more important than three-factor interactions ?And so on ? Taylor’s series seems to support the idea A B C AB AC BC D AD BD CD ABC ABD BCDACD ABCD ! )( )( )( 0 n af ax n n n | f  Inheritance ? Two-factor interactions are most likely when both participating factors (parents?) are strong ? Two-way interactions are least likely when neither parent is strong ? And so on A B C AB AC BC D AD BD CD ABC ABD BCD ACD ABCD Resolution ? II Main effects are aliased with main effects ? III Main effects are clear of other main effects but aliased with two-factor interactions ? IV Main effects are clear of other main effects and clear of two-factor interactions but main effects are aliased with three-factor interactions and two-factor interactions are aliased with other two-factor interactions ? V Two-factor interactions are clear of other two-factor interactions but are aliased with three factor interactions… Discussion Point ? What are the four most important factors affecting swirl time? ? If you want to have sparsity of effects and hierarchy, how would you formulate the variables? Important Concepts in DOE ? Resolution – the ability of an experiment to provide estimates of effects that are clear of other effects ? Sparsity of Effects – factor effects are few ? Hierarchy – interactions are generally less significant than main effects ? Inheritance – if an interaction is significant, at least one of its “parents” is usually significant ? Efficiency – ability of an experiment to estimate effects with small error variance Plan for the Session ? Basic concepts in probability and statistics ? Review design of experiments ? Basics of Robust Design ? Research topics – Model-based assessment of RD methods – Faster computer-based robust design – Robust invention Major Concepts of Taguchi Method ? Variation causes quality loss ? Two-step optimization ? Parameter design via orthogonal arrays ? Inducing noise (outer arrays) ? Interactions and confirmation y L(y) Loss Function Concept ? Quantify the economic consequences of performance degradation due to variation What should the function be? y L(y) A o Fraction Defective Fallacy ? ANSI seems to imply a “goalpost” mentality ? But, what is the difference between – 1 and 2? – 2 and 3? 3 21 Isn’t a continuous function more appropriate? m m+' R m-' R A Generic Loss Function ? Desired properties – Zero at nominal value – Equal to cost at specification limit – C1 continuous ? Taylor series y L(y) A o )()( ! 1 )( )( 0 afax n xf nn n | | f m m+' R m-' R y L(y) quadratic quality loss function "goal post" loss function A o m m+' R m-' R Nominal-the-best ? Defined as ? Average loss is proportional to the 2 nd moment about m 2 2 )()( my A yL o o  ' y L(y) quadratic quality loss function A o m m+' R m-' R Average Quality Loss >@ 22 2 )()]([ m A yLE o o  ' PV P V probability density function Other Loss Functions ? Smaller the better ? Larger the better ? Asymmetric 2 2 )( y A yL o o ' 2 2 1 )( y AyL oo ' mymy A mymy A yL Lower o Upper o d ' ! ' if)( if)( )( 2 2 2 2 Who is the better target shooter? Sam John Who is the better target shooter? Sam John Sam can just adjust his sights John requires lengthy training The “P” Diagram Product / Process Response Noise Factors Control Factors There are usually more control factors than responses Exploiting Non-linearity Control Factor Response Use your extra “degrees of freedom” and search for robust set points. Inner and Outer (Crossed) Arrays ? Induce the same noise factor levels for each row in a balanced manner Control Factors Expt. No. ABCD 1 1111 2 1222 3 1333 4 2123 5 2231 6 2312 7 3132 8 3213 9 3321 1122N1 1212N2 1221N3 inner x outer = L9xL4= 36 Compounding Noise ? If the physics are understood qualitatively, worst case combinations may be identified a priori Control Factors Expt. No. ABCD 1 1111 2 1222 3 1333 4 2123 5 2231 6 2312 7 3132 8 3213 9 3321 1122N1 1212N2 1221N3 inner x outer = L9xL4= 36 18 Signal to Noise Ratio ? PERformance Measure Independent of Adjustment PERMIA (two-step optimization) Control Factors Expt. No. ABCD 1 1111 2 1222 3 1333 4 2123 5 2231 6 2312 7 3132 8 3213 9 3321 1122N1 1212N2 1221N3 ? ? o ? ? a 2 2 10 log10 V P K For each row, take an average P and standard deviation V Factor Effects on S/N Ratio A1 A2 A3 B1 B2 B3 C1 C2 C3 D1 D2 D3 10.0 11.0 12.0 13.0 14.0 15.0 Factor Effect Plots P edcbaDCBA ikjiikji  PK ),,,( Prediction Equation Choose the best levels Scaling factor? Factor Effects on S/N Ratio A1 A2 A3 B1 B2 B3 C1 C2 C3 D1 D2 D3 10.0 11.0 12.0 13.0 14.0 15.0 Confirmation P edcbaDCBA ikjiikji  PK ),,,( Build the best plane Check result against prediction What is an Interaction? ? If I carry out this experiment, I will find that: 19 20 21 22 23 24 25 26 B1 B2 B3 A1 A2 A3 If there are significant interactions, the prediction may fail to confirm Control Factors Expt. No. ABCD K 1 112224.8 2 122221.78 3 132220.17 4 212221.38 5 22222.62 6 23222.02 7 312225.03 8 322219.3 9 332220.58 Major Concepts of Taguchi Method ? Variation causes quality loss ? Two-step optimization ? Parameter design via orthogonal arrays ? Inducing noise (outer arrays) ? Interactions and confirmation Some Concerns with Taguchi Methods ? Interactions can often cause failure to confirm ? Two step optimization not really needed ? Use of S/N often not a useful as modeling the response explicitly ? Some experts consider crossed arrays are less efficient than putting noise in the inner array References ? Byrne, Diane M. and Taguchi, Shin “The Taguchi Approach to Parameter Design” Quality Progress, Dec 1987. ? Phadke, Madhav S., 1989, Quality Engineering Using Robust Design Prentice Hall, Englewood Cliffs, 1989. ? Logothetis and Wynn, Quality Through Design, Oxford Series on Advanced Manufacturing, 1994. ? Wu and Hamada, 2000, Experiments: Planning, Analysis and Parameter Design Optimization, Wiley & Sons, Inc., NY. Plan for the Session ? Basic concepts in probability and statistics ? Review design of experiments ? Basics of Robust Design ? Research topics – Model-based assessment of RD methods – Faster computer-based robust design – Robust invention A Model ˉ ? - 1if),0( 0if)1,0( )( i 2 i G G GE cN N f ii p i )1Pr(G effects are normally distributed two classes – strong and weak effect sparsity Chipman, H., M. Hamada, and C. F. J. Wu, 2001, “A Bayesian Variable Selection Approach for Analyzing Designed Experiments with Complex Aliasing”, Technometrics 39(4)372-381. ° ˉ ° ? -    2if 1if 0if ),1Pr( ji11 ji01 ji00 GG GG GG GGG p p p jiij effect hierarchy & inheritance HEEE  |||||| ! ! ! kji n i n ij j n jk k ijkji n i n ij j ij n i iin xxxxxxxxxy 111111 21 ),,,(! Robust Design Method Evaluation Approach 1. Instantiate models of multiple “engineering systems” 2. For each system, simulate different robust design methods 3. For each system/method pair, perform a confirmation experiment 4. Analyze the data Frey, D. D., and X. Li, 2004, “Validating Robust Design Methods, accepted for ASME Design Engineering Technical Conference, September 28 - October 2, Salt Lake City, UT. Including Noise Factors in the Model 1),0(~ 1 miwNIDx i !? ^`11,1 nmix i !?? ),0(~ 2 2 wNIDH HEEE  |||||| ! ! ! kji n i n ij j n jk k ijkji n i n ij j ij n i iin xxxxxxxxxy 111111 21 ),,,(! The first m are noise factors The rest are control factors with two levels Observations of the response y are subject to experimental error Confirmation Using a polynomial response has the advantage that response variance is easily computable ||||| | |||| ! ! ! !  !  !  !    ? ? ? ? o ? ? ? ? a ?  ? ? ? ? o ? ? ? ? a ??? m i m ij j m jk k ijk m i m ij j n jk mk kijkij m i n ij mj n jk mk kjijk n ij mj jijinmm x xxxwxxx 111 2 11 2 1 2 1111 2 121 2 ),,,( EEE EEEV! Fitting the Model to Data ? Collect published full factorial data on various engineering systems – More than data 100 sets collected so far ? Use Lenth method to sort “active” and “inactive” effects ? Estimate the probabilities in the model ? Use other free parameters to make model pdf fit the data pdf -100 -80 -60 -40 -20 0 20 40 60 80 100 0 1 2 3 4 5 6 7 8 9 10 Effects P er c ent age Distribution of Effects -100 -80 -60 -40 -20 0 20 40 60 80 100 0 1 2 3 4 5 6 7 8 9 10 Effects P e rc ent age(% ) Distribution of Effects Different Variants of the Model pp 11 p 01 p 00 p 111 p 011 p 001 p 000 Basic WH 0.25 0.25 0.1 0 0.25 0.1 0 0 Basic low w 0.25 0.25 0.1 0 0.25 0.1 0 0 Basic 2 nd order 0.25 0.25 0.1 0 N/A N/A N/A N/A Fitted WH 0.43 0.31 0.04 0 0.17 0.08 0.02 0 Fitted low w 0.43 0.31 0.04 0 0.17 0.08 0.02 0 Fitted 2 nd order 0.43 0.31 0.04 0 N/A N/A N/A N/A c s 1 s 2 w 1 w 2 Basic WH 10 1 1 1 1 Basic low w 10 1 1 0.1 0.1 Basic 2 nd order 10 1 0 1 1 Fitted WH 15 1/3 2/3 1 1 Fitted low w 15 1/3 2/3 0.1 0.1 Fitted 2 nd order 15 1/3 0 1 1 The model that drives much of DOE & Robust Design The model I think is most realistic Results Basic Fitted Method Experiments WH low w 2 nd order WH low w 2 nd order 37 22 u 1,024 60% 81% 58% 50% 58% 40% 137 22  u III 512 44% 80% 52% 45% 58% 40% 410 2  64 8% 8% 56% 18% 9% 38% 510 2  32 9% 3% 33% 16% 9% 17% 1347 22  u IIIIII 32 12% 8% 51% 16% 25% 38% 13 2  u III OFAT 32 39% 56% 43% 36% 42% 35% OFATOFAT u 32 31% 37% 41% 33% 31% 27% 610 2  16 4% 4% 8% 4% 2% 0% The single array is extremely effective if the typical modeling assumptions of DOE hold Results Basic Fitted Method Experiments WH low w 2 nd order WH low w 2 nd order 37 22 u 1,024 60% 81% 58% 50% 58% 40% 137 22  u III 512 44% 80% 52% 45% 58% 40% 410 2  64 8% 8% 56% 18% 9% 38% 510 2  32 9% 3% 33% 16% 9% 17% 1347 22  u IIIIII 32 12% 8% 51% 16% 25% 38% 13 2  u III OFAT 32 39% 56% 43% 36% 42% 35% OFATOFAT u 32 31% 37% 41% 33% 31% 27% 610 2  16 4% 4% 8% 4% 2% 0% The single array is terribly ineffective if the more realistic assumptions are made Results Basic Fitted Method Experiments WH low w 2 nd order WH low w 2 nd order 37 22 u 1,024 60% 81% 58% 50% 58% 40% 137 22  u III 512 44% 80% 52% 45% 58% 40% 410 2  64 8% 8% 56% 18% 9% 38% 510 2  32 9% 3% 33% 16% 9% 17% 1347 22  u IIIIII 32 12% 8% 51% 16% 25% 38% 13 2  u III OFAT 32 39% 56% 43% 36% 42% 35% OFATOFAT u 32 31% 37% 41% 33% 31% 27% 610 2  16 4% 4% 8% 4% 2% 0% Taguchi’s crossed arrays are more effective than single arrays A Comparison of Taguchi's Product Array and the Combined Array in Robust Parameter Design We have run an experiment where we have done both designs simultaneously (product and combined). In our experiment, we found that the product array performed better for the identification of effects on the variance. An explanation for this might be that the combined array relies too much on the factor sparsity assumption. Joachim Kunert, Universitaet Dortmund The Eleventh Annual Spring Research Conference (SRC) on Statistics in Industry and Technology will be held May 19-21, 2004. Results Basic Fitted Method Experiments WH low w 2 nd order WH low w 2 nd order 37 22 u 1,024 60% 81% 58% 50% 58% 40% 137 22  u III 512 44% 80% 52% 45% 58% 40% 410 2  64 8% 8% 56% 18% 9% 38% 510 2  32 9% 3% 33% 16% 9% 17% 1347 22  u IIIIII 32 12% 8% 51% 16% 25% 38% 13 2  u III OFAT 32 39% 56% 43% 36% 42% 35% OFATOFAT u 32 31% 37% 41% 33% 31% 27% 610 2  16 4% 4% 8% 4% 2% 0% An adaptive approach is quite effective if the more realistic assumptions are made Results Basic Fitted Method Experiments WH low w 2 nd order WH low w 2 nd order 37 22 u 1,024 60% 81% 58% 50% 58% 40% 137 22  u III 512 44% 80% 52% 45% 58% 40% 410 2  64 8% 8% 56% 18% 9% 38% 510 2  32 9% 3% 33% 16% 9% 17% 1347 22  u IIIIII 32 12% 8% 51% 16% 25% 38% 13 2  u III OFAT 32 39% 56% 43% 36% 42% 35% OFATOFAT u 32 31% 37% 41% 33% 31% 27% 610 2  16 4% 4% 8% 4% 2% 0% An adaptive approach is a solid choice (among the fast/frugal set) no matter what modeling assumptions are made Plan for the Session ? Basic concepts in probability and statistics ? Review design of experiments ? Basics of Robust Design ? Research topics – Model-based assessment of RD methods – Faster computer-based robust design – Robust invention Sampling Techniques for Computer Experiments Random Sampling Stratified Sampling Latin Hypercube Sampling Proposed Method ? Simply extend quadrature to many variables ? Will be exact to if factor effects of 4 th polynomial order linearly superpose ? Lacks projective property ? Poor divergence z 1 z 2 z 3 1.3556 2.8750 -1.3556 -2.8750 Why Neglect Interactions? |||| ||| || |       ? ? 1 · ¨ ¨ ? §    ? ? ? ? ? 1 · ¨ ¨ ¨ ¨ ¨ ? §     ? ? ? ? ? 1 · ¨ ¨ ¨ ¨ ¨ ? §     n i n ji j n kj k n lk l ijkkijllikklijjljkkliijlijllijkkikllijjk jklliijkjjkkiilljjlliikkkklliijjijkl n i n ji j n kj k iijkjkkkiijkjjjkijkkijjjijjkiiik ijkkiiijjjkkiikkiikkiijjjjkkiijjikkijj jjkiikjkkiijijkkijjkiijkijk n i n ji j iijjjjjjiijjiiiiiijjjjijjiiiijjjij iiijijiijjjjiijjiiiijjijji iijjijjjiiijijjiijij n i iiiiiiiiiiiiiiiiiiii 1111 2 111 2222 11 222222 1 2222 2 22222 2222 6666 64442 22333 2424666 64422 8151533 96241562))(( EEEEEEEEEE EEEEEEEEE EEEEEEEE EEEEEEEEEE EEEEEEEE EEEEEEEEEE EEEEEEEEEE EEEEEE EEEEEEEEKV z lk n j n ji i n jk k n kl l jiijklk n j n ji i n kk k jiijkji n j n ji i ij n i ii zzzzzzzzzzz |||||||||| d d d d d d  11 1111 1111 0 )( EEEEEK If the response is polynomial Then the effects of single factors have larger contributions to V than the mixed terms. Fourth Order – RWH Model Fit to Data Legend Quadrature 29 samples Cubature 73 samples HSS 29 samples HSS 290 samples LHS 29 samples LHS 290 samples d=7 4d+1=29 d 2 +3d+3=73 0 5 10 15 20 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 % Error in Estimating Standard Deviation Cumulative Probability Continuous-Stirred Tank Reactor ? Objective is to generate chemical species B at a rate of 60 mol/min )()( RBBRAAip HrHrVTTCFQ  U W RTE A Ai A A ek C C /0 1   W W RTEB B A RTE ABi B ek CekC C A /0 /0 1     A RTE AA Cekr A /0   A RTE AB RTE BB CekCekr AB /0/0    Q F T i C Ai C Bi F T C A C B Adapted from Kalagnanam and Diwekar, 1997, “An Efficient Sampling Technique for Off-Line Quality Control”, Technometrics (39 (3) 308-319. Comparing HSS and Quadrature Hammersley Sequence ? Required ~150 points ? 1% accuracy V 2 ? V 2 from 1,638 to 232 ? Nominally on target ? Mean 15% off target Quadrature ? Used 25 points ? 0.3% accuracy in P ? 9% accuracy in (y-60) 2 far from optimum ? 0.8% accuracy in (y-60) 2 near to optimum ? Better optimum, on target and slightly lower variance ? E(L(y)) = 208.458 0 2040608010 0 0.01 0.02 0.03 Production Rate (mol/min) Probability density (min/mol) HSS quadrature Plan for the Session ? Basic concepts in probability and statistics ? Review design of experiments ? Basics of Robust Design ? Research topics – Model-based assessment of RD methods – Faster computer-based robust design – Robust invention Problem definition Concept design Detail design Manufacture Use Percentage of total 100 50 75 25 Quality determined & costs committed Design flexibility Lifecycle phase Source: Russell B. Ford and Philip Barkan Robust parameter design An opportunity Defining “Robustness Invention” ? A “robustness invention” is a technical or design innovation whose primary purpose is to make performance more consistent despite the influence of noise factors ? The patent summary and prior art sections usually provide clues Example -- A Pendulum Robust to Temperature Variations ? Period of the swing is affected by length ? Length is affected by temperature ? Consistency is a key to accurate timekeeping ? Using materials with different thermal expansion coefficients, the length can be made insensitive to temp Theory of Inventive Problem Solving (TRIZ) ? Genrich Altshuller sought to identify patterns in the patent literature ? Defined problems as contradictions ? Provided a large database of solutions ? Stimulate designer’s creativity by presenting past designs appropriate to their current challenge Searching for Robustness Inventions ? Keyword search in USPTO database ? There seem to be several thousand Search Term Number of Hits Independent 114,201 Uncoupling 2,189 Decoupling 6,505 Noise compensation 22,092 Noise control 142,138 Noise conditioning 10,787 Resistant 3,535 Acclimation 712 Desensitize 447 Sweet spot 1317 Operating window 728 TOTAL 867,472 Search Term Number of Hits Insensitive 35,708 Less sensitive 12,253 Robust 27,913 Accurate 221,600 Reliable 211,533 Repeatable 16,458 Tolerant 13,765 Despite changes 1,323 Regardless of changes 1,147 Independent of 20,521 Self compensating 1,269 Force Cancellation 59 Signal Response Noise Classifying Inventions via the P-Diagram Patent #5,024,105 – Viscosity-insensitive variable-area flowmeter Patent #5,483,840 – “System for Measuring Flow” Patent #4,487,333 – “Fluid Dispensing System” Courtesy of the United States Patent and Trademark Office, http://www.uspto.gov.?? Discussion Point Ball Ramp Funnel Response = the time the ball remains in the funnel Noise Factor = 2 Types of Ball Name some ways that you might modify the ball and ramp equipment or procedure to make the system robust to ball type. Conclusions So Far ? Effective strategies for experimentation should be adaptive (not always, but under a broad range of scenarios) ? Resolution is not always required for reliable improvement ? Simulating the process of experimentation provides insights I can’t get from deduction alone Questions? Dan Frey Assistant Professor of Mechanical Engineering and Engineering Systems