Analysis of Variance ANOVA Robust System Design 16.881 Session #7 MIT Proposed Schedule Changes ? Switch lecture ?No quiz – Informal (ungraded) presentation of term project ideas ? Read Phadke ch. 7 -- Construction Orthogonal Arrays – Quiz on ANOVA – Noise experiment due Robust System Design 16.881 Session #7 MIT Learning Objectives ? Introduce hypothesis testing ? Introduce ANOVA in statistic practice ? Introduce ANOVA as practiced in RD ? Compare to ANOM ? Get some practice applying ANOVA in RD ? Discuss / compare / contrast Robust System Design 16.881 Session #7 MIT Hypothesis Testing A technique that uses sample data from a population to come to reasonable conclusions with a certain degree of confidence Robust System Design 16.881 Session #7 MIT Hypothesis Testing Terms ? Null Hypothesis (H o ) -- The hypothesis to be tested (accept/reject) ? Test statistic -- A function of the parameters of the experiment on which you base the test ? Critical region -- The set of values of the test statistic that lead to rejection of H o Robust System Design 16.881 Session #7 MIT Hypothesis Testing Terms (cont.) ? Level of significance (α) -- A measure of confidence that can be placed in a result not merely being a matter of chance ? p value -- The smallest level of significance at which you would reject H o Robust System Design 16.881 Session #7 MIT Robust System Design Session #7 MIT16.881 Comparing the Variance of Two Samples ? Null Hypothesis -- H o : ? Test Statistic -- ? Acceptance criteria -- ? Assumes independence & normal dist. r= 2 1 σ σ F 1 r 2 Var X1( Var X2( . 2 1 5.0)2,1,( α? <?ddFpF ) ) F Distribution ? Three arguments – d1 (numerator DOF) – d2 (denominator DOF) – x (cutoff) F(x,d1,d2) Γ d1 d2 2 d1 d1 2 . d2 d2 2 . Γ d1 2 Γ d2 2 . x d1 2 1 d1 x . d2( d1 d2 2 . for x > 0 x ) Robust System Design 16.881 Session #7 MIT Rolling Dice ? Population 1 -- Roll one die ? Population 2 -- Roll two die ? Go to excel sheet “dice_f_test.xls” Robust System Design 16.881 Session #7 MIT One-way ANOVA ? Null Hypothesis -- H o : μ 1 =μ 2 =μ 3 = L ? Test Statistic -- F SSB SSW dfB dfW ? Acceptance criteria -- pF(F , dfB , dfW) < (1 α) ? Assumes independence & normal dist. Robust System Design 16.881 Session #7 MIT ANOVA & Robust Design Noise Factors H: This noise Product / Process Response Signal Factor factor affects the mean H: Factor setting A1 is more robust than factor Optimize robustness setting A2 Control Factors Robust System Design 16.881 Session #7 MIT ANOVA and the Noise Experiment ? Did the noise factors we experimented with really have an effect on mean? ? Switch to Excel sheet “catapult_L4_static_anova.xls” Robust System Design 16.881 Session #7 MIT Why Test This Hypothesis? ? Factor setting PP3 is more robust than setting PP1 ? Phadke -- “In Robust Design, we are not concerned with such probability statements, we use the F ratio for only qualitative understanding of the relative factor effects” Factor Effects on the S/N Ratio 10 11 12 13 14 15 16 17 SP1 SP3 DA1 DA3 CU P 1 CU P3 PP 1 P P3 S/ N Ra t i o ( d B) Robust System Design 16.881 Session #7 MIT Analysis of Variance (ANOVA) ? ANOVA helps to resolve the relative magnitude of the factor effects compared to the error variance ? Are the factor effects real or just noise? ? I will cover it in Lecture 7. ? You may want to try the Mathcad “resource center” under the help menu Robust System Design 16.881 Session #7 MIT Additive Model ? Assume each parameter affects the response independently of the others η( A i , B j , C k , D i ) =μ+ a i + b j + c k + d i + e A: Stop Pin B: Draw Angle C: Cup Position D: Post Pin Mean Distance Std Deviation Variance S/N Ratio A1 B1 C1 D1 16.9 3.0 8.8 15.1 A1 B2 C2 D2 46.6 8.1 65.7 15.2 A1 B3 C3 D3 91.9 13.4 178.5 16.7 A2 B1 C2 D3 25.8 5.8 34.1 12.9 A2 B2 C3 D1 49.2 11.9 141.6 12.3 A2 B3 C1 D2 67.2 8.7 75.2 17.8 A3 B1 C3 D2 18.1 6.4 41.5 9.0 A3 B2 C1 D3 45.9 8.7 76.3 14.4 A3 B3 C2 D1 53.0 11.2 125.1 13.5 GRAND MEANS 46.1 8.6 83.0 14.1 Robust System Design 16.881 Session #7 MIT Analysis of Means (ANOM) ? Analyze the data to discover m A1 , a i ... Factor Effects on the S/N Ratio 10 11 12 13 14 15 16 17 SP 1 SP 3 D A1 D A 3 C U P 1 C U P 3 PP 1 PP 3 S / N Ra t i o ( d B) Robust System Design 16.881 Session #7 MIT Analysis of Variance (ANOVA) ? Analyze data to understand the relative contribution of control factors compared to “error variance” Factor Effects on the S/N Ratio 10 11 12 13 14 15 16 17 S P 1 SP 3 DA 1 D A3 C UP 1 C UP 3 PP 1 P P3 S/ N Rat i o ( d B) Robust System Design 16.881 Session #7 MIT Breakdown of Sum Squares GTSS SS due to mean Total SS SS due to factor A SS due to factor B SS due to error etc. Robust System Design 16.881 Session #7 MIT Breakdown of DOF Robust System Design Session #7 MIT16.881 n 1 SS due to mean n-1 (# levels) -1 factor A n = Number of η values (# levels) -1 factor B DOF for error etc. Computation of Sum of Squares n ? Grand total sum of squares GTSS = ∑ η i 2 i=1 ? Sum of squares due to mean = nμ 2 n ? Total sum of squares = ∑ (η i ?μ) 2 i=1 ? Sum of squares due to a factor = replication# [(m A1 ?μ) 2 + (m A2 ?μ) 2 + (m A3 ?μ) 2 ] ? Sum of squares due to error – Zero with no replicates – Estimated by “pooling” Robust System Design 16.881 Session #7 MIT Pooling ? Provides an estimate of error without empty columns or replicates ? Procedure – Select the bottom half of the factors (in terms of contribution to Total SS) Robust System Design 16.881 Session #7 MIT F-statistic ? ? ? Error variance = sum of squares due to error degrees of freedom for error F = mean square for factor Error variance SS for factor mean square for factor = DOF for factor – F=1 Factor effect is on par with the error – F=2 The factor effect is marginal – F>4 The factor effect is substantial Robust System Design 16.881 Session #7 MIT Confidence Intervals for Factor Effects ? Phadke – Variance in a i is error variance / replication # – 95% confidence interval for factor effects is two standard deviations in a i ? How does one interpret this value? Robust System Design 16.881 Session #7 MIT Example Catapult Experiment ? Switch to Excel “Catapult_L9_2.xls” A: Stop Pin B: Draw Angle C: Cup Position D: Post Pin Mean Distance Std Deviation Variance S/N Ratio A1 B1 C1 D1 16.9 3.0 8.8 15.1 A1 B2 C2 D2 46.6 8.1 65.7 15.2 A1 B3 C3 D3 91.9 13.4 178.5 16.7 A2 B1 C2 D3 25.8 5.8 34.1 12.9 A2 B2 C3 D1 49.2 11.9 141.6 12.3 A2 B3 C1 D2 67.2 8.7 75.2 17.8 A3 B1 C3 D2 18.1 6.4 41.5 9.0 A3 B2 C1 D3 45.9 8.7 76.3 14.4 A3 B3 C2 D1 53.0 11.2 125.1 13.5 GRAND MEANS 46.1 8.6 83.0 14.1 Robust System Design 16.881 Session #7 MIT Homework ? Grades are exceptionally high ? Some are spending vast amounts of time ? This represents 20% of the final grade Mean 94.2 101.0 96.5 95.4 Standard deviation 2.7 4.4 6.3 1.5 Maximum 98 109 101 97.3 Robust System Design 16.881 Session #7 MIT Quizzes ? Some consistently score high ? Others struggling, but learning ? Remember, this is only 10% Quiz #1 Quiz #2 Quiz #3 Quiz #4 Mean 74.3 83.4 77.5 82.8 Standard deviation 19.3 10.4 22.3 17.6 Maximum 100 100 100 110 Robust System Design 16.881 Session #7 MIT Next Steps ? Hand in homework #5 ? Homework #7 due on Lecture 10. ? Next session tomorrow – Present your ideas for a term project ? Following session – Quiz on ANOVA – Homework #6 (Noise Exp.) due – Constructing orthogonal arrays (read ch. 7) Robust System Design 16.881 Session #7 MIT