Test Matrices 16.621 Prof. Earll Murman 2 Variables z Independent Variable: A quantity that you will vary and control – E.G.; angle of attack, chamber pressure or temperature, coefficient in algorithm, luminosity, gain, ... z Parameter: A quantity that is set or otherwise determined, which you will not vary but which needs to be recorded – E.G.; atmospheric pressure, constant in algorithm, battery voltage,... z Dependent Variable: A measureable output quantity of your experiment which is a function of the input variables and parameters – E.G.; reaction time, force, energy consumed, temperature 3 Exercise z With your partner, write down your expected variables and parameters. z Independent Variables z Parameters z Dependent variables 4 Test Matrices z A graphical display of your experimental independent variables to help: – Covey the scope of your experiment to your audience – Plan and execute your experiment z Each cell represents a “data point” for your experiment for which you will collect values for the dependent variables. 4000 RPM 0 mph 5 mph 10 mph 15 mph 1000 RPM 2000 RPM 3000 RPM0 RPM Propeller RPM Speed . Courtesy of Cyndi Vongvanith and Lester McCoy IVs DVs 5 Multi-Variable Experiments – Factor = Number of independent variables ? Four-Factor experiment has 4 independent variables – Level = A given value of an independent variable ? Numerical – 200, 300, 400 … ? Qualitative – Brand x, Brand y, Brand z … – Full-Factorial Experiment ? All factors at all levels ? May lead to a huge number of data points. – Fractional-Factorial Experiment ? Expert judgement: carefully selected subset ? Adaptive: decide as you get some data ? Design of Experiments: Taguchi, orthogonal arrays – Beyond scope of 16.62X 6 Presentation of Test Matrices: Full Factorial – Test matrix used for graphical representation of test plan ? Define: A n = n th level of factor A – One variable matrix – Two variable matrix – Three variable matrix mm2m1m n22212 n12111 BABABA BABABA BABABA Λ ΜΟΜΜ Λ Λ A 1 B 1 C 1 A 4 B 3 C 2 IV A 1 ... A n DV1 DV2 DV3 7 – N- variable matrix ? Creativity needed ? Stamina will probably also be required! D 1 E 1 F 1 D 4 E 3 F 2 D 1 E 1 F 1 D 4 E 3 F 2 For A 1 B 1 : For A 2 B 7 : Presentation of Test Matrices: Full Factorial 8 Expert Judgement Approach – Eliminate some combinations of independent variables to reduce the total number of data points – Often required to make experiment feasible within time and budget constraints – Strategies for elimination ? Insight from previous theory or experiments ? Wisdom from advisor or other subject matter expert ? Logical thought about interrelationship of variables on the physics of the problem 9 Adaptive Approach – Preliminary Runs ? Use theory to bracket range ? 2 or 3 test cases to check set-up ? Compare with theory – Production Runs ? Data range and spacing ? May not be uniform – Cluster samples in “interesting areas” Run Data Run Data 10 Additional Considerations z Repeatability: Is there reason to believe that the measurement accuracy will be increased if multiple “runs” are made with the same independent variables and parameters? z Hysteresis: Is there reason to believe the physical effect being studied may depend upon the sequence or rate in which you vary the independent variable? z Learning: Is the reason to believe your human subjects or intelligent software will become more capable during the experiment through learning? z Fatigue: Will your subjects become less capable during the test due to tiring? Refer to backup slides for more information. 11 Hysteresis z Hysteresis - “The lagging of an effect behind its cause, as when the change in magnetism of a body lags behind changes in the magnetic field.” http://www.dictionary.com/ – Feature of physical problem be studied – Feature of measurement device (undesirable) z Example - pitching vs fixed delta wing α increasing α decreasing α fixed Hysteresis depends upon pitching rate 12 Learning z “The act, process, or experience of gaining knowledge or skill” http://www.dictionary.com/ z The response of a human subject changes as an experiment proceeds because they gain skill or knowledge - the experiment changes the subject! z E.G. measuring a response of a human to a video game experiment – Test 1 and Test 2 are different, but subject learns how to play the game in test 1 and can respond more quickly in test 2. z Typical mitigation strategies – Train subjects to fix skill level – Test many subjects and vary the order of the test sequence to average out the learning effect. 13 Fatigue z Fatigue: “Physical or mental weariness resulting from exertion.” http://www.dictionary.com/ z This is different than learning – Learning leads to a new skill level – Fatigue is a temporary loss of capability z Fatigue can effect both the subject and the experimenter – Needs to be considered in the design of the execution of the experiment z Fatigue can also apply to physical materials