Experimental Design 16.621 Elements of Designing an Experiment ? Goal is to assess your hypothesis – The main thing is to keep the main thing the main thing - [Tom Curran, AFRL] ? The design goal for your experiment is to achieve your objective and satisfy your success criteria. ? Your HOS are the touchstones for your experimental design. SOME FRAMING QUESTIONS ? Once the HOS is set we need to define in depth the path to get there ? Need to think through the whole process on an end-to-end basis ? Need to identify the hard parts ? Need to identify the key milestones (“mid term exams”) ? This will include (at some level) questions such as: - What will we do? - Why will we do it? - Where will we do it? - Who will help us? - How will we do it? - How well do we have to do it? - When will we do it? Project Specific Applications of these Questions ? What is the best overall approach to achieve my objective? – e.g., should I test children’s car seats in the lab or in actual airplane environment? ? What are my choices of facilities? – e.g., should I use a water tunnel or a wind tunnel to determine the performance of a split-tip keel? ? How will I measure my independent variables? – e.g., Should I measure rotational speed of my airspeed indicator with a stroboscope or a shaft encoder? Observations About Design ? Design is an inductive process. ? Ways to learn design – Study past designs that have been successful and extract patterns ? Look at the online models – Work with a “master” designer ? Involve your advisor – Do it yourself and receive feedback ? Basically this is the 16.621 process The 621 Design Process Idea! HOS Conceptual Design Detailed Design Version II Overall experimental approach; “scope and scale”; type of model(s), materials, software; data to be taken; sources of error,…. Version I Version III Detailed drawings and dimensions; pseudo code; specific equipment; make/buy decisions; construction methods; calibration methods; detailed error analysis, budget and schedule,…. Concpetual Design Process ? Start with an open mind. Avoid preconceived approach. ? Brainstorm with your partner – Think of multiple ways to accomplish your objective(s) – Ask other people, keep your “eyes open” – Avoid rejecting ideas, particularly wild ones, at this stage. – Keep on the cusp of creative thinking. – Write these ideas in your notebooks. ? Develop a set of desired attributes to use to compare your different designs, keeping your HOS in mind. ? Sort through the different options to arrive at a best design to achieve your objective. ? Verify your conceptual design supports your HOS – Revise HOS if original set is not feasible ? Then proceed with “detailed design”. A 621 Example The potential strategies for creating the aerated ice are as follows: 1 Drill air holes into the ice. 2 To use a mixture of crushed ice and water close to freezing point. 3 Suspend small pockets of air in the ice mix (using tethered balloons) 4 Construction of a specialized rig that constantly blows air through the ice mix. 5 The use of soda water or carbonated water. 6 A combination of these strategies. The most effective method for creating aerating ice will be determined during the preliminary testing stage of the project. The method will be chosen based on the best combination of speed, ease of production, and quality of ice produced. Source: Bennett, W. “Simulating and Testing Ice Screw Performance in the Laboratory”, 16.621 Design Proposal, Fall 2002 Fidelity of Your Approach ? An experiment is usually a “model” of the “real thing”. – It may be too complex or expensive to test the real thing – It usually will be difficult to have control over the variables in a real operational environment – e.g. a part task simulator will be used to test a cockpit display instead of modifying a real cockpit. ? The “fidelity”* of your (model) experiment needs to be assessed as part of the design process. ? Lack of fidelity is one source of errors. ? A good way to consider these errors in the design stage is to write them down, assess their importance, and consider mitigation strategies. *Fidelity: Exact correspondence with fact or with a given quality, condition, or event; accuracy. http://dictionary.reference.com/ A 621 Example A number of potential error sources in the experiment have been identified. These are summarized in Table 1 along with a brief description of the problem and how to decrease or eliminate the error. Table 1: Sources of Error Source of Error Description Method of Mitigation Hysteresis Shape memory alloys have a hysteresis effect that could potentially cause the actuators to get “stuck” in a particular state In order to decrease the effects of hysteresis, ample cooling time will be given between consecutive test runs Cyclic Loading Depending upon the frequency and duration of the perturbations, the actuators could experience several thousand cycles of heating and cooling throughout the run of the experiment A low frequency of perturbations, and periodic checks of a baseline condition will be used to ensure that the actuators have not deformed over time. Sensors Commercially available accelerometers will have a certain offset, and will drift with time These errors will be corrected during data analysis by shifting and rescaling raw data Inconsistent Release If the actuators start functioning before the parachute is released or fully inflated, errors resulting from initial conditions could occur. In order to ensure that the parachute has completely inflated before a test function is sent to the actuators, a time delay will be built into the function generator based on the initial trial run of the experiment. Source: Wong, B. “Assessment of Shape Memory Alloys for Steered Parachutes ”, 16.621 Design Proposal, Fall 2002 Summary ? Design your experiment to achieve your objective, satisfy your success criteria and assess your hypothesis. ? The conceptual design stage starts with an open mind and brainstorming, and then uses objective criteria to select best approach, considering modeling fidelity (errors). ? Continue to refer to or revise your HOS Active Learning ? Working with your partner, pick one aspect of your experimental design (facility, measurement,…) and develop two or more approaches. ? Pick two or three attributes you would use to select one approach over the other. ? Be prepared to report to the class at the end of the hour.