16.422 Human Supervisory Control Judgment Under Uncertainty: Heuristics & Biases The Uncertain State of the World 16.422 Probability Theory Objective Probability Subjective Probability Statistical Probability Axiomatic Probability ?SEU ?MAUT ? Bayesian Nets Subjective Assessment 16.422 ? Subjective assessment of probabilities is akin to assessment of size and distance ? Perception versus expectation ? Heuristics are useful but can be misleading The Ames Room 16.422 http://psylux.psych.tu-dresden.de/i1/kaw/diverses%20Material/www.illusionworks.com/html/ames_room.html (Image removed due to copyright considerations.) Expectations Can Fool You… 16.422 These checkershadow images may be reproduced and distributed freely. ?1995, Edward H. Adelson. Used with permission. Human Estimation & Cue Integration 16.422 ? Humans as intuitive statisticians – Good at estimating means, reasonably good at mid-range proportions ? Not good on the tails – Not so good at estimating variances and correlations – Also not good at extrapolating non-linear trends ? Underestimate exponential growth ? Cue assimilation issues – Missing – Information overload –Salience ? Underestimate cues that require calculation – The need for heuristics As-if Heuristic 16.422 ? Cues are equally weighted and differential weights are not considered – Regression to the mean – Reliability of cues – Letters of recommendation – content v. tone ? Humans are poor intuitive or clinical predictors as compared to computers – Multiple cues of different information value ? Cognitive parsimony – Humans tend to reduce load on working memory. – Avoid processing of cues that require mental calculation Representative Heuristic 16.422 ? Probabilities are evaluated by the degree to which A resembles B ? Problems – Prior probability (or base-rate frequency) of outcomes ? Engineers vs. lawyers ? No specific evidence vs. worthless evidence – Insensitivity to sample size ? Large vs. small hospital – Misconceptions of chance – Insensitivity to prediction – Illusion of validity ? Stereotypes – Regression to the mean Availability Heuristic 16.422 ? Assessing probability or frequency bases on information that is most readily recalled ? Problems: – Retrievability of instances ? Familiarity, salience, recency – driven by experience – Effectiveness of a search set ? Searching for solutions in your long term memory – Imaginability ? Simplicity ? Decision making & alternatives – Illusory Correlation ? How frequently two events co-occur Effectiveness of search set….abstract words as opposed to concrete Adjustment &Anchoring Heuristic 16.422 ? People start with an initial guess and adjust answers based on available information ? Problems – Insufficient adjustment ? 1x2x3x4x5x6x7x8 v. 8x7x6x5x4x3x2x1 ? 512, 2250, 40320 – Evaluation of simple, conjunctive (and) & disjunctive (or) events ? Overestimate conjunctive, underestimate disjunctive – Ordering matters – Assessment of subjective probability distribution ? Overly narrow confidence intervals Simple – draw red from bag 50/50 red and white Conjunctive – draw seven successive reds from a bag 90/10 Disjunctive – draw a red at least once in seven tries from a bag 10/90 .50/.48/.522 Confidence Intervals 16.422 ? A confidence interval gives an estimated range of values which is likely to include an unknown population parameter, given set of sample data. – The confidence interval is the likely range of the true value. ? Precision determined by the width of the confidence interval ? http://www.sportsci.org/resource/stats/generalize.html (Courtesy of Will Hopkins. Used with permission.) Some Other Biases 16.422 ? Overconfidence Bias – Confidence exceeds prediction accuracy – Prevents people from seeking additional information/cues ? Confirmation Bias – People seek information that supports a preformed hypothesis ? Do not seek or discount contrary information ? Automation Bias – People tend to believe computer recommendations and do not seek out conflicting evidence Effectiveness of search set….abstract words as opposed to concrete Framing Effect ? Prospect theory - People value a certain gain more than a probable gain with an equal or greater expected value; the opposite is true for losses. – Would you rather win (or lose) $1 and 0% risk or $2 with 50/50 risk? – Take the sure thing? ? Framing Effect: Choices are made differently depending on whether the choices are framed in terms of gains or losses ? Sunk cost bias ? How does this impact design? 16.422 Bounded Rationality ? Criticism for the application of classical decision theory – Unbounded rationality - humans make rational decisions, but they do so without time constraints, with complete a priori knowledge, and unlimited computational abilities ? Bounded rationality: concept of satisficing, which occur when decision makers stop the search for a solution when the first alternative is found that meets all constraints – Probably not the optimal solution ? Fast & frugal heuristics 16.422 Fast & Frugal Heuristics 16.422 Execute Search Rule Does Alternative Meet Stopping Rule? Make Decision Yes No ? Some search rules: – Take the last ? The decision that worked for this problem last time – Take the best ? Consider primarily the cue with highest validity ? Seed for recognition primed decision making Implications for Design 16.422 ? Ways to combat problems with decision biases: –Training – Emphasis on feedback ? Ambiguous/Delayed/Selective processing – Debiasing – Proceduralization – Development of decision aids ? Decision guidance – Interactive – Participative-suggestive – Jiang & Klein paper Social Judgment Theory ? SJT attempts to model human decision-making using classical decision theory through an ecological approach. ? Lens Model – Attempt to model how well a person's judgments match the environment they are trying to predict. 16.422 Y s X 1 r s,1 r a Y E X 2 X 3 X 4 r s,4 r s,2 r s,3 r E,1 r E,1 r E,3 r E,4 Ecology (Criterion) Subject Judgments Achievement, r a Ecological Validities, r E,i Cue Utilizations, r S,i Cues, X i More on the Lens Model 16.422 ? Uses regression – Prediction (dependent) from cues (independent) – Determines not only the degree of agreement between a human judge and the state of the environment, but also the predictability of the environment as well as the level of consistency of human judgments. ? Problems: – Assumption that relationship is linear – Identification of cues eYY ss +=+= ? Error Model Model Predicted )1()1( 22 sesea RRCRGRr ??+= References 16.422 ? http://members.lycos.co.uk/brisray/optill/vision1.htm ? http://www.yorku.ca/eye/funthing.htm ? Gigerenzer, G. and P. M. Todd (1999). Simple Heuristics That Make Us Smart. New York, Oxford University Press ? Simon, H. A. (1955). "A behavioral model of rational choice." Quarterly Journal of Economics 69(99-118). ? Simon, H. A., R. Hogarth, et al. (1986). Decision Making and Problem Solving. Research Briefings 1986: Report of the Research Briefing Panel on Decision Making and Problem Solving, Washington D.C., National Academy Press. ? Svenson, O. (1979). "Process Descriptions of Decision Making." Organizational Behavior and Human Performance 23: 86-112