16.422 Human Supervisory Control Classical Decision Theory & Bayes’ Theorem Decision Theory & Supervisory Control ? Two broad areas of decision theory – Normative ? Prescriptive – Descriptive ? Why & how do we make decisions? – How does technology support or detract from “optimal” decision making ? Informing Design 16.422 ? Normative: How decisions should be made ? Prescriptive: How decisions can be made, given human limitations ? Descriptive: How decisions are made Elements of Decision Making 16.422 ? The rational decision making process – Define the problem – Information gathering – Identify alternatives – Evaluate alternatives – Select and implement decision ? Why decisions often go wrong: – Certainty vs. uncertainty – Bounded rationality – Nonlinearity – Habits & heuristics – Path of least resistance Classic Decision Theory ? Maximizing expected value of the outcome – Primary assumption of rationality ? Mathematical models of human decision making ? Following assumptions are made about how people make decisions: – All alternatives are considered – Information acquisition is perfect – Probabilities are calculated correctly ? Problems: – Humans are not rational decision makers – No universal agreement on the worth associated with various outcomes. 16.422 Basic Concepts in Decision Analysis 16.422 ? State: A description of the world ? Events: A set of states {S 1 , S 2 ,…, S j } ? Consequences: A set of states ? Acts: Results from decisions S 1 S 2 Act 1 (A 1 )C 11 C 12 Act 2 (A 2 )C 21 C 22 Basic Terminology 16.422 ? Ordering of preferences – Alternatives can be quantified and ordered – A > B, A preferred to B – A = B, A is equivalent to B –A ≥ B, B is not preferred to A ? Transitivity of preference –if A 1 ≥ A 2 , & A 2 ≥ A 3 , then A 1 ≥ A 3 , Decision Trees 16.422 ? Decisions over time and/or events Decision Node S 1 S 2 C 11 C 12 C 21 C 22 A 2 A 1 A 1 A 2 Chance Nodes Decision Making Under Certainty 16.422 ? Each alternative leads to one and only one consequence – Consequences are known ? Lexicographic ordering ? Dominance ? Satisficing ? Maximin ? Minimax Lexicographic Ordering 16.422 ? All options are first compared in terms of the criterion deemed most important. – If there is a unique best performing option, that option is selected as the most preferred. ? In case of a tie, the selection process moves to the second ranked criterion – Seeks the remaining option which scores best on the second criterion. ? In case of a tie on the second criterion, the process is repeated for options tying in terms of both the first and second ranked criteria, – And so on until a unique option is identified or all criteria have been considered. Dominance 16.422 ? Effective for both quick qualitative and quantitative comparisons ? In the real world, solutions rarely are outwardly dominant S 1 S 2 S 3 S 4 13 3 3 2 0 2 20 A 1 0 A 2 0 A 3 -1 Satisficing 16.422 ? Otherwise known as Minimum Aspiration ? Select any A i over all events j such that C ij ≥ aspiration level ? Cease to search for alternatives when you find an alternative whose expected utility or level of preference satisfaction exceeds some previously determined threshold. ? Stopping rule ? Is this in keeping with the rational approach to decision making? Maximin 16.422 ? Select any A i over all events j such that you minimize the maximum loss – Maximum of the (row) minima. – Conservative approach but overly pessimistic S 1 S 2 S 3 15 -6 -2 1 9 3 A 1 3 A 2 4 A 3 2 Maximin 16.422 ? Select any A i over all events j such that you minimize the maximum loss – Maximum of the (row) minima. – Conservative approach but overly pessimistic S 1 S 2 S 3 Row min 15 -6 -2 1 9 -6 -2 31 A 1 3 A 2 4 A 3 2 Minimax (Regret) 16.422 ? Avoid regrets that could result from making a non- optimal decision. ? Regret: the opportunity loss if A i is chosen – Opportunity loss is the payoff difference between the best possible outcome under S j and the actual outcome from choosing A i . – Convert payoff matrix to opportunity loss (regret) table S 1 S 2 S 3 15 -6 -2 A 3 321 Best 15 4 1 9 A 1 3 A 2 4 S 1 S 2 S 3 07 3 A 3 12 2 0 6 A 1 1 A 2 0 Maximin v. Minimax 16.422 ? Minimax contains more problem information through opportunity losses (e.g. actual monetary losses plus unrealized potential profits) – Still conservative but more middle-of-the-road S 1 S 2 S 3 Maximin Minimax -6 7 6 12 -2 1 -6 -2 1 15 9 3 A 1 3 A 2 4 A 3 2 The Uncertain State of the World 16.422 Probability Theory Objective Probability Subjective Probability Statistical Probability Axiomatic Probability Decision Making Under Risk ? Each alternative has several possible consequences – The probability of occurrence for each consequence, C, is known. ? Expected value: E (A i ) = Σ j p j (C ij ) S 1 S 2 S 3 .3 -6 -2 1 P(S j ).1 .6 A 1 15 9 3 3 A 2 4 A 3 2 16.422 Decision Making Under Uncertainty 16.422 ? Alternatives are known – Consequences are not known – Probability of outcomes are not known ? Minimax/Maximin can still be used. ? Utility a key concept Swerve Don’t Swerve Swerve 0,0 10, 0 0, 10 Don’t Swerve -1000, -1000 Other Driver You Utility Theory 16.422 ? Utility theory is an attempt to infer subjective value, or utility, from choices. – Applies to both decision making under risk & decision making under uncertainty ? Two types – Expected utility ? Same as EV but utility is the value associated with some outcome, not necessarily monetary. ? Subjective Expected Utility (SEU) – Subjective focuses on decision making behavior ? Risk neutral/adverse/seeking – Multi-attribute utility ? Multiple objective extension of SEU ? Interval scale is critical MAUT Example ? Want to buy a car based on price, fuel economy, & reliability. ? Major drawback: Assumption of rationality 16.422 Price Fuel Reliability .7 156 Geo Metro 100 99 50 164.2 166.6 100 98 .5 20 60 Attribute Weight .8 BMW Z3 95 Subaru Outback 85 Uncertainty, Conditional Probability, & Bayes’ Theorem ? Bayes’ Theorem – P(A) - prior probability of A – P(A|B) - posterior probability – P(B|A), for a specific value of B, is the likelihood function for A – P(B) - the prior probability of B – P(A|B) ≠ P(B|A) 16.422 )A)P(A|P(B A)P(A)|P(B A)P(A)|P(B B)|P(A + = P(B) A)P(A)|P(B B)|P(A = Absolute versus conditional P(Tails) = .5 vs P(heart|red) = .5 Bayesian Networks ? Bayes nets are models which reflect the states of some part of a world and describe how those states are related by probabilities. – Causal chains ? If it might rain today and might rain tomorrow, what is the probability that it will rain on both days? – Not independent events with isolated probabilities. – Joint probabilities ? Useful for monitoring & alerting systems ? Problems – Can’t represent recursive systems in a straightforward manner – Acquiring probabilities – The number of paths to explore grows exponentially with each node 16.422 Good paper - Display of Information for Time-Critical (1995) Eric Horvitz, Matthew Barry Human probability estimation is unreliable for infrequent events (tails) References 16.422 ? Choices: An Introduction to Decision Theory, Michael D. Resnick (1987) ? Making Hard Decisions: An Introduction to Decision Analysis, 2nd ed., Robert T. Clemen, (1996) ? Decision Analysis, http://groups.msn.com/DecisionModeling/decision analysis1.msnw ? Bayes Net Tutorial http://www.norsys.com/tutorials/netica/nt_toc_A.htm