16.422 Human Supervisory Control Function Allocation and Task Analysis Human Systems Engineering 16.422 Planning →Analysis→Detail Design→Test & Evaluation Functions & Tasks 16.422 Planning Mission & Scenario Analysis Function Analysis Function Allocation Task Analysis System Design Analysis Design Test & Evaluation Fitts’ List 16.422 Attribute Machine Human Speed Superior Comparatively slow Power Output Superior in level in consistency Comparatively weak Consistency Ideal for consistent, repetitive action Unreliable, learning & fatigue a factor Information Capacity Multi-channel Primarily single channel Memory Ideal for literal reproduction, access restricted and formal Better for principles & strategies, access versatile & innovative Reasoning Computation Deductive, tedious to program, fast & accurate, poor error correction Inductive, easier to program, slow, accurate, good error correction Sensing Good at quantitative assessment, poor at pattern recognition Wide ranges, multi-function, judgment Perceiving Copes with variation poorly, susceptible to noise Copes with variation better, susceptible to noise Hollnagel, 2000 inductive and deductive. Induction is usually described as moving from the specific to the general, while deduction begins with the general and ends with the specific; arguments based on experience or observation are best expressed inductively, while arguments based on laws, rules, or other widely accepted principles are best expressed deductively. Some problems with Fitts… ? Tasks/functions defined in machine terms, not human-oriented – Introduces a bias – “Laws of human behavior” ? Environmental/ecologic context ? Learning, fatigue, stress, anxiety generally not incorporated into design picture ? Task division vs. task complement ? Static vs. dynamic allocation – Adaptive allocation/automation – Function allocation is not binary 16.422 ? Bandwidth ?Trust ? Machine/computer metaphors Designing automation to support information processing 16.422 Human Sensory Processing Response Selection Decision Making Perception/ Working Memory Information Acquisition Action Implementation Decision & Action Selection Information Analysis Automation *Parasuraman, Sheridan, Wickens, 2000 A Model of Types and Levels of Automation* Information Acquisition Action Implementation Decision & Action Selection Information Analysis What should be automated? Identify types of automation Identify levels of automation Apply primary evaluative criteria: Human Performance Consequences ? Mental workload ? Situation awareness ? Complacency ? Skill degradation Initial types & levels of automation Final types & levels of automation Apply secondary evaluative criteria: ? Automation reliability ? Costs of action outcomes Low (manual) High (full automation) *Parasuraman, Sheridan, Wickens, 2000 Sheridan and Verplank’s 10 Levels of Automation of Decision and Action Selection 16.422 Automation Level Automation Description 1 The computer offers no assistance: human must take all decision and actions. 2 The computer offers a complete set of decision/action alternatives, or 3 narrows the selection down to a few, or 4 suggests one alternative, and 5 executes that suggestion if the human approves, or 6 allows the human a restricted time to veto before automatic execution, or 7 executes automatically, then necessarily informs humans, and 8 informs the human only if asked, or 9 informs the human only if it, the computer, decides to. 10 The computer decides everything and acts autonomously, ignoring the human. Information Acquisition Action Implementation Decision & Action Selection Information Analysis What should be automated? Identify types of automation Identify levels of automation Apply primary evaluative criteria: Human Performance Consequences ? Mental workload ? Situation awareness ? Complacency ? Skill degradation Initial types & levels of automation Final types & levels of automation Apply secondary evaluative criteria: ? Automation reliability ? Costs of action outcomes Low (manual) High (full automation) A Model of Types and Levels of Automation* *Parasuraman, Sheridan, Wickens, 2000 Function Allocation Criteria 16.422 1: No difference in the relative capabilities of human & machine. 2: Human performance > machine performance. 3: Machine performance > human. 4: Machine performance is so poor that the functions should be allocated to humans. 5: Human performance is so poor that the functions should be allocated to machine. 6: Unacceptable performance by both human and machine. Three function allocation criteria: ? Balance of value ? Utilitarian & cost-based allocation ? Allocation for affective or cognitive support. 1 2 3 4 5 6 Unsatisfactory Excellent Human Unsatisf actory Machine Excel lent Price, 1985 Functions & Tasks 16.422 Planning Mission & Scenario Analysis Function Analysis Function Allocation Task Analysis System Design Analysis Design Test & Evaluation Task Analysis 16.422 ? Determining what an operator must accomplish to meet a mission goal – Interactions both on a local and system level are critical – Will contain actions and/or cognitive processes ? Flow process charts, operational sequence diagrams, critical task analysis – Attempt to understand how a particular task could exceed human limitations, both physical and cognitive ? Cognitive task analysis – Not the only system analytic method but a critical one – Shift away from system control to systems management. Supervisory control becoming more about cognitive tasking than manual tasking, especially with increasing automation Cognitive Task Analysis (CTA) 16.422 ? Goal: To analyze and represent the knowledge and cognitive activities needed in complex work domains ? CTA is generally a descriptive modeling technique of workers’ knowledge and cognition – As opposed to Computational Cognitive Models (CCM) – Knowledge Elicitation techniques provide input to CTA and CCM ? Experts vs. Novices ? Evolutionary systems vs. revolutionary systems ? Background Research – Standards, procedures, manuals, organizational charts ? Field Studies – In both real environments and high fidelity simulations ? Questionnaires/Surveys http://www.ul.ie/~infopolis/methods/incident.html CTA, Cont. ? Interviews – Individuals vs. focus groups – Critical Incident Technique/Critical Decision Method ? Observations – PARI Method (Precursor (reason for action), Action, Result, Interpretation (of result)) – Verbal protocols ? Design Reviews – Usability, Expert, Heuristic ? Problems with CTA – Labor intensive – Generate much data that is difficult to analyze – Gap between CTA and design – Opportunistic 16.422 CTA: A Bootstrapping Process ? critical decision method in which participants are asked to describe a specific decision-making incident in detail and then to respond to probes seeking elaboration of important aspects of the decision sequences. ? Semantic mapping (a.k.a., mind-mapping, idea mapping, word webbing, etc.) is a term which describes a variety of strategies designed to show how key words or concepts are related to one another through graphic representations. Mapping is an effective technique for teaching vocabulary and textual patterns of organization; and it is also effective for improving note taking and creative thinking skills. Goal : Understand/model expertise, knowledge, strategies, and error Discovering support for how people will operate in their world Techniques : Ethnographic / Observational investigations Critical Incident Technique Critical Decision Method Structured Interview Techniques Goal : Understand/model complexities, demands, variability, and complicating factors Techniques : Scratch CTA Model G r o w t h o f U n d e r s t a n d i n g TimeTime Discovering how to support the way the world will workUnderstanding the way the world works Design Basis Understanding the way people operate in their world Exploring the Current World P r a c t i t i o n e r ( s ) F i e l d o f P r a c t i c e T h e D o m a i n C T A R e p r e s e n t a t i o n P r o t o t y p e R e p r e s e n t a t i o n Exploring the Envisioned World COGNITIVE TASK ANALYSIS Adapted from Carnegie Group, Inc. Semantic Mapping Functional/Means-ends Analysis Ethnographic / Observational investigations Functional Task and Workflow Modeling Structured Interview Techniques Functional Means/Ends Analysis 16.422 Decomposition (Whole/Part) Abstraction (Means/ends) Total System Subsystems Subassembly Component Functional Purpose Abstract Function Generalized Function Physical Function Physical Form CTA: A Bootstrapping Process WoZ: http://www.dcs.napier.ac.uk/marble/Usability/WizardOfOz.html Scratch G r o w t h o f U n d e r s t a n d i n g TimeTime Discovering how to support the way the world will work Understanding the way the world works Design Basis Understanding the way people operate in their world Exploring the Current World P r a c t i t i o n e r ( s ) F i e l d o f P r a c t i c e T h e D o m a i n C T A R e p r e s e n t a t i o n P r o t o t y p e R e p r e s e n t a t i o n Exploring the Envisioned World Adapted from Carnegie Group, Inc. Goal: Discovery of unsupported expertise, knowledge, and strategies Techniques: Storyboard walkthroughs Participatory design Wizard-of-Oz technique High-fidelity simulations Goal: Understand/model expertise, knowledge, strategies, and error Goal: Techniques: Semantic Mapping Ethnographic / Observational investigations Critical Incident Technique Critical Decision Method Structured Interview Techniques Understand/model complexities, demands, variability, and complicating factors Techniques: Functional/Means-ends Analysis Ethnographic / Observational investigations Functional Task and Workflow Modeling Structured Interview Techniques Goal: Discovery of unsupported complexities, demands, variability, and complicating factors Techniques: Scenario generation based on: - Textbook cases - Complicating factors - Cascading effects - Exceptions Artifacts as Hypotheses Discovering support for how people will operate in their world COGNITIVE TASK ANALYSIS Work Analysis v. Task Analysis ? Descriptive v. Normative v. Prescriptive ? Ecological focus – Constraints v. instructions – Map v. directions ? Not a mutually exclusive set 16.422 Criterion Task Analysis Work Domain Analysis Mental Economy Efficient Effortful Unforeseen Circumstances Brittle Flexible Scope of Applicability Narrow Broad normative analysis focuses on "how decision makers should ideally perform" an activity. Bell et al. (1988) have claimed that "normative theory has something to do with how idealized, rational, super-intelligent people should think and should act." Normative analysis is often contrasted with prescriptive analysis, which is usually said to be geared toward examining what real people ought to do given their real-world constraints and cognitive limitations (or how decision aids might aid real decision makers). Resources 16.422 ? A Survey of Cognitive Engineering Methods and Uses – http://mentalmodels.mitre.org/cog_eng/index.htm ? ONR/Aptima Cognitive Task Analysis website – http://www.ctaresource.com/