16.422
Human Supervisory Control
Function Allocation
and
Task Analysis
Human Systems Engineering
16.422
Planning →Analysis→Detail Design→Test & Evaluation
Functions & Tasks
16.422
Planning
Mission &
Scenario Analysis
Function
Analysis
Function
Allocation
Task
Analysis
System
Design
Analysis
Design Test &
Evaluation
Fitts’ List
16.422
Attribute Machine Human
Speed Superior Comparatively slow
Power
Output
Superior in level in consistency Comparatively weak
Consistency
Ideal for consistent, repetitive action Unreliable, learning & fatigue a
factor
Information
Capacity
Multi-channel Primarily single channel
Memory
Ideal for literal reproduction, access
restricted and formal
Better for principles & strategies,
access versatile & innovative
Reasoning
Computation
Deductive, tedious to program, fast
& accurate, poor error correction
Inductive, easier to program, slow,
accurate, good error correction
Sensing
Good at quantitative assessment,
poor at pattern recognition
Wide ranges, multi-function,
judgment
Perceiving
Copes with variation poorly,
susceptible to noise
Copes with variation better,
susceptible to noise
Hollnagel, 2000
inductive and deductive. Induction is usually described as moving from the specific to the general, while deduction begins with the general and ends with the specific; arguments based on
experience or observation are best expressed inductively, while arguments based on laws, rules, or other widely accepted principles are best expressed deductively.
Some problems with Fitts…
? Tasks/functions defined in machine terms, not
human-oriented
– Introduces a bias
– “Laws of human behavior”
? Environmental/ecologic context
? Learning, fatigue, stress, anxiety generally not
incorporated into design picture
? Task division vs. task complement
? Static vs. dynamic allocation
– Adaptive allocation/automation
– Function allocation is not binary
16.422
? Bandwidth
?Trust
? Machine/computer metaphors
Designing automation to support
information processing
16.422
Human
Sensory
Processing
Response
Selection
Decision
Making
Perception/
Working
Memory
Information
Acquisition
Action
Implementation
Decision
& Action
Selection
Information
Analysis
Automation
*Parasuraman, Sheridan, Wickens, 2000
A Model of
Types and
Levels of
Automation*
Information
Acquisition
Action
Implementation
Decision
& Action
Selection
Information
Analysis
What should be automated?
Identify types of automation
Identify levels of automation
Apply primary evaluative criteria:
Human Performance Consequences
? Mental workload
? Situation awareness
? Complacency
? Skill degradation
Initial types & levels of automation
Final types & levels of automation
Apply secondary evaluative criteria:
? Automation reliability
? Costs of action outcomes
Low (manual) High (full automation)
*Parasuraman,
Sheridan,
Wickens, 2000
Sheridan and Verplank’s 10 Levels of
Automation of Decision and Action Selection
16.422
Automation
Level
Automation Description
1
The computer offers no assistance: human must take all decision and actions.
2 The computer offers a complete set of decision/action alternatives, or
3 narrows the selection down to a few, or
4 suggests one alternative, and
5 executes that suggestion if the human approves, or
6
allows the human a restricted time to veto before automatic execution, or
7
executes automatically, then necessarily informs humans, and
8
informs the human only if asked, or
9
informs the human only if it, the computer, decides to.
10
The computer decides everything and acts autonomously, ignoring the human.
Information
Acquisition
Action
Implementation
Decision
& Action
Selection
Information
Analysis
What should be automated?
Identify types of automation
Identify levels of automation
Apply primary evaluative criteria:
Human Performance Consequences
? Mental workload
? Situation awareness
? Complacency
? Skill degradation
Initial types & levels of automation
Final types & levels of automation
Apply secondary evaluative criteria:
? Automation reliability
? Costs of action outcomes
Low (manual) High (full automation)
A Model of
Types and
Levels of
Automation*
*Parasuraman,
Sheridan,
Wickens, 2000
Function Allocation Criteria
16.422
1: No difference in the relative
capabilities of human & machine.
2: Human performance > machine
performance.
3: Machine performance > human.
4: Machine performance is so poor that
the functions should be allocated to
humans.
5: Human performance is so poor that
the functions should be allocated to
machine.
6: Unacceptable performance by both
human and machine.
Three function allocation criteria:
? Balance of value
? Utilitarian & cost-based allocation
? Allocation for affective or
cognitive support.
1
2
3
4
5
6
Unsatisfactory Excellent
Human
Unsatisf
actory
Machine
Excel
lent
Price, 1985
Functions & Tasks
16.422
Planning
Mission &
Scenario Analysis
Function
Analysis
Function
Allocation
Task
Analysis
System
Design
Analysis
Design Test &
Evaluation
Task Analysis
16.422
? Determining what an operator must accomplish to
meet a mission goal
– Interactions both on a local and system level are critical
– Will contain actions and/or cognitive processes
? Flow process charts, operational sequence diagrams,
critical task analysis
– Attempt to understand how a particular task could exceed
human limitations, both physical and cognitive
? Cognitive task analysis
– Not the only system analytic method but a critical one
– Shift away from system control to systems management.
Supervisory control becoming more about cognitive tasking than manual tasking, especially with increasing automation
Cognitive Task Analysis (CTA)
16.422
? Goal: To analyze and represent the knowledge and cognitive
activities needed in complex work domains
? CTA is generally a descriptive modeling technique of workers’
knowledge and cognition
– As opposed to Computational Cognitive Models (CCM)
– Knowledge Elicitation techniques provide input to CTA and CCM
? Experts vs. Novices
? Evolutionary systems vs. revolutionary systems
? Background Research
– Standards, procedures, manuals, organizational charts
? Field Studies
– In both real environments and high fidelity simulations
? Questionnaires/Surveys
http://www.ul.ie/~infopolis/methods/incident.html
CTA, Cont.
? Interviews
– Individuals vs. focus groups
– Critical Incident Technique/Critical Decision Method
? Observations
– PARI Method (Precursor (reason for action), Action,
Result, Interpretation (of result))
– Verbal protocols
? Design Reviews
– Usability, Expert, Heuristic
? Problems with CTA
– Labor intensive
– Generate much data that is difficult to analyze
– Gap between CTA and design
– Opportunistic
16.422
CTA: A Bootstrapping Process
? critical decision method in which participants are asked to describe a specific decision-making incident in detail and then to respond to probes seeking elaboration of important aspects of the
decision sequences.
? Semantic mapping (a.k.a., mind-mapping, idea mapping, word webbing, etc.) is a term which describes a variety of strategies designed to show how key words or concepts are related to one
another through graphic representations. Mapping is an effective technique for teaching vocabulary and textual patterns of organization; and it is also effective for improving note taking and
creative thinking skills.
Goal :
Understand/model expertise, knowledge,
strategies, and error
Discovering support for how people
will operate in their world
Techniques :
Ethnographic / Observational investigations
Critical Incident Technique
Critical Decision Method
Structured Interview Techniques
Goal :
Understand/model complexities,
demands, variability, and complicating
factors
Techniques :
Scratch
CTA
Model
G
r
o
w
t
h
o
f
U
n
d
e
r
s
t
a
n
d
i
n
g
TimeTime
Discovering how to support the way the world
will workUnderstanding the way the world works
Design
Basis
Understanding the way people operate
in their world
Exploring the Current World
P
r
a
c
t
i
t
i
o
n
e
r
(
s
)
F
i
e
l
d
o
f
P
r
a
c
t
i
c
e
T
h
e
D
o
m
a
i
n
C
T
A
R
e
p
r
e
s
e
n
t
a
t
i
o
n
P
r
o
t
o
t
y
p
e
R
e
p
r
e
s
e
n
t
a
t
i
o
n
Exploring the Envisioned World
COGNITIVE TASK ANALYSIS
Adapted from Carnegie Group, Inc.
Semantic Mapping
Functional/Means-ends Analysis
Ethnographic / Observational investigations
Functional Task and Workflow Modeling
Structured Interview Techniques
Functional Means/Ends Analysis
16.422
Decomposition (Whole/Part)
Abstraction
(Means/ends)
Total System Subsystems Subassembly Component
Functional
Purpose
Abstract
Function
Generalized
Function
Physical
Function
Physical
Form
CTA: A Bootstrapping Process
WoZ: http://www.dcs.napier.ac.uk/marble/Usability/WizardOfOz.html
Scratch
G
r
o
w
t
h
o
f
U
n
d
e
r
s
t
a
n
d
i
n
g
TimeTime
Discovering how to support the way the world
will work
Understanding the way the world works
Design
Basis
Understanding the way people operate
in their world
Exploring the Current World
P
r
a
c
t
i
t
i
o
n
e
r
(
s
)
F
i
e
l
d
o
f
P
r
a
c
t
i
c
e
T
h
e
D
o
m
a
i
n
C
T
A
R
e
p
r
e
s
e
n
t
a
t
i
o
n
P
r
o
t
o
t
y
p
e
R
e
p
r
e
s
e
n
t
a
t
i
o
n
Exploring the Envisioned World
Adapted from Carnegie Group, Inc.
Goal:
Discovery of unsupported
expertise, knowledge,
and strategies
Techniques:
Storyboard walkthroughs
Participatory design
Wizard-of-Oz technique
High-fidelity simulations
Goal:
Understand/model expertise,
knowledge,
strategies, and error
Goal:
Techniques:
Semantic Mapping
Ethnographic / Observational
investigations
Critical Incident Technique
Critical Decision Method
Structured Interview
Techniques
Understand/model
complexities,
demands, variability,
and complicating
factors
Techniques:
Functional/Means-ends Analysis
Ethnographic / Observational
investigations
Functional Task and Workflow
Modeling
Structured Interview Techniques
Goal:
Discovery of unsupported
complexities,
demands, variability, and
complicating factors
Techniques:
Scenario generation based on:
- Textbook cases
- Complicating factors
- Cascading effects
- Exceptions
Artifacts as
Hypotheses
Discovering support for how people
will operate in their world
COGNITIVE TASK ANALYSIS
Work Analysis v. Task Analysis
? Descriptive v. Normative v. Prescriptive
? Ecological focus
– Constraints v. instructions
– Map v. directions
? Not a mutually exclusive set
16.422
Criterion Task Analysis Work Domain
Analysis
Mental
Economy
Efficient Effortful
Unforeseen
Circumstances
Brittle
Flexible
Scope of
Applicability
Narrow Broad
normative analysis focuses on "how decision makers should ideally perform" an activity. Bell et al. (1988) have claimed that "normative theory has something to do with how idealized, rational,
super-intelligent people should think and should act." Normative analysis is often contrasted with prescriptive analysis, which is usually said to be geared toward examining what real people ought
to do given their real-world constraints and cognitive limitations (or how decision aids might aid real decision makers).
Resources
16.422
? A Survey of Cognitive Engineering Methods
and Uses
– http://mentalmodels.mitre.org/cog_eng/index.htm
? ONR/Aptima Cognitive Task Analysis
website
– http://www.ctaresource.com/