Modulation of Attention by Faces Expressing Emotion,
Evidence from Visual Marking
Fang Hao
1,2
,Hang Zhang
1,2
,and Xiaolan Fu
1
1
State Key Laboratory of Brain and Cognitive Science,
Institute of Psychology,Chinese Academy of Sciences,
Beijing,100101,China
{haof,zhangh,fuxl}@psych.ac.cn
2
Graduate School of the Chinese Academy of Sciences,Beijing,100049,China
Abstract,Recent findings demonstrated that negative emotional faces (sad,an-
ger or fear) tend to attract attention more than positive faces do,This study
used the paradigm of visual marking to test the perspective that mentioned and
explored whether the preview benefit still existed when using schematic faces
as materials,The results found that preview benefit was significant in the
search of affective materials,In a gap condition,it was faster to search negative
faces than to search positive faces,However,this advantage did not appear in
half-element condition when negative faces as distractors,which indicated that
the view that negative faces capture attention more efficiently is not always like
this,
1 Introduction
There has been a long history that researchers make study on attention by the para-
digm of visual search [15] [19],In recent years,researchers turned their interest to
emotion and facial expression by this classical paradigm and applied their research
results into facial expression aspect [2],They testified that different kinds of emotion
would play a distinct role in the search of facial expression,
The majority of the experiments that used the paradigm of visual search indicated
that negative faces capture attention more efficiently than positive faces do [3][6],
Hansen et al,(1988) showed that RTs (reaction times) in negative target-positive
distractors condition were faster than positive target-negative distractors condition,
which also reflected the asymmetry search of different valences of facial expression
[8],White (1995) received the similar conclusion instead of using schematic facial
expression as materials [18],Ohman et al (2001) found it was faster to search angry
faces or fear-relevant stimuli than to search positive faces [11],
The paradigm of visual search is especially well suited for examining whether fa-
cial expression can be perceived outside the focus of attention and can guide focal
attention,If a face expressing either positive or negative emotion is embedded among
different numbers of distractors and the time taken to locate the target face is meas-
ured,the relative contribution of unattended positive and negative facial expressions
in guiding focal attention can be assessed by comparing the performance of the search
[14],
Many experiments made schematic faces as materials to study emotion,Although
the schematic faces were impoverished as compared with photographs of real human
faces,they appeared to be potent affective facial stimuli,Schematic faces appeared to
communicate emotional meaning effectively [1] and showed disruptions in perception
when inverted similar to those found with photographed faces [5],Furthermore,it has
recently been shown that schematic faces elicit event-related potentials that are simi-
lar to those elicited by photographs of faces [4],Therefore,given that schematic faces
contain fewer feature confounds than photographs and appear to be effective affective
face stimuli,it may actually be preferable to use schematic faces in studies in which
the perception of facial expression is studied,
Visual marking,which is postulated recently,has more reasonably explained the
phenomena of prioritizing selection that occur in temporal asynchrony between two
groups of items in visual search [17],In this present experiment,we altered some
aspects of the standard paradigm and applied it into visual search of the emotional
faces,In visual marking,participants search for the presence or absence of a target
item among some distractor elements,There are three conditions,In a half-element
condition,the target differs from the distractors by the possession of a unique feature,
In an all-element condition,the other distrcators which had different features were
also presented besides the distractors in half-element condition,In a gap condition,
some items (old items) are presented for 1000ms,and then the other items (new items)
are displayed,If the target appears,it will only be presented in the new items [7] [12],
Researchers found that the search performance in the gap condition is as efficient as
the half-element condition in which old items are not displayed and more efficient
than the all-element condition in which two groups of items are presented synchro-
nously,This performance benefit has been called ‘preview benefit’ [17],
Detecting negative faces quickly and responding to them as efficiently as possible
has an evolutionary benefit that is argued to have formed the basis for selection [10],
In addition,visual system can anticipate the appearance of new visual information,It
would be adaptive to be able to selectively prioritize (or deprioritize) relevant objects,
even if those faces have yet to emerge,
The present study sought to test whether it was faster to search a negative face than
to search a positive face in the paradigm of visual marking [13],We also especially
wanted to know whether the negative faces captured more attention than the positive
faces when the negative faces were used to be target and distractors respectively,On
the other hand,the experimental materials were often letters in the study of visual
marking,It was important to use schematic faces that express emotion as materials
either because they accessed more to the reality or because they attributed to the re-
search of shape change in visual marking [16],Therefore,we also tested whether the
preview benefit still exist when using affective materials,
2 Method
2.1 Participants
Eleven undergraduates (5 men and 6 women) were paid to serve as participants,All
had self-reported normal or corrected-to-normal vision,
2.2 Stimuli and apparatus
All stimulus generation and response recording were conducted by Matlab 7.0 pro-
gram,Stimuli were randomly plotted in the cell of a 10× 10 virtual matrix,The over-
all matrix dimensions were 20cm wide× 20cm high,The visual search items con-
sisted of schematic faces of positive emotion (happy faces,see Fig,1,the left one),
neutral emotion(neutral faces,see Fig,1,the middle one) and negative emotion (sad
faces,see Fig,1,the right one) presented on a black background,The individual
stimuli faces were 18mm wide× 18mm high,In the all-element and gap conditions,
the display size was 6,8 or 10 items,An equal number of old items and new items
were always present in the display,The target was present on 50% of the trials,
Fig,1,The Example of the stimulus displays used in Experiment
2.3 Design and procedure
Each trial started with a white fixation cross in the center of the screen for 1000ms,
Following this,the search display was presented and remained until the participants
responded to either the presence or absence of the face that searched,After the par-
ticipants responded or did not respond after 3000ms,the display disappeared and a
new trial began,
There were three main conditions,half-element,all-element and gap conditions,
Every condition had six search types,The six search types of half-element condition
were as following,search a happy face from neutral faces or negative faces,search a
negative face from happy faces or neutral faces,and search a neutral face from happy
faces or negative faces,The search types of all-element condition were the same as
the half-element condition except that another kind of faces were also presented in the
display,such as searching a happy face from negative faces and neutral faces,and so
on; thus the display size was double that of the half-element condition,For the gap
condition,following the fixation cross,some faces were first displayed for 1000ms,
after which the other kinds of faces were added to the display,For example,some
happy faces appeared for 1000ms,and then a negative face and some neutral faces
were displayed,
All participants completed 18 blocks of trials lasting approximately 90min,with
block order counterbalanced across the participants,Each block had 60 trials,with an
equal number of present and absent trials within a block was presented in random
order,Before each block of trials,the participants received a short practice block,If
the participants had a low correct percent,they would have to engage the practice
again until they passed,
3 Results
Before the data analyse,error trials were first identified and then removed,Outliers,
identified as RTs of less than 250msec,and outside ± 3 SD,were removed from the
data,resulting in a loss of 1.18% of the trials,
In the data analyse,at first,we analyzed whether the preview benefit existed when
we searched the schematic faces and then analyzed the difference of the three kinds
of schematic faces in search in detail,
3.1 Preview benefit
Half-element condition,all-element condition and gap condition,The RTs for
correct trials for each of the three conditions were entered into a three-way within-
subject analysis of variance (ANOVA) with condition (half-element,all-element or
gap condition),target (present or absent) and display size (6,8,10) as the main vari-
ables,For the half-element condition,the display size was matched to those in the all-
element condition,
All of three main effects were significant,Condition,F(2,9) = 393.92,p < 0.001;
Target,F(1,10) = 398.62,p < 0.001; and Display size,F(2,9) = 193.11,p < 0.001,
Present trials were faster than absent trials,RTs increased as the display size in-
creased,and RTs were fastest in the half-element condition and slowest in the all-
element condition (see Fig,1.),
The two separate two-way within-subject ANOVAs showed the comparison of the
gap condition and each of the two other conditions,RTs in the gap condition and
half-element condition had no significant difference,F(1,10) = 1.437,p = 0.258,
which showed the performance in the gap condition was as efficient as the half-
element condition,RTs were faster in the gap condition than in the all-element condi-
tion,F(1,10) = 593.63,p < 0.001,which showed the performance in the gap condi-
tion was more efficient than the all-element condition,
Fig,2,Mean correct reaction times (RTs) as a function of condition and display size for pre-
sent trials (the left) and absent trials (the right),
3.2 The difference among happy faces,sad faces and neutral faces in visual
marking
Search a happy face,a sad face and a neutral face respectively,All of the three
main effects were significant,Face type,F(2,9) = 128.64,p < 0.001; Target,F(1,10)
= 381.90,p < 0.001; and Display Size,F(2,9) = 182.46,p < 0.001,RTs were fast-
est in the search of a neutral face and slowest in the search of a happy face,present
trials were faster than absent trials,and RTs increased as the display size increased,
There was also a significant Face Type× Display Size interaction,F(4,7) = 9.51,p
< 0.01,and Target× Display Size interaction,F(2,9) = 459.26,p < 0.001,No other
interactions were significant,
Search a happy face in sad faces versus search a sad face in happy faces,The
difference of the two kinds of search was significant,RTs in the search of a sad face
in happy faces were faster than in the search of a happy face in sad faces,F(1,10) =
8.69,p < 0.05,The results showed the search was asymmetry,
Search a happy face in neutral faces versus search a sad face in neutral faces,
The difference of the two kinds of search was significant,RTs in the search of a sad
face in some neutral faces were faster than in the search of a happy face,F(1,10) =
6.09,p < 0.05,
3.3 The analyses of the sad face as the target or distractors separately
At first,we concerned the condition that a sad face was used to be the target,When
the target was presented in gap condition (see Fig,3.),the difference of the search of
the three kinds of face was significant,F(2,9) = 135.59,p < 0.001,In addition,RTs
were faster in the search of a sad face than in the search of a happy face,F(1,10) =
22.31,p = 0.001,
On the other hand,we concerned the condition that sad faces were used to be dis-
tractors so that we analyzed the difference between searching a neutral face in sad
faces and in happy faces respectively,So we analyzed the half-element conditions
(see Fig,4.),Searching a neutral face in sad faces was faster than searching a neutral
face in happy faces,F (1,10) = 6.82,p < 0.05,
Fig,3,Mean correct reaction times (RTs) as a function of three kinds of faces in gap condition
and display size for present trials (the left) and absent trials (the right),
Fig,4,Mean correct reaction times (RTs) as a function of three kinds of faces in half-element
condition and display size for present trials (the left) and absent trials (the right),
3 Discussion
The present experiment showed that the performance in gap condition was as effi-
cient as the half-element condition and more efficient than all-element condition,
which proved that the preview benefit still existed when using affective materials,
When a sad face was used as the target,it was faster to search a sad face than to
search a happy face,However,when sad faces were used to be distractors,they did
not slow the RTs of searching a neutral face,
The results showed that it was faster when searching a sad face than searching a
happy face in gap condition,However,in half-element condition,when sad faces and
happy faces were both to be used as distractors,it was faster when searching a neutral
face in sad faces than in happy faces,Mack and Rock (1998) found the participants
detect the presence of a positive face more easily than detect a negative face,By this
view,it seemed suspectable to draw a conclusion that negative faces capture attention
more effectively than positive faces,One possible explanation is that the results are
different in different search condition and different tasks [4],
The results showed that it was usually faster to search a neutral face than to search
a happy face or a sad face,This difference might be induced by the local feature of
the mouth in the faces (see Fig,1.),However,Eastwood et al,(2001) approved that
the differential guidance of focal attention was due to holistic face perception instead
of part differences [4],
In gap condition,the preview benefit still existed when using the emotional faces
as experimental materials and distractors could be effectively and efficiently ignored
when they preceded by a preview display,In the all-element condition,the additional
set of faces distractors affected search to the extent that it became slower than in the
half-element condition,in which only two kinds of faces were present,In gap condi-
tion,search performance was as efficient as the half-element condition,indicating
that the previously presented faces had no influence on search efficiency,We could
confidently conclude that the present combination of stimuli and compound search
task provides a solid basis for the occurrence of the preview benefit [9],In addition,
the results of this experiment provided more materials to the research of visual mark-
ing which has few kinds of figures as materials in the past and attributed to the influ-
ence of the shape change of stimulus on the preview benefit [16],
In summary,the present findings extended previous studies of the deployment of
attention to emotionally expressive faces,while also contributing to our understand-
ing of visual marking and attention,Future studies need to explore more specifically
the constriction of attention by negative faces and positive faces,
References
1,Aronoff,J.,Barclay,A,M.,Stevenson,L,A.,The recognition of threatening facial stimuli,
Journal of Personality & Social Psychology,54 (1988) 647-655
2,Dolan,R,J.,Emotion,cognition and behavior,Science,298 (2002) 1191-1194
3,Eastwood,J,D.,Smilek,D.,Merikle,P,M.,Negative facial expression captures attention
and disrupts performance,Perception & Psychophysics,65 (2003) 352-358
4,Eastwood,J.,Smilek,D.,Merikle,P,M.,Differential attentional guidance by unattended
faces expressing positive and negative emotion,Perception & Pshchophysics,63 (2001)
1004-1013
5,Farah,M,J.,Wilson,K,D.,Drain,M.,Tanaka,J,N.,What is,special” about face percep-
tion? Psychological Review,105 (1998) 482-498
6,Fenske,M,J.,Eastwood,J,D.,Modulation of focused attention by faces expressing emotion,
evidence from flanker tasks,Emotion,3 (2003) 327-343
7,Gibson,B.,S.,Jiang,Y.,Visual marking and the perception of salience in visual search,
Perception & Psychophysics,63 (2001) 59-73
8,Hansen,C,N.,Hansen,R,D.,Finding the face in the crowd,an anger superiority effect,
Journal of Personality & Social Psychology,54 (1988) 917-924
9,Kunar M A,Humphreys G W,Smith K J,etc,When a re-appearance is old news,visual
marking survives occlusion,Journal of Experimental Psychology,Human Perception and
Performance,29 (2003) 185-198
10,Lipp,O,V.,Derakshan,N.,Waters,A,M.,Logies,S.,Snakes and cats in the flower bed,
fast detection is not specific to pictures of fear-relevant animals,Emotion,4 (2004) 233-250
11,Ohman,A.,Lundqvist,D.,Esteves,F.,The face in the crowd revisited,A threat advantage
with schematic stimuli,Journal of Personality & Social Psychology,80 (2001) 381–396
12,Olivers,C,N,L.,& Humphreys,G,W.,Visual marking inhibits singleton capture,Cogni-
tive Psychology,47 (2003),1-42
13,Olivers,C,N,L.,& Humphreys,G,W.,When visual marking meets the attentional blink,
more evidence for top-down,limited-capacity inhibition,Journal of Experimental Psychol-
ogy,Human Perception and Performance,28 (2002) 22-42
14,Smilek,D.,Eastwood,J,D.,Merikle,P.,Does unattended information facilitate change
detection? Journal of Experimental Psychology,Human Perception & Performance,26
(2000) 480-487
15,Treisman,A.,Souther,J.,Search asymmetry,a diagnostic for preattentive processing of
separable features,Journal of Experimental Psychology,General,114 (1985) 285-310
16,Watson D G,Humphreys G W,Visual marking and visual change,Journal of Experimental
Psychology,Human Perception and Performance,28 (2002) 379-395
17,Watson,D,G.,Humphreys,G,W.,Visual marking,prioritizing selection for new objects
by top-down attentional inhibition of old objects,Psychological Review,104 (1997) 90-122
18,White,M.,Preattentive analysis of facial expressions of emotion,Cognition & Emotion,9
(1995) 439-460
19,Wolfe,J.,Asymmetries in visual search,an introduction,Perception & Psychophysics,63
(2001) 381-389
Acknowledgement
This research was supported in part by grants from 973 Program of Chinese Ministry
of Science and Technology (2002CB312103),from the National Natural Science
Foundation of China (60433030 and 30270466),and from the Chinese Academy of
Sciences (0302037),