Economics 20 - Prof,Anderson 1
Instrumental Variables & 2SLS
y = b0 + b1x1 + b2x2 +,,, bkxk + u
x1 = p0 + p1z + p2x2 +,,, pkxk + v
Economics 20 - Prof,Anderson 2
Why Use Instrumental Variables?
Instrumental Variables (IV) estimation is
used when your model has endogenous x’s
That is,whenever Cov(x,u) ≠ 0
Thus,IV can be used to address the
problem of omitted variable bias
Additionally,IV can be used to solve the
classic errors-in-variables problem
Economics 20 - Prof,Anderson 3
What Is an Instrumental Variable?
In order for a variable,z,to serve as a valid
instrument for x,the following must be true
The instrument must be exogenous
That is,Cov(z,u) = 0
The instrument must be correlated with the
endogenous variable x
That is,Cov(z,x) ≠ 0
Economics 20 - Prof,Anderson 4
More on Valid Instruments
We have to use common sense and
economic theory to decide if it makes sense
to assume Cov(z,u) = 0
We can test if Cov(z,x) ≠ 0
Just testing H0,p1 = 0 in x = p0 + p1z + v
Sometimes refer to this regression as the
first-stage regression
Economics 20 - Prof,Anderson 5
IV Estimation in the Simple
Regression Case
For y = b0 + b1x + u,and given our
assumptions
Cov(z,y) = b1Cov(z,x) + Cov(z,u),so
b1 = Cov(z,y) / Cov(z,x)
Then the IV estimator for b1 is
? ?? ?
? ?? ??
?
??
??
?
xxzz
yyzz
ii
ii
1
?b
Economics 20 - Prof,Anderson 6
Inference with IV Estimation
The homoskedasticity assumption in this case is
E(u2|z) = s2 = Var(u)
As in the OLS case,given the asymptotic
variance,we can estimate the standard error
? ?
? ?
2
,
2
1
2
,
2
2
1
?
?
?
zxx
zxx
RS S T
se
n
V a r
s
b
?s
s
b
?
?
Economics 20 - Prof,Anderson 7
IV versus OLS estimation
Standard error in IV case differs from OLS
only in the R2 from regressing x on z
Since R2 < 1,IV standard errors are larger
However,IV is consistent,while OLS is
inconsistent,when Cov(x,u) ≠ 0
The stronger the correlation between z and
x,the smaller the IV standard errors
Economics 20 - Prof,Anderson 8
The Effect of Poor Instruments
What if our assumption that Cov(z,u) = 0 is false?
The IV estimator will be inconsistent,too
Can compare asymptotic bias in OLS and IV
Prefer IV if Corr(z,u)/Corr(z,x) < Corr(x,u)
x
u
x
u
uxC o r r
xzC o r r
uzC o r r
s
s
bb
s
s
bb
???
???
),(
~
p lim,O L S
),(
),(?
p lim,IV
11
11
Economics 20 - Prof,Anderson 9
IV Estimation in the Multiple
Regression Case
IV estimation can be extended to the
multiple regression case
Call the model we are interested in
estimating the structural model
Our problem is that one or more of the
variables are endogenous
We need an instrument for each
endogenous variable
Economics 20 - Prof,Anderson 10
Multiple Regression IV (cont)
Write the structural model as y1 = b0 + b1y2
+ b2z1 + u1,where y2 is endogenous and z1
is exogenous
Let z2 be the instrument,so Cov(z2,u1) = 0
and
y2 = p0 + p1z1 + p2z2 + v2,where p2 ≠ 0
This reduced form equation regresses the
endogenous variable on all exogenous ones
Economics 20 - Prof,Anderson 11
Two Stage Least Squares (2SLS)
It’s possible to have multiple instruments
Consider our original structural model,and
let y2 = p0 + p1z1 + p2z2 + p3z3 + v2
Here we’re assuming that both z2 and z3 are
valid instruments – they do not appear in
the structural model and are uncorrelated
with the structural error term,u1
Economics 20 - Prof,Anderson 12
Best Instrument
Could use either z2 or z3 as an instrument
The best instrument is a linear combination
of all of the exogenous variables,y2* = p0 +
p1z1 + p2z2 + p3z3
We can estimate y2* by regressing y2 on z1,
z2 and z3 – can call this the first stage
If then substitute ?2 for y2 in the structural
model,get same coefficient as IV
Economics 20 - Prof,Anderson 13
More on 2SLS
While the coefficients are the same,the
standard errors from doing 2SLS by hand
are incorrect,so let Stata do it for you
Method extends to multiple endogenous
variables – need to be sure that we have at
least as many excluded exogenous variables
(instruments) as there are endogenous
variables in the structural equation
Economics 20 - Prof,Anderson 14
Addressing Errors-in-Variables
with IV Estimation
Remember the classical errors-in-variables
problem where we observe x1 instead of x1*
Where x1 = x1* + e1,and e1 is uncorrelated
with x1* and x2
If there is a z,such that Corr(z,u) = 0 and
Corr(z,x1) ≠ 0,then
IV will remove the attenuation bias
Economics 20 - Prof,Anderson 15
Testing for Endogeneity
Since OLS is preferred to IV if we do not
have an endogeneity problem,then we’d
like to be able to test for endogeneity
If we do not have endogeneity,both OLS
and IV are consistent
Idea of Hausman test is to see if the
estimates from OLS and IV are different
Economics 20 - Prof,Anderson 16
Testing for Endogeneity (cont)
While it’s a good idea to see if IV and OLS
have different implications,it’s easier to use
a regression test for endogeneity
If y2 is endogenous,then v2 (from the
reduced form equation) and u1 from the
structural model will be correlated
The test is based on this observation
Economics 20 - Prof,Anderson 17
Testing for Endogeneity (cont)
Save the residuals from the first stage
Include the residual in the structural
equation (which of course has y2 in it)
If the coefficient on the residual is
statistically different from zero,reject the
null of exogeneity
If multiple endogenous variables,jointly
test the residuals from each first stage
Economics 20 - Prof,Anderson 18
Testing Overidentifying
Restrictions
If there is just one instrument for our
endogenous variable,we can’t test whether
the instrument is uncorrelated with the error
We say the model is just identified
If we have multiple instruments,it is
possible to test the overidentifying
restrictions – to see if some of the
instruments are correlated with the error
Economics 20 - Prof,Anderson 19
The OverID Test
Estimate the structural model using IV and
obtain the residuals
Regress the residuals on all the exogenous
variables and obtain the R2 to form nR2
Under the null that all instruments are
uncorrelated with the error,LM ~ cq2 where
q is the number of extra instruments
Economics 20 - Prof,Anderson 20
Testing for Heteroskedasticity
When using 2SLS,we need a slight
adjustment to the Breusch-Pagan test
Get the residuals from the IV estimation
Regress these residuals squared on all of
the exogenous variables in the model
(including the instruments)
Test for the joint significance
Economics 20 - Prof,Anderson 21
Testing for Serial Correlation
When using 2SLS,we need a slight
adjustment to the test for serial correlation
Get the residuals from the IV estimation
Re-estimate the structural model by 2SLS,
including the lagged residuals,and using the
same instruments as originally
Can do 2SLS on a quasi-differenced model,
using quasi-differenced instruments
Instrumental Variables & 2SLS
y = b0 + b1x1 + b2x2 +,,, bkxk + u
x1 = p0 + p1z + p2x2 +,,, pkxk + v
Economics 20 - Prof,Anderson 2
Why Use Instrumental Variables?
Instrumental Variables (IV) estimation is
used when your model has endogenous x’s
That is,whenever Cov(x,u) ≠ 0
Thus,IV can be used to address the
problem of omitted variable bias
Additionally,IV can be used to solve the
classic errors-in-variables problem
Economics 20 - Prof,Anderson 3
What Is an Instrumental Variable?
In order for a variable,z,to serve as a valid
instrument for x,the following must be true
The instrument must be exogenous
That is,Cov(z,u) = 0
The instrument must be correlated with the
endogenous variable x
That is,Cov(z,x) ≠ 0
Economics 20 - Prof,Anderson 4
More on Valid Instruments
We have to use common sense and
economic theory to decide if it makes sense
to assume Cov(z,u) = 0
We can test if Cov(z,x) ≠ 0
Just testing H0,p1 = 0 in x = p0 + p1z + v
Sometimes refer to this regression as the
first-stage regression
Economics 20 - Prof,Anderson 5
IV Estimation in the Simple
Regression Case
For y = b0 + b1x + u,and given our
assumptions
Cov(z,y) = b1Cov(z,x) + Cov(z,u),so
b1 = Cov(z,y) / Cov(z,x)
Then the IV estimator for b1 is
? ?? ?
? ?? ??
?
??
??
?
xxzz
yyzz
ii
ii
1
?b
Economics 20 - Prof,Anderson 6
Inference with IV Estimation
The homoskedasticity assumption in this case is
E(u2|z) = s2 = Var(u)
As in the OLS case,given the asymptotic
variance,we can estimate the standard error
? ?
? ?
2
,
2
1
2
,
2
2
1
?
?
?
zxx
zxx
RS S T
se
n
V a r
s
b
?s
s
b
?
?
Economics 20 - Prof,Anderson 7
IV versus OLS estimation
Standard error in IV case differs from OLS
only in the R2 from regressing x on z
Since R2 < 1,IV standard errors are larger
However,IV is consistent,while OLS is
inconsistent,when Cov(x,u) ≠ 0
The stronger the correlation between z and
x,the smaller the IV standard errors
Economics 20 - Prof,Anderson 8
The Effect of Poor Instruments
What if our assumption that Cov(z,u) = 0 is false?
The IV estimator will be inconsistent,too
Can compare asymptotic bias in OLS and IV
Prefer IV if Corr(z,u)/Corr(z,x) < Corr(x,u)
x
u
x
u
uxC o r r
xzC o r r
uzC o r r
s
s
bb
s
s
bb
???
???
),(
~
p lim,O L S
),(
),(?
p lim,IV
11
11
Economics 20 - Prof,Anderson 9
IV Estimation in the Multiple
Regression Case
IV estimation can be extended to the
multiple regression case
Call the model we are interested in
estimating the structural model
Our problem is that one or more of the
variables are endogenous
We need an instrument for each
endogenous variable
Economics 20 - Prof,Anderson 10
Multiple Regression IV (cont)
Write the structural model as y1 = b0 + b1y2
+ b2z1 + u1,where y2 is endogenous and z1
is exogenous
Let z2 be the instrument,so Cov(z2,u1) = 0
and
y2 = p0 + p1z1 + p2z2 + v2,where p2 ≠ 0
This reduced form equation regresses the
endogenous variable on all exogenous ones
Economics 20 - Prof,Anderson 11
Two Stage Least Squares (2SLS)
It’s possible to have multiple instruments
Consider our original structural model,and
let y2 = p0 + p1z1 + p2z2 + p3z3 + v2
Here we’re assuming that both z2 and z3 are
valid instruments – they do not appear in
the structural model and are uncorrelated
with the structural error term,u1
Economics 20 - Prof,Anderson 12
Best Instrument
Could use either z2 or z3 as an instrument
The best instrument is a linear combination
of all of the exogenous variables,y2* = p0 +
p1z1 + p2z2 + p3z3
We can estimate y2* by regressing y2 on z1,
z2 and z3 – can call this the first stage
If then substitute ?2 for y2 in the structural
model,get same coefficient as IV
Economics 20 - Prof,Anderson 13
More on 2SLS
While the coefficients are the same,the
standard errors from doing 2SLS by hand
are incorrect,so let Stata do it for you
Method extends to multiple endogenous
variables – need to be sure that we have at
least as many excluded exogenous variables
(instruments) as there are endogenous
variables in the structural equation
Economics 20 - Prof,Anderson 14
Addressing Errors-in-Variables
with IV Estimation
Remember the classical errors-in-variables
problem where we observe x1 instead of x1*
Where x1 = x1* + e1,and e1 is uncorrelated
with x1* and x2
If there is a z,such that Corr(z,u) = 0 and
Corr(z,x1) ≠ 0,then
IV will remove the attenuation bias
Economics 20 - Prof,Anderson 15
Testing for Endogeneity
Since OLS is preferred to IV if we do not
have an endogeneity problem,then we’d
like to be able to test for endogeneity
If we do not have endogeneity,both OLS
and IV are consistent
Idea of Hausman test is to see if the
estimates from OLS and IV are different
Economics 20 - Prof,Anderson 16
Testing for Endogeneity (cont)
While it’s a good idea to see if IV and OLS
have different implications,it’s easier to use
a regression test for endogeneity
If y2 is endogenous,then v2 (from the
reduced form equation) and u1 from the
structural model will be correlated
The test is based on this observation
Economics 20 - Prof,Anderson 17
Testing for Endogeneity (cont)
Save the residuals from the first stage
Include the residual in the structural
equation (which of course has y2 in it)
If the coefficient on the residual is
statistically different from zero,reject the
null of exogeneity
If multiple endogenous variables,jointly
test the residuals from each first stage
Economics 20 - Prof,Anderson 18
Testing Overidentifying
Restrictions
If there is just one instrument for our
endogenous variable,we can’t test whether
the instrument is uncorrelated with the error
We say the model is just identified
If we have multiple instruments,it is
possible to test the overidentifying
restrictions – to see if some of the
instruments are correlated with the error
Economics 20 - Prof,Anderson 19
The OverID Test
Estimate the structural model using IV and
obtain the residuals
Regress the residuals on all the exogenous
variables and obtain the R2 to form nR2
Under the null that all instruments are
uncorrelated with the error,LM ~ cq2 where
q is the number of extra instruments
Economics 20 - Prof,Anderson 20
Testing for Heteroskedasticity
When using 2SLS,we need a slight
adjustment to the Breusch-Pagan test
Get the residuals from the IV estimation
Regress these residuals squared on all of
the exogenous variables in the model
(including the instruments)
Test for the joint significance
Economics 20 - Prof,Anderson 21
Testing for Serial Correlation
When using 2SLS,we need a slight
adjustment to the test for serial correlation
Get the residuals from the IV estimation
Re-estimate the structural model by 2SLS,
including the lagged residuals,and using the
same instruments as originally
Can do 2SLS on a quasi-differenced model,
using quasi-differenced instruments
