Economics 20 - Prof,Anderson 1
Multiple Regression Analysis
y = b0 + b1x1 + b2x2 +,,, bkxk + u
3,Asymptotic Properties
Economics 20 - Prof,Anderson 2
Consistency
Under the Gauss-Markov assumptions OLS
is BLUE,but in other cases it won’t always
be possible to find unbiased estimators
In those cases,we may settle for estimators
that are consistent,meaning as n ? ∞,the
distribution of the estimator collapses to the
parameter value
Economics 20 - Prof,Anderson 3
Sampling Distributions as n ?
b1
n1
n2
n3 n
1 < n2 < n3
Economics 20 - Prof,Anderson 4
Consistency of OLS
Under the Gauss-Markov assumptions,the
OLS estimator is consistent (and unbiased)
Consistency can be proved for the simple
regression case in a manner similar to the
proof of unbiasedness
Will need to take probability limit (plim) to
establish consistency
Economics 20 - Prof,Anderson 5
Proving Consistency
? ?? ? ? ?? ?
? ?? ? ? ?? ?
? ? ? ?
? ? 0,b e c a u s e
,
?
p l im
?
1
11111
2
11
1
11
1
1
2
11111
?
???
????
???
??
??
??
uxC o v
xV a ruxC o v
xxnuxxn
xxyxx
iii
iii
bbb
b
b
Economics 20 - Prof,Anderson 6
A Weaker Assumption
For unbiasedness,we assumed a zero
conditional mean – E(u|x1,x2,…,xk) = 0
For consistency,we can have the weaker
assumption of zero mean and zero
correlation – E(u) = 0 and Cov(xj,u) = 0,for
j = 1,2,…,k
Without this assumption,OLS will be
biased and inconsistent!
Economics 20 - Prof,Anderson 7
Deriving the Inconsistency
Just as we could derive the omitted variable bias
earlier,now we want to think about the
inconsistency,or asymptotic bias,in this case
? ? ? ?
121
21122
110
22110
,w h e r e
~
p l i m a n d,
t h a tso,:Y o u t h i n k
,m o d e l T r u e
xV a rxxC o v
vxu
uxy
vxxy
?
????
???
????
?
?bbbb
bb
bbb
Economics 20 - Prof,Anderson 8
Asymptotic Bias (cont)
So,thinking about the direction of the
asymptotic bias is just like thinking about
the direction of bias for an omitted variable
Main difference is that asymptotic bias uses
the population variance and covariance,
while bias uses the sample counterparts
Remember,inconsistency is a large sample
problem – it doesn’t go away as add data
Economics 20 - Prof,Anderson 9
Large Sample Inference
Recall that under the CLM assumptions,
the sampling distributions are normal,so we
could derive t and F distributions for testing
This exact normality was due to assuming
the population error distribution was normal
This assumption of normal errors implied
that the distribution of y,given the x’s,was
normal as well
Economics 20 - Prof,Anderson 10
Large Sample Inference (cont)
Easy to come up with examples for which
this exact normality assumption will fail
Any clearly skewed variable,like wages,
arrests,savings,etc,can’t be normal,since a
normal distribution is symmetric
Normality assumption not needed to
conclude OLS is BLUE,only for inference
Economics 20 - Prof,Anderson 11
Central Limit Theorem
Based on the central limit theorem,we can show
that OLS estimators are asymptotically normal
Asymptotic Normality implies that P(Z<z)?F(z)
as n ??,or P(Z<z) ? F(z)
The central limit theorem states that the
standardized average of any population with mean
m and variance s2 is asymptotically ~N(0,1),or
? ?1,0~ N
n
YZ aY
s
m??
Economics 20 - Prof,Anderson 12
Asymptotic Normality ? ?
? ?
? ?
? ? ? ? ? ?1,0N o r m a l ~?? ( i i i )
ofe s t i m a t o r c o n s i s t e n t a is ? ( i i )
?p l i m w h e r e
,,0N o r m a l ~
?
( i )
s,a s s u m p t i o n M a r k o v-G a u s s U n d e r t h e
22
212
22
a
jjj
ijj
j
a
jj
se
rna
an
bbb
ss
sbb
?
?
?
?
?
Economics 20 - Prof,Anderson 13
Asymptotic Normality (cont)
Because the t distribution approaches the
normal distribution for large df,we can also
say that
? ? ? ? 1~?? ??? knajjj tse bbb
Note that while we no longer need to
assume normality with a large sample,we
do still need homoskedasticity
Economics 20 - Prof,Anderson 14
Asymptotic Standard Errors
If u is not normally distributed,we sometimes
will refer to the standard error as an asymptotic
standard error,since
? ?
? ?
? ? ncse
RS S T
se
jj
jj
j
?
?
?
b
s
b
?
,
1
?
?
2
2
So,we can expect standard errors to shrink at a
rate proportional to the inverse of √n
Economics 20 - Prof,Anderson 15
Lagrange Multiplier statistic
Once we are using large samples and
relying on asymptotic normality for
inference,we can use more that t and F stats
The Lagrange multiplier or LM statistic is
an alternative for testing multiple exclusion
restrictions
Because the LM statistic uses an auxiliary
regression it’s sometimes called an nR2 stat
Economics 20 - Prof,Anderson 16
LM Statistic (cont)
Suppose we have a standard model,y = b0 + b1x1
+ b2x2 +,,, bkxk + u and our null hypothesis is
H0,bk-q+1 = 0,...,bk = 0
First,we just run the restricted model
r e g t h i sf r o m is w h e r e,
) v a r ia b le s t h e ( i,e,,...,,on
~
r e g r e s s a n d,
~
r e s i d u a ls,t h e t a k eN o w
~~
...
~~
22
21
110
uu
k
qkqk
RnRLM
a llxxxu
u
uxxy
?
?????
??
bbb
Economics 20 - Prof,Anderson 17
LM Statistic (cont)
2
2
2
f o r v a lu e-p a c a lc u la t eju s t
or on,d is t r ib u t i a f r o m,v a lu e,
c r it ic a l a c h o o s ec a n so,~
q
q
q
a
c
LM
?
?
?
With a large sample,the result from an F test and
from an LM test should be similar
Unlike the F test and t test for one exclusion,the
LM test and F test will not be identical
Economics 20 - Prof,Anderson 18
Asymptotic Efficiency
Estimators besides OLS will be consistent
However,under the Gauss-Markov
assumptions,the OLS estimators will have
the smallest asymptotic variances
We say that OLS is asymptotically efficient
Important to remember our assumptions
though,if not homoskedastic,not true