Chapter 7 Multiple Regression,
－－ Estimation and Hypothesis Testing
Multiple Regression Model,A regression
model with more than one explanatory
variable,multiple because multiple
influences (i.e.,variables)affect the
dependent variable.
7.1 The Three-variable Linear Regression Model
three-variable PRF:
nonstochastic form,E(Yt)=B1+B2X2t+B3X3t (7.1)
stochastic form,Yt=B1+B2X2t+B3X3t+ut (7.2)
= E(Yt)+ut
B2,B3～ partial regressioncoefficients,partial slope coefficient
B2,the change in the mean value of Y,E(Y),per unit change in X2,
holding the value of X3 constant.
B3,the change in the mean value of Y per unit change in X3,
holding the value of X2 constant.
7.2 Assumptions of Multiple Linear Regression
Model
A7.1,X2 and X3 are uncorrelated with the disturbance term u.
A7.2,The error term u has a zero mean value E(ui)=0 (7.7)
A7.3,Homoscedasticity,that is,the variance of u,is constant:
var(ui)=σ2 （ 7.8）
A7.6,For hypothesis testing,the error term u follows the
normal distribution with mean zero and (homoscedastic)
variance σ2, That is,ui ~N(0,σ2 ) (7.10 )
A7.4,No autocorrelation exists between the error terms ui and uj,
cov(ui,uj) i≠j (7.9)
A7.5,No exact collinearity exists between X2 and X3 ; that is,
there is no exact linear relationship between the two
explanatory variables.
－－ no collinearity,orno multicollinearity,
① exact linear relationship
② high or near perfect collinearity
7.3 Estimation of Parameters of Multiple
Regression
7.3.1 Ordinary Least Squares（ OLS） Estimators
SRF,Stochastic form,Yt=b1+b2X2t+b3X3t+et (7.13)
Nonstochastic form,=b1+b2X2t+b3X3t (7.14)
et=Yt－
et=Yt –b1－ b2X2t+b3X3t (7.15)
RSS,(7.16)
OLS Estimators:
b1= (7.20)
b2= (7.21)
b3= (7.22)
tY?
23t32t21t2t )XbXbbY(e ???? ??
3322 XbXbY ??
2
3t2t
2
3t
2
2t
3t2t3tt
2
3t2t
)xx()x)(x(
)xx)(xy)(x)(txy(
???
????
?
2
3t2t
2
3t
2
2t
3t2t3tt
2
3t2tt
)xx()x)(x(
)xx)(xy()x)(xy(
??? ???? ?
?
tY?
7.3.2 Variance and Standard Errors of OLS Estimators
We need the standard errors for two main purposes:
(1) to establish confidence intervals for the true parameter values
(2) to test statistical hypotheses.
1,ui ~N(0,σ2 ) →b1～ N（ B1,var(b1)）
b2～ N（ B2,var(b2)）
b3～ N（ B3,var(b3)）
var(b1)= · (7.23)
se(b1)= (7.24)
var(b2)= · (7.25)
se(b2)= (7.26)
var(b3)= · (7.27)
se(b3)= (7.28)
? ? ? ?? ? ?
????
?
??
?
?
23t2t
2
3t
2
2t
3t2t32
2
2t
2
3
2
3t
2
2
)xx(xx
xxXX2xXxX
n
1 2σ
)var(b1
2
3t2t
2
3t
2
2t
2
3t
)xx()x)(x(
x
??? ? ?2σ
)var(b2
2
3t2t
2
3t
2
2t
2
2t
)xx()x)(x(
x
??? ? ?
)var(b3
2σ
2,In practice,is unknown,so we use its estimator,,
then b1,b2,b3～ t（ n-k）
(7.29)
(7.30)
7.3.3 Properties of OLS Estimators of Multiple Regression
－－ BLUE
7.4 An Illustrative Example
2σ 2σ?
3n
eσ 2t2
??
??
2σσ ?? ?
7.5 Goodness of Fit of Estimated Multiple Regression,
Multiple Coefficient of Determination,R2
? Multiple coefficient of determination,R2
TSS=ESS+RSS (7.33)
R2= (7.34)
R2= (7.36)
R2 also lies between 0 and 1（ just as r2）
R,coefficient of multiple correlation,the degree of linear association
between Y and all the X variables jointly.
R is always taken to be positive.（ r can be positive or negative）
7.6 Hypothesis Testing,General Comments:
TSS
ESS
?
? ??
2
t
3tt32tt2
y
xybxyb
7.7 Individual Hypothesis Testing,
H0:B2=0,or B3=0
Hypothesis testing,t test
d.f.=n-k
k～ the number of parameters estimated（ including
the intercept）
? 7.7.1 The Test of Significance Approach
? 7.7.2 The Confidence Interval Approach
7.8 Joint Hypothesis Testing
－－ Testing the Joint Hypothesis That B2=B3=0 or R2=0
1.Null hypothesis:
H0:B2=B3=0 (7.45)
H0:R2=0 (7.46)
Which means that the two explanatory variables together have
no influence on Y,means the two explanatory variables explain zero
percent of the variation in the dependent variable.
(1)Why this test?
In practice,in a multiple regression one or more variables
individually have no effect on the dependent variable but collectively
they have a significant impact on it.
2,How to test
Analysis of variance (ANOVA) －－ A study of the two
components of TSS
TSS=ESS+RSS (7.33)
Under the assumption of the CLRM,
（ 1） H0:B2=B3=0
（ 2） Get a F statistic:
F= (7.48)
= variance explained by X2 and X3
unexplained variance
= (7.49)
k)R S S / ( n
1)E S S / ( k
R S S / d,f,
E S S / d,f,
?
??
?? ??
?
3)/ ( ne
) / 2xybxy(b
2
t
3tt32tt2
（ 3） Make a judgement:
F＞ Fk-1,n-k,reject H0
F＜ Fk-1,n-k,not reject H0
(k ～ the number of the explanatory variables,
including the intercept
n～ the number of observations)
F=0 R2=0,all the variance are come fromμi
F↑ R2↑,
F=∞,R2=1,all the variance are come from X2,X3
Or,p value of F statistic＜ level of significance,reject H0
p value of F statistic＞ level of significance,not reject H0
TABLE 7-3
ANOVA table for the three-variable regression
Source of
variation
Sum of squares(SS)
d.f
．
MSS=
Due to
regression
(ESS)
Due to
residual
(RSS)
2
n-
3
Total (TSS) n-
1
? ?? 3tt32tt2 xybxyb
2
3t2t
2
3t
2
2t
2
2t
)xx()x)(x(
x
???
?
?
? 2ty
2
xybxyb 3tt32tt2 ? ??
3n
e2t
?
?
d.f.
SS
3,An important relationship between F and R2,
F= (7.50)
R2=0,F=0
R2↑,F↑
R2=0,F=∞
k)) / ( nR(1
1)/ ( kR
2
2
??
?
TABLE 7-5
ANOVA table in terms of R2
Source of
variation
Sum of squares(SS)
d.f
．
MSS=
Due to
regression(ESS)
Due to residual
(RSS)
2
n-3
Total (TSS) n-1
d.f.
SS
)y(R 2t2 ?
)y(R(1 2t2 ??
2
)y(R 2t2 ?
3n
)y)(R(1 2t2
?
? ?
? 2ty
7.9 Two-variable Regression in Context of Multiple
Regression:
Introduction to Specification Bias:
Three-variable regression model,study the net
effect of X2 on Y,with X3 held constant
Two-variable regression model,study the effect of
X2 on Y,the effect of X3 on Y not being netted.
(the X3 variable is not included in the model,but
its effect is included in ui)
Model Specification Errors,Errors of omitting an
important variable from the model.
7.10 Comparing Two R2 Values,The Adjusted R2
A property of R2,the larger the number of explanatory
variables in a model,the higher the R2 will be.
That is because R2= does not take into account the d.f.
The Adjusted R2:,compare two regressions that have
the same dependent variable but a different number of
explanatory variable.
(7.53)
The features of the adjusted R2:
(1) k>1,≤R2
(2) R2>0,>0&<0
2R
kn
1n)R(11R 22
?
????
7.11 When to Add an Additional Explanatory
Variable to the Model
Add variables as long as the adjusted R2 increases.
The Adjusted R2 will increases if the ︱ t︱
（ absolute t） value of the coefficient of the added
variable is larger than 1,where the t value is
computed under the null hypothesis that the
population value of the said coefficient is zero.
7.12 Testing for Structural Stability of Regression
Models,The Chow Test
There is a structural change in the relationship
between the dependent variable Y and the explanatory
variables,Xs
Yt=A+BXt+μ1t （ 7.54）
Divide the time period into two:
Yt=A1+A2Xt+μ2t n1 (7.55）
Yt=B1+B2Xt+μ3t n2 n1+ n2=n (7.56）
Assume:
(1) μ2t ~N (0,σ2),μ3t ~ N (0,σ2)
(2) μ1t,μ2t are independently distributed.
Steps:
（ 1） Get S1,the restricted residual sum of squares（ RSSR）
Estimate regression 7.54,and obtain the RSS,S1,with d.f.=n1+n2-k
（ 2） Get the unrestricted residual sum of squares（ RSSUR）
Estimate regression 7.55,and obtain the RSS,S2,with d.f.=n1-k
Estimate regression 7.56,and obtain the RSS,S3,with d.f.=n2-k
Add S2,S3 to obtain the unrestricted residual sum of squares
（ RSSUR） RSSUR= S2+S3 with d.f.=n1+n2-2k
（ 3） Hypotheses testing:
H0,there is no structural change,the two regressions:7.55 and 7.56,
are statistically the same
Get F statistic,F= (7.57)
If the computed F value＜ the critical F Value,not reject the H0,
If the computed F value＞ the critical F Value,reject the H0,
2knnk,
21UR
URR
21F~2 k )n) / ( n( R S S
) / kR S S( R S S
????
?
Note:
(1)The assumptions underlying the test must be
fulfilled.
(2)The Chow test will only tell us if the two
regressions 7.55 and 7.56 are different,without
telling us whether the difference is on account of the
intercepts or the slopes or both.
(3)The test assumes that we know the point（ s） of
structural break.