Chapter 14
Selected Topics in Single
Equation Regression Models
14.1 Restricted Least Squares（ RLS）
?1,OLS and RLS
（ 1） Unrestricted least squares(ULS):
When using the ordinary least square
method(OLS) to estimate the parameters,
we do not put any prior constraint（ s） or
restriction(s) on the parameters,So we can
estimate the parameters without any
restrictions,This is ULS.
（ 2） Restricted least squares(RLS)
In Yi=B1+B2X2i+B3X3i+ui
If we put any restrictions on the parameters,
such as B2=2,or B2+ B3=1,we use RLS method to
estimate.
The steps of RLS:
·transform the data to take into account
the restrictions suggested by the relevant
theory,
·apply the least squares method (OLS).
?2.Test of the validity of the restriction(s):
Let
R2 =R2 from the unrestricted regression
R*2=R2 from the restricted regression
m =the number of linear restrictions imposed
k =the number of parameters estimated in
the unrestricted regression
n =the number of observations
H0,the restriction（ s） is valid
（ 14.8）
·Estimate the ULS regression and obtain
the R2
·Estimate the RLS regression and obtain
R*2
·Find out the number of restrictions(m).
·Find out the coefficients estimated in the
unrestricted regression(K)
·Compute F value
? ?
)(2
2*2
~
)/()1(
/
knFmknR
mRRF
???
??
Hypothesis testing:
If F>Fc,refuse H0,the restriction(s)
imposed by the theory is not valid
（ statistically speaking）,reject the
restricted least squares regression,use
the standard OLS method.
If F<Fc,accept H0,the given restriction is
valid,the RLS regression is preferred to
ULS.
14.2 Dynamic Economic Models,
Autoregressive and Distributed Lag Models
?1,Definition
Dynamic models/ Distributed lag models:
－－ There is a non-contemporaneous,or
lagged,relationship between Y and the Xs,
for the effect of a unit change in the
value of the explanatory variable is
spread over,or distributed over,a
number of time periods.
The reasons of the dependent variable
respond to a unit change in the
explanatory variable(s) with a time lag.
· Psychological reasons.
· Technological reasons,such as the
purchase of PC,automobile
· Institutional reasons,such as multiyear
contracts.
k-period distributed lag model
Yt=A+B0Xt+B1Xt-1+B2Xt-2+… +BkXt-k+ut （ 14.13）
B0＞ B1＞ B2
The effect of a unit change in the value of the
explanatory variable is felt over k periods.
? B0,the short-run /impact multiplier,which means
“the change in the mean value of Y following a unit
change in X in the same period”
? (B0+B1),(B0+B1+B2):interim/intermediate multipliers.
“……… in the next,following period”
long-run/total multiplier.
?
?
?????
k
ni
ki BBBBB ?210
? 2.Problems in estimation of Distributed Lag
Models:
The distributed lag model （ 14.13）
does not violate any of the standard
assumptions of the classical linear
regression model (CLRM),but when we use
the OLS to estimate,there are some
practical problems:
（ 1） Economic theory does not tell us how
many lagged values of the explanatory
variables should be introduced.
（ 2） With a small sample～ the problem of
degree of freedom
If we introduce too many lagged values,we
will lose the same number of the degrees of
freedom,plus the intercept and current
value,As the number of degrees of
freedom dwindles,statistical inference
becomes increasingly less reliable.
If we have more than one explanatory
variable in the model,for every coefficient
estimated,we will lose 1 d.f,The degrees of
freedom can be consumed even faster.
（ 3） With a large sample ～ the problem of
multicollinearity and the wrong sign of the
coefficient estimated.
·Multicollinearity leads to imprecise
estimation; that is,the standard errors
tend to be large,and the t ratios tends to
be small.
·Coefficients of successive lagged terms
sometimes alternate in sign,
14.3 The Koyck,Adaptive Expectations,and
Stock Adjustment Models Approaches to
Estimating Distributed Lag Models
－－ The Koyck model can reduce the
above problems:
·the number of lagged terms in the
distributed lag models
·the multicollinearity
1,The Koyck Model
Koyck,the adaptive expectations,and the partial /stock
adjustment models:
Yt=A+B0Xt+B1Xt-1+B2Xt-2+… +BkXt-k+ut
Yt=C1+C2Xt+C3 Yt-1 +vt （ 14.16）
All lagged terms in the regression are replaced by a single
lagged value of Y.
C2,short-run impact of a unit change in Xt on mean Yt
C2/(1-C3),Long-run impact of a (sustained) unit
change in Xt on mean Yt;
?2,The Estimation Problems of Koyck Model:
1) When estimate the model by OLS,we must
make sure that the error termυ t and Yt-1 are
not correlated;
（ 1） Ifυ t and Yt-1 are uncorrelated,the OLS
estimators are biased but consistent,That is,
in a large sample the bias tends to disappear.
（ 2） If,υ t is correlated with Yt-1,the OLS
estimators are biased but inconsistent.
2) Ifυ t follows the first-order Markov
scheme,AR(1) scheme,vt=pvt-1+wt,
Then,OLS estimators are biased as well
as inconsistent and the traditional t and
F testing procedure becomes invalid.
3) The conventional Durbin-Watson d test
is not applicable,We should use the
Durbin h statistic or the runs test to
detect the first order autocorrelation.
14.4 What Happens When The
Dependent Variable is Dummy?
?1.Linear probability model(LPM)
Yi=Bi+B2X2i+B3X3i+B4X4i +ui
E(Yi)=Pi the conditional probability
pi =bi+b2X2i+b3X3i+b4X4i
OLS estimation problems.
（ 1） The error term ui in the model does not follow
the normal distribution; rather,it follows the
binomial (probability) distribution.
This problem can be solved as the sample size
increases,the binomial distribution converges to
the normal distribution.
（ 2） The error term ui is heteroscedastic.
We can solve the problem by using
appropriate transformations to make the
error term homoscedastic.
（ 3） The real problem with the model gives
the probability that the event Y will occur
must lie between the limits of 0 and 1.
There is no guarantee that the estimated
Pi will in fact lie between these limits.
? 2,The Logit Model（ 14.5）
Let Pi be the probability of obtaining an A
grade.
Pi/(1-Pi),the odds ratio,.
The logit model tells us that the log of
the odds ratio is a linear function of
explanatory variables.
iiii
i
i uP S IBT U CEBG P ABB
P
P ?????
? 4321
)
1
l n (
（ 1） The probabilities estimated from the
logit model will always lie within the logical
bounds of 0 and 1.
（ 2） The probability of securing an A does
not increase linearly with a unit change in
the value of the explanatory variable.
? 3,Estimation of the Logid Model:
（ 1） Individual Data～ the method of maximum
likelihood(ML)
－－ Interpret the estimated parameters:
Holding all other things constant,if the X2i
goes up by a unit,on the average,the logit,or
log of the odds ration in favor of getting an A,
goes up by B2 units.
－－ Compute the actual probabilities Pi
·Putting the values of explanatory variables
into the estimation equation,get the
numerical value of the logit In(Pi/1-Pi)
·Take the antilog of In(Pi/1-Pi)
·Get Pi
Usually,the higher the value of the logit
is,the higher the actual probability will be.
? （ 2） Grouped Data～ OLS method
～ relative frequency,the
estimator of the true probability Pi if Ni is
fairly large.
substitute pi for Pi,then OLS can be directly
applied.
Problem,heteroscedasticity.
i
i
i N
np ?
ii
i
i uXBB
P
P ???
? 211ln
14.6 The Phenomenon of Spurious
Regression
1,Spurious regression.
In some regression,superficially the results
look good but on further investigation
they look suspect.
Spurious Regression,An R2＞ d is a good
rule of thumb to suspect that the
estimated regression suffers from
spurious (or nonsense) regression,that is,
in actuality there may not be any
meaningful relationship between y and x.
2,Stationary and nonstationary time
series:
Stochastic process is said to be
stationary if its mean and variance are
constant over time and the value of the
covariance between two time periods
depends only on the distance of lag between
the two time periods not on the actual time at
which the covariance is computed.
Letting Yt represent a stochastic time
series,it is stationary if the following
conditions are satisfied:
Mean:
Variance:
Covariance:
? ? ?? ?iE
22)( ??? ??
iE
)])([( ????? ??? ? kttk E
3,Test of nonstationary test---Unit root
test
1,Estimate the following regression:
△ Yt=A1+A2t+A3Yt-1+υ t
2,H0,A3=0,the underlying time series is no
stationary,Unit root hypothesis.
3,τ(tau) test Dickey-Fuller(DF)test.
τ ＞ τ c reject Ho,the time series is stationary.
τ<τ c not reject Ho,the time series is no
stationary
4,Cointegrated time series
It is quite possible that two time series variables is
nonstationary,but there is still a (long-run) stable
or equilibrium relationship between the two,Such
time series are co integrated.
If we are dealing with time series data,we must
make sure that the individual time series are
either stationary or that they are co integrated.
This is not case,we may be open to the charge or
engaging in spurious (or nonsense) regression
analysis.
5,The random walk model
,random walk”,the values of these variables
today will not enable us to predict what these
values will be tomorrow.
(1)Random walk without drift
Yt=Yt-1+u t ut~N（ o,σ ）
Yt=Y0+
Ε(Y t)= Y0
Var(Yt)=var(ut+ut-1+… +u)=Tσ 2
The variance of Yt is not only not constant but
continuously increases with T.
?tu
(2) Rondom walk model with drift,
△ Yt=（ Yt-Yt-1） +u t
Yt=d+Yt-1+u t
E(Yt)=Y0+Td
var(Yt)=Tσ 2
Both the mean and the variance
continuously increase over time.
Selected Topics in Single
Equation Regression Models
14.1 Restricted Least Squares（ RLS）
?1,OLS and RLS
（ 1） Unrestricted least squares(ULS):
When using the ordinary least square
method(OLS) to estimate the parameters,
we do not put any prior constraint（ s） or
restriction(s) on the parameters,So we can
estimate the parameters without any
restrictions,This is ULS.
（ 2） Restricted least squares(RLS)
In Yi=B1+B2X2i+B3X3i+ui
If we put any restrictions on the parameters,
such as B2=2,or B2+ B3=1,we use RLS method to
estimate.
The steps of RLS:
·transform the data to take into account
the restrictions suggested by the relevant
theory,
·apply the least squares method (OLS).
?2.Test of the validity of the restriction(s):
Let
R2 =R2 from the unrestricted regression
R*2=R2 from the restricted regression
m =the number of linear restrictions imposed
k =the number of parameters estimated in
the unrestricted regression
n =the number of observations
H0,the restriction（ s） is valid
（ 14.8）
·Estimate the ULS regression and obtain
the R2
·Estimate the RLS regression and obtain
R*2
·Find out the number of restrictions(m).
·Find out the coefficients estimated in the
unrestricted regression(K)
·Compute F value
? ?
)(2
2*2
~
)/()1(
/
knFmknR
mRRF
???
??
Hypothesis testing:
If F>Fc,refuse H0,the restriction(s)
imposed by the theory is not valid
（ statistically speaking）,reject the
restricted least squares regression,use
the standard OLS method.
If F<Fc,accept H0,the given restriction is
valid,the RLS regression is preferred to
ULS.
14.2 Dynamic Economic Models,
Autoregressive and Distributed Lag Models
?1,Definition
Dynamic models/ Distributed lag models:
－－ There is a non-contemporaneous,or
lagged,relationship between Y and the Xs,
for the effect of a unit change in the
value of the explanatory variable is
spread over,or distributed over,a
number of time periods.
The reasons of the dependent variable
respond to a unit change in the
explanatory variable(s) with a time lag.
· Psychological reasons.
· Technological reasons,such as the
purchase of PC,automobile
· Institutional reasons,such as multiyear
contracts.
k-period distributed lag model
Yt=A+B0Xt+B1Xt-1+B2Xt-2+… +BkXt-k+ut （ 14.13）
B0＞ B1＞ B2
The effect of a unit change in the value of the
explanatory variable is felt over k periods.
? B0,the short-run /impact multiplier,which means
“the change in the mean value of Y following a unit
change in X in the same period”
? (B0+B1),(B0+B1+B2):interim/intermediate multipliers.
“……… in the next,following period”
long-run/total multiplier.
?
?
?????
k
ni
ki BBBBB ?210
? 2.Problems in estimation of Distributed Lag
Models:
The distributed lag model （ 14.13）
does not violate any of the standard
assumptions of the classical linear
regression model (CLRM),but when we use
the OLS to estimate,there are some
practical problems:
（ 1） Economic theory does not tell us how
many lagged values of the explanatory
variables should be introduced.
（ 2） With a small sample～ the problem of
degree of freedom
If we introduce too many lagged values,we
will lose the same number of the degrees of
freedom,plus the intercept and current
value,As the number of degrees of
freedom dwindles,statistical inference
becomes increasingly less reliable.
If we have more than one explanatory
variable in the model,for every coefficient
estimated,we will lose 1 d.f,The degrees of
freedom can be consumed even faster.
（ 3） With a large sample ～ the problem of
multicollinearity and the wrong sign of the
coefficient estimated.
·Multicollinearity leads to imprecise
estimation; that is,the standard errors
tend to be large,and the t ratios tends to
be small.
·Coefficients of successive lagged terms
sometimes alternate in sign,
14.3 The Koyck,Adaptive Expectations,and
Stock Adjustment Models Approaches to
Estimating Distributed Lag Models
－－ The Koyck model can reduce the
above problems:
·the number of lagged terms in the
distributed lag models
·the multicollinearity
1,The Koyck Model
Koyck,the adaptive expectations,and the partial /stock
adjustment models:
Yt=A+B0Xt+B1Xt-1+B2Xt-2+… +BkXt-k+ut
Yt=C1+C2Xt+C3 Yt-1 +vt （ 14.16）
All lagged terms in the regression are replaced by a single
lagged value of Y.
C2,short-run impact of a unit change in Xt on mean Yt
C2/(1-C3),Long-run impact of a (sustained) unit
change in Xt on mean Yt;
?2,The Estimation Problems of Koyck Model:
1) When estimate the model by OLS,we must
make sure that the error termυ t and Yt-1 are
not correlated;
（ 1） Ifυ t and Yt-1 are uncorrelated,the OLS
estimators are biased but consistent,That is,
in a large sample the bias tends to disappear.
（ 2） If,υ t is correlated with Yt-1,the OLS
estimators are biased but inconsistent.
2) Ifυ t follows the first-order Markov
scheme,AR(1) scheme,vt=pvt-1+wt,
Then,OLS estimators are biased as well
as inconsistent and the traditional t and
F testing procedure becomes invalid.
3) The conventional Durbin-Watson d test
is not applicable,We should use the
Durbin h statistic or the runs test to
detect the first order autocorrelation.
14.4 What Happens When The
Dependent Variable is Dummy?
?1.Linear probability model(LPM)
Yi=Bi+B2X2i+B3X3i+B4X4i +ui
E(Yi)=Pi the conditional probability
pi =bi+b2X2i+b3X3i+b4X4i
OLS estimation problems.
（ 1） The error term ui in the model does not follow
the normal distribution; rather,it follows the
binomial (probability) distribution.
This problem can be solved as the sample size
increases,the binomial distribution converges to
the normal distribution.
（ 2） The error term ui is heteroscedastic.
We can solve the problem by using
appropriate transformations to make the
error term homoscedastic.
（ 3） The real problem with the model gives
the probability that the event Y will occur
must lie between the limits of 0 and 1.
There is no guarantee that the estimated
Pi will in fact lie between these limits.
? 2,The Logit Model（ 14.5）
Let Pi be the probability of obtaining an A
grade.
Pi/(1-Pi),the odds ratio,.
The logit model tells us that the log of
the odds ratio is a linear function of
explanatory variables.
iiii
i
i uP S IBT U CEBG P ABB
P
P ?????
? 4321
)
1
l n (
（ 1） The probabilities estimated from the
logit model will always lie within the logical
bounds of 0 and 1.
（ 2） The probability of securing an A does
not increase linearly with a unit change in
the value of the explanatory variable.
? 3,Estimation of the Logid Model:
（ 1） Individual Data～ the method of maximum
likelihood(ML)
－－ Interpret the estimated parameters:
Holding all other things constant,if the X2i
goes up by a unit,on the average,the logit,or
log of the odds ration in favor of getting an A,
goes up by B2 units.
－－ Compute the actual probabilities Pi
·Putting the values of explanatory variables
into the estimation equation,get the
numerical value of the logit In(Pi/1-Pi)
·Take the antilog of In(Pi/1-Pi)
·Get Pi
Usually,the higher the value of the logit
is,the higher the actual probability will be.
? （ 2） Grouped Data～ OLS method
～ relative frequency,the
estimator of the true probability Pi if Ni is
fairly large.
substitute pi for Pi,then OLS can be directly
applied.
Problem,heteroscedasticity.
i
i
i N
np ?
ii
i
i uXBB
P
P ???
? 211ln
14.6 The Phenomenon of Spurious
Regression
1,Spurious regression.
In some regression,superficially the results
look good but on further investigation
they look suspect.
Spurious Regression,An R2＞ d is a good
rule of thumb to suspect that the
estimated regression suffers from
spurious (or nonsense) regression,that is,
in actuality there may not be any
meaningful relationship between y and x.
2,Stationary and nonstationary time
series:
Stochastic process is said to be
stationary if its mean and variance are
constant over time and the value of the
covariance between two time periods
depends only on the distance of lag between
the two time periods not on the actual time at
which the covariance is computed.
Letting Yt represent a stochastic time
series,it is stationary if the following
conditions are satisfied:
Mean:
Variance:
Covariance:
? ? ?? ?iE
22)( ??? ??
iE
)])([( ????? ??? ? kttk E
3,Test of nonstationary test---Unit root
test
1,Estimate the following regression:
△ Yt=A1+A2t+A3Yt-1+υ t
2,H0,A3=0,the underlying time series is no
stationary,Unit root hypothesis.
3,τ(tau) test Dickey-Fuller(DF)test.
τ ＞ τ c reject Ho,the time series is stationary.
τ<τ c not reject Ho,the time series is no
stationary
4,Cointegrated time series
It is quite possible that two time series variables is
nonstationary,but there is still a (long-run) stable
or equilibrium relationship between the two,Such
time series are co integrated.
If we are dealing with time series data,we must
make sure that the individual time series are
either stationary or that they are co integrated.
This is not case,we may be open to the charge or
engaging in spurious (or nonsense) regression
analysis.
5,The random walk model
,random walk”,the values of these variables
today will not enable us to predict what these
values will be tomorrow.
(1)Random walk without drift
Yt=Yt-1+u t ut~N（ o,σ ）
Yt=Y0+
Ε(Y t)= Y0
Var(Yt)=var(ut+ut-1+… +u)=Tσ 2
The variance of Yt is not only not constant but
continuously increases with T.
?tu
(2) Rondom walk model with drift,
△ Yt=（ Yt-Yt-1） +u t
Yt=d+Yt-1+u t
E(Yt)=Y0+Td
var(Yt)=Tσ 2
Both the mean and the variance
continuously increase over time.
