Chapter 5:
Basic Ideas of Linear Regression,
the Two-Variable Model
The Population Regression
Function(PRF)
The Meaning of Regression
Analysis
Stochastic Specification of the
Population Regression Function(PRF)
The Nature of the Stochastic
Error Term
The Sample Regression
Function（ SRF）
The Special Meaning
of the Term ―Linear‖
Regression
Two –Variable versus
Multiple Linear
Regression
Estimation of
Parameters
5.1 The Meaning of Regression
Analysis
? 1.Regression analysis——the study of the
relationship between one variable Y (the
explained,or dependent variable) and
one or more other variables X/Xs
(explanatory,or independent
variables)．
Y,the explained,or dependent variable
X/Xs,the explanatory,or independent
variables
5.1 The Meaning of Regression
Analysis
2.The objective of regression analysis:
（ 1） To estimate the mean,or average，
value of the dependent variable,given the
values of the independent variables．
（ 2） To test hypotheses about the nature of
the dependence-hypotheses suggested by the
underlying economic theory．
（ 3） To predict,or forecast,the mean
value of the dependent variable,given the
values of the independent variable(s)．
5.1 The Meaning of Regression
Analysis
? 3,Note:
(1) Regression does not necessarily imply
causation,Causality must be justified，
or inferred,from the theory that
underlies the phenomenon that is tested
empirically．
（ 2）
（ 3）
5.2 The Population Regression
Function(PRF)
1.Population Regression Line (PRL)
? PRL——a line that tells us how the average/mean value of Y (or any
dependent variable) is related to each value of X(or any independent
variable)．
——a line that passes through the conditional means of Y．
? E(Y|Xi) = B1+B2Xi (5.1)
E(Y|Xi):the mean,or expected,value of Y corresponding to,or
conditional upon,a given value of X．
～ conditional expectation or conditional expected value of Y．
B1 and B2,the parameters,the regression coefficients．
B1,the intercept (coefficient)
B2,the slope (coefficient),—— The slope coefficient measures
the rate of change in the (conditional) mean value of Y per unit change
in X．
2,Population Regression Function（ PRF）
E(Y|Xi) = B1+B2Xi
3,Note:
Usually expressions like E(Y|Xi) is
simply written as E(Y),with the
explicit understanding that the latter in
fact stands for the former．
5.2 The Population Regression
Function(PRF)
5.3 Stochastic Specification of the
Population Regression Function(PRF)
? The deterministic/nonstochastic PRF:
E(Y|Xi)=B1+B2Xi…… (5.1)
which represents the means of the various Y values
corresponding to the specified Xs，
? The stochastic PRF:
Yi=B1+B2Xi+μi (5.2)
which tells us how individual Ys vary around their mean
values due to the presence of the stochastic error term
(1)(B1+B2Xi),the systematic,or deterministic,component
of PRF
(2)μi,which may be called the nonsystematic,or random
component of PRF．
where μ is known as the stochastic,or random error,term,
or simply the error term----a random variable（ r.v.）
5.4 The Nature of the Stochastic
Error Term.
?the influence of those variables that are not
explicitly included in the model．
?Some ―intrinsic‖ randomness,such as human
behavior,that cannot be explained in the model.
?errors of measurement
?some variables might affect Y,but their
combined influence on Y is so small and
nonsystematic.
Note,Error term plays an extremely crucial role
in regression model
5.5 The Sample Regression Function
（ SRF）
? Deterministic SRF,(5.3)
? =the estimator of E(Y|Xi),the estimator of the
population conditional mean
? b1=the estimator of B1
? b2=the estimator of B2
? Stochastic SRF,Y1=b1+b2Xi+ei (5.4)
? ei,the residual term,or simply the residual,it is
analogous to μi and can be regarded as the where ei is
the estimator of μi．
1.Sample Regression Lines (SRLs) and
Sample Regression Function（ SRF）．
i21i XbbY ???
iY?
5.5 The Sample Regression
Function（ SRF）
2.The Objective in Regression Analysis:
--------To estimate the stochastic PRF on the basis
of SRF because more often than not our analysis is
based on a single sample from some population,
But because of sampling variation,our estimate of
the PRF based on the SRF is only approximate.
Figure 5-4 p132
SRF,(5.5)
PRF,(5.6)iii eY?Y ??
iii μ)X|E ( YY ??
5.6 The Special Meaning of the
Term,Linear” Regression
?Linearity in the Variables
?Linearity in the Parameters
?―linear regression‖ means a regression that
is linear in the parameters,the Bs(i.e.，
the parameters are raised to the power of 1
only);it may or may not be linear in
explanatory variables．
5.7 Two –Variable versus Multiple
Linear Regression
?Two-variable regression model:
?Multiple linear regression model
5.8 Estimation of Parameters,
the Method of Ordinary Least
Squares(OLS)
? 1,Get ei,
PRF,(5.2)
SRF,(5.4)
ei=actual Yi-predicted Yi =Yi- =Yi-b1-b2Xi
? (5.12)
? 2,Use least squares principle to Minimize
Least Square Principle,p136
Minimize,
(5.13)
ii21i μXBBY ???
i121i eXbbY ???
iY?
2
i21i
2
ii
2
i
)XbbY(
)Y?Y(e
?
??
???
??
5.8 Estimation of Parameters,
the Method of Ordinary Least Squares(OLS)
3.Get OLS estimators,b1 and b2
? ??? i21iY Xbnb (5.14)
? ? ??? 2i2i1iiY XbXbX
(5.15)
XbYb 21 ?? (5.16)
?
?
?
?
?
?
?
?
?
?
?
?
?
22
i
2
2
i
ii
2
XnX
YXniYXi
)X( X i
)Y-iY)(X( X i
x
yx
b
(5.17)
xi = (Xi- X ) and yi = ( YYi? )
deviations from the sample mean values
(5.15)
5.8 Estimation of Parameters,
the Method of Ordinary Least Squares(OLS)
（ 1） The SRF obtained by the method of OLS always
passes through the sample mean values of X and Y
XbbY 21i ?? (5.19)
（ 2） that is,the mean value of the
residuals is always equal to zero．
?? /nee i
（ 3） The sum of the product of the residuals e and the
explanatory variable X is zero； that is,these two
variables are uncorrelated:
? ? 0Xe ii （ 5.20）
4.Features of the OLS estimators
Basic Ideas of Linear Regression,
the Two-Variable Model
The Population Regression
Function(PRF)
The Meaning of Regression
Analysis
Stochastic Specification of the
Population Regression Function(PRF)
The Nature of the Stochastic
Error Term
The Sample Regression
Function（ SRF）
The Special Meaning
of the Term ―Linear‖
Regression
Two –Variable versus
Multiple Linear
Regression
Estimation of
Parameters
5.1 The Meaning of Regression
Analysis
? 1.Regression analysis——the study of the
relationship between one variable Y (the
explained,or dependent variable) and
one or more other variables X/Xs
(explanatory,or independent
variables)．
Y,the explained,or dependent variable
X/Xs,the explanatory,or independent
variables
5.1 The Meaning of Regression
Analysis
2.The objective of regression analysis:
（ 1） To estimate the mean,or average，
value of the dependent variable,given the
values of the independent variables．
（ 2） To test hypotheses about the nature of
the dependence-hypotheses suggested by the
underlying economic theory．
（ 3） To predict,or forecast,the mean
value of the dependent variable,given the
values of the independent variable(s)．
5.1 The Meaning of Regression
Analysis
? 3,Note:
(1) Regression does not necessarily imply
causation,Causality must be justified，
or inferred,from the theory that
underlies the phenomenon that is tested
empirically．
（ 2）
（ 3）
5.2 The Population Regression
Function(PRF)
1.Population Regression Line (PRL)
? PRL——a line that tells us how the average/mean value of Y (or any
dependent variable) is related to each value of X(or any independent
variable)．
——a line that passes through the conditional means of Y．
? E(Y|Xi) = B1+B2Xi (5.1)
E(Y|Xi):the mean,or expected,value of Y corresponding to,or
conditional upon,a given value of X．
～ conditional expectation or conditional expected value of Y．
B1 and B2,the parameters,the regression coefficients．
B1,the intercept (coefficient)
B2,the slope (coefficient),—— The slope coefficient measures
the rate of change in the (conditional) mean value of Y per unit change
in X．
2,Population Regression Function（ PRF）
E(Y|Xi) = B1+B2Xi
3,Note:
Usually expressions like E(Y|Xi) is
simply written as E(Y),with the
explicit understanding that the latter in
fact stands for the former．
5.2 The Population Regression
Function(PRF)
5.3 Stochastic Specification of the
Population Regression Function(PRF)
? The deterministic/nonstochastic PRF:
E(Y|Xi)=B1+B2Xi…… (5.1)
which represents the means of the various Y values
corresponding to the specified Xs，
? The stochastic PRF:
Yi=B1+B2Xi+μi (5.2)
which tells us how individual Ys vary around their mean
values due to the presence of the stochastic error term
(1)(B1+B2Xi),the systematic,or deterministic,component
of PRF
(2)μi,which may be called the nonsystematic,or random
component of PRF．
where μ is known as the stochastic,or random error,term,
or simply the error term----a random variable（ r.v.）
5.4 The Nature of the Stochastic
Error Term.
?the influence of those variables that are not
explicitly included in the model．
?Some ―intrinsic‖ randomness,such as human
behavior,that cannot be explained in the model.
?errors of measurement
?some variables might affect Y,but their
combined influence on Y is so small and
nonsystematic.
Note,Error term plays an extremely crucial role
in regression model
5.5 The Sample Regression Function
（ SRF）
? Deterministic SRF,(5.3)
? =the estimator of E(Y|Xi),the estimator of the
population conditional mean
? b1=the estimator of B1
? b2=the estimator of B2
? Stochastic SRF,Y1=b1+b2Xi+ei (5.4)
? ei,the residual term,or simply the residual,it is
analogous to μi and can be regarded as the where ei is
the estimator of μi．
1.Sample Regression Lines (SRLs) and
Sample Regression Function（ SRF）．
i21i XbbY ???
iY?
5.5 The Sample Regression
Function（ SRF）
2.The Objective in Regression Analysis:
--------To estimate the stochastic PRF on the basis
of SRF because more often than not our analysis is
based on a single sample from some population,
But because of sampling variation,our estimate of
the PRF based on the SRF is only approximate.
Figure 5-4 p132
SRF,(5.5)
PRF,(5.6)iii eY?Y ??
iii μ)X|E ( YY ??
5.6 The Special Meaning of the
Term,Linear” Regression
?Linearity in the Variables
?Linearity in the Parameters
?―linear regression‖ means a regression that
is linear in the parameters,the Bs(i.e.，
the parameters are raised to the power of 1
only);it may or may not be linear in
explanatory variables．
5.7 Two –Variable versus Multiple
Linear Regression
?Two-variable regression model:
?Multiple linear regression model
5.8 Estimation of Parameters,
the Method of Ordinary Least
Squares(OLS)
? 1,Get ei,
PRF,(5.2)
SRF,(5.4)
ei=actual Yi-predicted Yi =Yi- =Yi-b1-b2Xi
? (5.12)
? 2,Use least squares principle to Minimize
Least Square Principle,p136
Minimize,
(5.13)
ii21i μXBBY ???
i121i eXbbY ???
iY?
2
i21i
2
ii
2
i
)XbbY(
)Y?Y(e
?
??
???
??
5.8 Estimation of Parameters,
the Method of Ordinary Least Squares(OLS)
3.Get OLS estimators,b1 and b2
? ??? i21iY Xbnb (5.14)
? ? ??? 2i2i1iiY XbXbX
(5.15)
XbYb 21 ?? (5.16)
?
?
?
?
?
?
?
?
?
?
?
?
?
22
i
2
2
i
ii
2
XnX
YXniYXi
)X( X i
)Y-iY)(X( X i
x
yx
b
(5.17)
xi = (Xi- X ) and yi = ( YYi? )
deviations from the sample mean values
(5.15)
5.8 Estimation of Parameters,
the Method of Ordinary Least Squares(OLS)
（ 1） The SRF obtained by the method of OLS always
passes through the sample mean values of X and Y
XbbY 21i ?? (5.19)
（ 2） that is,the mean value of the
residuals is always equal to zero．
?? /nee i
（ 3） The sum of the product of the residuals e and the
explanatory variable X is zero； that is,these two
variables are uncorrelated:
? ? 0Xe ii （ 5.20）
4.Features of the OLS estimators
