Ch. 8 Nonspherical Disturbance This chapter will assume that the full ideal conditions hold except that the covari- ance matrix of the disturbance , i.e. E(""0) = 2 , where is not the identity matrix. In particular, may be nondiagonal and/or have unequal diagonal ele- ments. Two cases we shall consider in details are heteroscedasticity and auto- correlation. Disturbance are heteroscedastic when they have di erent variance. Heteroscedasticity usually arise in cross-section data where the scale of the de- pendent variable and the explanatory power of the model tend to vary across observations. The disturbance are still assumed to be uncorrelated across obser- vation, so 2 would be 2 = 2 66 66 66 4 21 0 : : : 0 0 22 : : : 0 : : : : : : : : : : : : : : : : : : 0 0 : : : 2T 3 77 77 77 5 : Autocorrelation is usually found in time-series data. Economic time-series often display a "memory" in that variation around the regression function is not independent from one period to the next. Time series data are usually ho- moscedasticity, so 2 would be 2 = 2 2 66 66 66 4 1 1 : : : T 1 1 1 : : : T 2 : : : : : : : : : : : : : : : : : : T 1 T 2 : : : 1 3 77 77 77 5 : In recent studies, panel data sets, constituting of cross sections observed at several points in time, have exhibited both characteristics. The next three chap- ter examines in details speci c types of generalized regression models. Our earlier results for the classical mode; will have to be modi ed. We rst consider the consequence of the more general model for the least squares estima- tors. 1 1 Properties of the Least Squares Estimators Theorem: The OLS estimator ^ is unbiased. Furthermore, if limT!1(X0 X=T) is nite, ^ is consistent. Proof: E(^ ) = + E(X0X) 1X0" = , which proves unbiasedness. Also plim ^ = + lim T!1 X0X T 1 plimX 0" T : But X0"T has zero mean and covariance matrix 2X 0 X T2 : If limT!1(X0 X=T) is nite, then 2T X0 XT = 0. Hence X0"T has zero mean and its covariance matrix vanishes asymptotically, which implies plim X0"T = 0, and therefore, plim ^ = . Theorem: The covariance matrix of ^ is 2(X0X) 1X0 X(X0X) 1. Proof: E(^ )(^ )0 = E(X0X) 1X0""0X(X0X) 1 = 2(X0X) 1X0 X(X0X) 1: Note that the covariance matrix of ^ is no longer equal to 2(X0X) 1. It may be either "larger" or "smaller", in the sense that (X0X) 1X0 X(X0X) 1 (X0X) 1 can be either positive semide nite, negative semide nite, or neither. Theorem: s2 = e0e=(T k) is (in general) a biased and inconsistent estimator of 2. 2 Proof: E(e0e) = E("0M") = trace E(M""0) = 2 trace M 6= 2(T k): Also since E(s2) 6= 2, it is hard to see that it is a consistent estimator of 2 from convergence in mean square error. 2 E cient Estimators To begin, it is useful to consider cases in which is a known, symmetric, positive de nite matrix. This assumption will occasionally be true, but in most models, will contains unknown parameters that must also be estimated. Example: Assume that 2t = 2x2t, then 2 = 2 66 66 66 4 2x21 0 : : : 0 0 2x22 : : : 0 : : : : : : : : : : : : : : : : : : 0 0 : : : 2x2T 3 77 77 77 5 = 2 2 66 66 66 4 x21 0 : : : 0 0 x22 : : : 0 : : : : : : : : : : : : : : : : : : 0 0 : : : x2T 3 77 77 77 5 ; therefore, we have a "known" . 2.1 Generalized Least Square (GLS) Estimators Since is a positive symmetric matrix, it can be factored into 1 = C 1C0 = C 1=2 1=2C0 = P0P; , where the column of C are the eigenvectors of and the eigenvalues of are arrayed in the diagonal matrix and P0 = C 1=2. 3 Theorem: Suppose that the regression model Y = X +" satisfy the ideal conditions except that is not the identity matrix. Suppose that lim T!1 X0 1X T is nite and nonsingular. Then the transformed equation PY = PX + P" satis es the full ideal condition. Proof: Since P is nonsingular and nonstochastic, PXis nonstochastic and of full rank if X is. (Condition 2 and 5). Also, for the consistency of OLS estimators lim T!1 (PX)0(PX) T = limT!1 X0 1X T is nite and nonsingular by assumption. Therefore the transformed regressors ma- trix satis es the required conditions, and we need consider only the transformed disturbance P". Clearly, E(P") = 0 (Condition 3). Also E(P")(P")0 = 2P P0 = 2( 1=2C0)(C C0)(C 1=2) = 2 1=2 1=2 = 2I (Condition 4): Finally, the normality (Condition 6) of P" follows immediately from the nor- mality of ". Theorem: The BLUE of is just ~ = (X0 1X) 1X0 1Y: Proof: Since the transformed equation satis es the full ideal conditions, the BLUE of 4 is just ~ = [(PX)0(PX)] 1(PX)0(PY) = (X0 1X) 1X0 1Y: Indeed, since ~ is the OLS estimator of in the transformed equation, and since the transformed equation satis es the ideal conditions, ~ has all the usual de- sirable properties{it is unbiased, BLUE, e cient, consistent, and asymptotically e cient. ~ is the OLS of the transformed equation, but it is a generalized least square (GLS) estimator of the original regression model which take the OLS as a sub- cases when = I. Theorem: The variance -covariance of the GLS estimator ~ is 2(X0 1X) 1. Proof: Viewing ~ as the OLS estimator in the transformed equation, it is clearly has covariance matrix 2[(PX)0(PX)] 1 = 2(X0 1X) 1: Theorem: An unbiased, consistent, e cient, and asymptotically e cient estimator of 2 is ~s2 = ~e 0 1~e T k ; where ~e = Y X~ . Proof: Since the transformed equation satis es the ideal conditions, the desired estimator of 2 is 1 T k(PY PX ~ )0(PY PX~ ) = 1 T k(Y X ~ )0 1(Y X~ ): 5 Finally, for testing hypothesis we can apply the full set of results in Chapter 6 to the transformed equation. For the testing the m restrictions R = q, the appropriate (one of) statistics is (R~ q)0[~s2R(PX)0(PX) 1R0] 1(R~ q) m = (R ~ q)0[~s2R(X0 1X) 1R0] 1(R~ q) m Fm;T k: Exercise: Derive the other three test statistics (in Chapter 6) of the F Ratio test statistics to test the hypothesis R = q when 6= I. 2.2 Maximum Likelihood Estimators Assume that " N(0; 2 ), if X are not stochastic, then by results from "func- tions of random variables" (n ) n transformation) we have Y N(X ; 2 ). That is, the log-likelihood function ln f( ; Y ) = T2 ln(2 ) 12 lnj 2 j 12(Y X )0( 2 ) 1(Y X ) = T2 ln(2 ) T2 ln 2 12 lnj j 12 2(Y X )0 1(Y X ) where = ( 1; 2; :::; k; 2)0 since by assumption is known. The necessary condition for maximizing L are @L @ = 1 2X 0 1(Y X ) = 0 @L @ 2 = T 2 2 + 1 2 4(Y X ) 0 1(Y X ) = 0 The solution are ^ ML = (X0 1X) 1X0 1Y; ^ 2ML = 1 T (Y X ^ ML)0 1(Y X^ ML); 6 which implies that with normally distributed disturbance, generalized l;east squares are also MLE. As is the classical regression model, the MLE of 2 is biased. An unbiased estimator is ^ 2 = 1 T k(Y X ^ ML)0 1(Y X^ ML): 3 Estimation When is Unknown If contains unknown parameters that must be estimated, then GLS is not feasible. But with an unrestricted , there are T(T +1)=2 additional parameters in 2 . This number is far too many to estimate with T observations. Obviously, some structures must be imposed on the model if we are to proceed. 3.1 Feasible Generalized Least Squares The typical problem involves a small set parameter such that = ( ). For example, we may assume autocorrelated disturbance in the beginning of this chapter as 2 = 2 2 66 66 66 4 1 1 : : : T 1 1 1 : : : T 2 : : : : : : : : : : : : : : : : : : T 1 T 2 : : : 1 3 77 77 77 5 = 2 2 66 66 66 4 1 1 : : : T 1 1 1 : : : T 2 : : : : : : : : : : : : : : : : : : T 1 T 2 : : : 1 3 77 77 77 5 ; then has only one additional unknown parameters, . A model of heteroscedas- ticity that also has only one new parameters, , is 2t = 2x 2t: De nition: If depends on a nite number of parameters, 1; 2; :::; p, and if ^ depends on consistent estimator, ^ 1; ^ 2; :::; ^ p, the ^ is called a consistent estimator of . De nition: Let ^ be a consistent estimator of . Then the feasible generalized least square estimator (FGLS) of is = (X0^ 1X) 1X0^ 1Y: 7 Conditions that imply that is asymptotically equivalent to ~ are lim T!1 1 T X 0^ 1X 1 T X 0 1X = 0 and lim T!1 1 pT X0^ 1" 1 pT X0 1" = 0: Theorem: An asymptotically e cient FGLS does not require that we have an e cient es- timator of ; only a consistent one is required to achieve full e ciency for the FGLS estimator. 8