Ch. 14 Stationary ARMA Process A general linear stochastic model is described that suppose a time series to be generated by a linear aggregation of random shock. For practical representation it is desirable to employ models that use parameters parsimoniously. Parsimony may often be achieved by representation of the linear process in terms of a small number of autoregressive and moving average terms. This chapter introduces univariate ARMA process, which provide a very useful class of models for de- scribing the dynamics of an individual time series. Throughout this chapter we assume the time index T to be T = f::: 2; 1; 0; 1; 2; :::g, 1 Moving Average Process 1.1 The First-Order Moving Average Process A stochastic process fYt; t 2 Tg is said to be a rst order moving average process (MA(1)) if it can be expressed in the form Yt = + "t + "t 1; where and are constants and "t is a white-noise process. Remember that a white noise process f"t; t 2 Tg is that E("t) = 0 and E("t"s) = 2 when t = s 0 when t 6= s : 1.1.1 Check Stationarity The expectation of Yt is given by E(Yt) = E( + "t + "t 1) = + E("t) + E("t 1) = ; for all t 2 T : The variance of Yt is E(Yt )2 = E("t + "t 1)2 = E("2t + 2 "t"t 1 + 2"2t 1) = 2 + 0 + 2 2 = (1 + 2) 2: 1 The rst autocovariance is E(Yt )(Yt 1 ) = E("t + "t 1)("t 1 + "t 2) = E("t"t 1 + "2t 1 + "t"t 2 + 2"t 1"t 2) = 0 + 2 + 0 + 0 = 2: Higher autocovariances are all zero: j = E(Yt )(Yt j ) = E("t + "t 1)("t j + "t j 1) = 0 for j > 1: Since the mean and the autocovariances are not functions of time, an MA(1) process is weakly-stationary regardless of the value of . 1.1.2 Check Ergodicity It is clear that the condition 1X j=0 j jj = (1 + 2) + j 2j < 1 is satis ed. Thus the MA(1) process is ergodic. 1.1.3 The Dependence Structure The jth autocorrelation of a weakly-stationary process is de ned as its jth autocovariance divided by the variance rj = j 0 : By Cauchy-Schwarz inequality, we have jrjj 1 for all j. From above results, the autocorrelation of an MA(1) process is rj = 8 < : 1 when j = 0 2 (1+ 2) 2 = (1+ 2) when j = 1 0 when j > 1 : The autocorrelation rj can be plotted as a function of j. This plot is usually called autocogram. See the plots of p.50. 2 1.2 The q-th Order Moving Average Process A stochastic process fYt; t 2 Tg is said to be a moving average process of order q (MA(q)) if it can be expressed in this form Yt = + "t + 1"t 1 + 2"t 2 + ::: + q"t q; where ; 1; 2; :::; q are constants and "t is a white-noise process. 1.2.1 Check Stationarity The expectation of Yt is given by E(Yt) = E( + "t + 1"t 1 + 2"t 2 + ::: + q"t q) = + E("t) + 1E("t 1) + 2E("t 2) + ::: + qE("t q) = ; for all t 2 T : The variance of Yt is 0 = E(Yt )2 = E("t + 1"t 1 + 2"t 2 + ::: + q"t q)2: Since "t’s are uncorrelated, the variance is 0 = 2 + 21 2 + 22 2 + :::: + 2q 2 = (1 + 21 + 22 + ::: + 2q) 2: For j = 1; 2; :::; q, j = E[(Yt )(Yt j )] = E[("t + 1"t 1 + 2"t 2 + ::: + q"t q) ("t j + 1"t j 1 + 2"t j 2 + ::: + q"t j q)] = E[ j"2t j + j+1 1"2t j 1 + j+2 2"2t j 2 + :::: + q q j"2t q]: Terms involving "’s at di erent dates have been dropped because their product has expectation zero, and 0 is de ned to be unity. For j > q, there are no "’s with common dates in the de nition of j, and so the expectation is zero. Thus, j = [ j + j+1 1 + j+2 2 + :::: + q q j] 2 for j = 1; 2; :::; q 0 for j > q : 3 For example, for an MA(2) process, 0 = [1 + 21 + 22] 2 1 = [ 1 + 2 1] 2 2 = [ 2] 2 3 = 4 = :::: = 0 For any value of ( 1; 2; :::; q), the MA(q) process is thus weakly-stationary. 1.2.2 Check Ergodicity It is clear that the condition 1X j=0 j jj < 1 is satis ed. Thus the MA(q) process is ergodic. 1.2.3 The Dependence Structure The autocorrelation function is zero after q lags. See the plots of p.50. 1.3 The In nite-Order Moving Average Process A stochastic process fYt; t 2 Tgis said to be an in nite-order moving average process (MA(1)) if it can be expressed in this form Yt = + 1X j=0 ’j"t j = + ’0"t + ’1"t 1 + ’2"t 2 + ::::: where ; ’0; ’1; ’2; :::; are constants with ’0 = 1 and "t is a white-noise process. 1.3.1 Is This a Well De ned Random Sequence? A sequence f’jg1j=0 is said to be square-summable if 1X j=0 ’2j < 1; 4 whereas a sequence f’jg1j=0 is said to be absolute-summable if 1X j=0 j’jj < 1: It is important to note that absolute summability implies square-summability, but the converse does not hold. Proposition: If the coe cients of the MA(1) is square-summable, then P1j=0 ’j"t j converges in mean square to some random variable Yt as T ! 1. Proof: The Cauchy criterion states that P1j=0 ’j"t j converges in mean square to some random variable Yt as T ! 1 if and only if, for any & > 0, there exists a suitably large N such that for any integer M > N E " MX j=0 ’j"t j NX j=0 ’j"t j #2 < &: (1) In words, once N terms have been summed, the di erence between that sum and the one obtained from summing to M is a random variable whose mean and variance are both arbitrarily close to zero. Now the left hand side of (1) is simply E [’M"t M + ’M 1"t M+1 + :::: + ’N+1"t N 1]2 = (’2M + ’2M 1 + ::: + ’2N+1) 2 = " MX j=0 ’2j NX j=0 ’2j #2 2: (2) But if P1j=0 ’2j < 1, then by the Cauchy criterion the right side of (2) may be made as small as desired by a suitable large N. Thus the MA(1) is well de ned sequence since the in nity series P1j=0 ’j"t j converges in mean squares. 1.3.2 Check Stationarity Assume the MA(1) process to be with absolutely summable coe cients. 5 The expectation of Yt is given by E(Yt) = lim T!1 E( + ’0"t + ’1"t 1 + ’2"t 2 + :::: + ’T "t T ) = The variance of Yt is 0 = E(Yt )2 = lim T!1 E(’0"t + ’1"t 1 + ’2"t 2 + :::: + ’T "t T )2 = lim T!1 (’20 + ’21 + ’22 + :::: + ’2T) 2: For j > 0, j = E(Yt )(Yt j ) = (’j’0 + ’j+1’1 + ’j+2’2 + ’j+3’3 + ::::) 2 = 2 1X k=0 ’j+k’k: Thus, E(Yt) and j are both nite and independent of t. The MA(1) process with absolute-summable coe cients is weakly-stationary. 1.3.3 Check Ergodicity Proposition: The absolute summability of the moving average coe cients implies that the pro- cess is ergodic. Proof: Recall the autocovariance of an MA(1) is j = 2 1X k=0 ’j+k’k: 6 Then j jj = 2 1X k=0 ’j+k’k 2 1X k=0 j’j+k’kj; and 1X j=0 j jj 2 1X j=0 1X k=0 j’j+k’kj = 2 1X j=0 1X k=0 j’j+kjj’kj = 2 1X k=0 j’kj 1X j=0 j’j+kj: But there exists an M < 1such that P1j=0 j’jj < M, and therefore P1j=0 j’j+kj < M for k = 0; 1; 2; :::, meaning that 1X j=0 j jj < 2 1X k=0 j’kjM < 2M2 < 1: Hence, the MA(1) process with absolute-summable coe cients is ergodic. 7 2 Autoregressive Process 2.1 The First-Order Autoregressive Process A stochastic process fYt; t 2 Tg is said to be a rst order autoregressive process (AR(1)) if it can be expressed in the form Yt = c + Yt 1 + "t; where c and are constants and "t is a white-noise process. 2.1.1 Check Stationarity and Ergodicity Write the AR(1) process is lag operator form: Yt = c + LYt + "t; then (1 L)Yt = c + "t: In the case j j < 1, we know from the properties of lag operator in last chapter that (1 L) 1 = 1 + L + 2L2 + ::::; thus Yt = (c + "t) (1 + L + 2L2 + ::::) = (c + Lc + 2L2c + :::) + ("t + L"t + 2L2"t + :::) = (c + c + 2c + :::) + ("t + "t 1 + 2"t 2 + :::) = c1 + "t + "t 1 + 2"t 2 + ::: This can be viewed as an MA(1) process with ’j given by j. When j < 1, this AR(1) is an MA(1) with absolute summable coe cient: 1X j=0 j’jj = 1X j=0 j jj = 11 j j < 1: Therefore, the AR(1) process is stationary and ergodic provided that j j < 1. 8 2.1.2 The Dependence Structure The expectation of Yt is given by E(Yt) = E( c1 + "t + 1"t 1 + 2"t 2 + :::) = c1 = : The variance of Yt is 0 = E(Yt )2 = E("t + 1"t 1 + 2"t 2 + ::::)2 = (1 + 2 + 4 + ::::) 2 = 1 1 2 2: For j > 0, j = E(Yt )(Yt j ) = E("t + 1"t 1 + 2"t 2 + :::: + j"t j + j+1"t j 1 + j+2"t j 2 + ::::) ("t j + 1"t j 1 + 2"t j 2 + ::::) = ( j + j+2 j+4 + :::) 2 = j(1 + 2 + 4 + ::::) 2 = j 1 2 2: It follows that the autocorrelation function rj = j 0 = j; which follows a pattern of geometric decay as the plot on p.50. 2.1.3 An Alternative Way to Calculate the Moments of a Stationary AR(1) Process Assume that the AR(1) process under consideration is weakly-stationary, then taking expectation on both side we have E(Yt) = c + E(Yt 1) + E("t): 9 Since by assumption that the process is stationary, E(Yt) = E(Yt 1) = : Therefore, = c + + 0 or = c1 ; reproducing the earlier result. To nd a higher moments of Yt in an analogous manner, we rewrite this AR(1) as Yt = (1 ) + Yt 1 + "t or (Yt ) = (Yt 1 ) + "t: (3) For j 0, multiply (Yt j ) on both side of (3) and take expectation: j = E[(Yt )(Yt j )] = E[(Yt 1 )(Yt j )] + E(Yt j )"t = j 1 + E(Yt j )"t: Next we consider the term E(Yt j )"t. When j = 0, multiply "t on both side of (3) and take expectation: E(Yt )"t = E[ (Yt 1 )"t] + E("2t ): Recall that Yt 1 is a linear function of "t 1; "t 2; ::: : Yt 1 = "t 1 + "t 2 + 2"t 3 + ::::: we have E[ (Yt 1 )"t] = 0: 10 Therefore, E(Yt )"t = E("2t ) = 2; and when j > 0, it is obvious that E(Yt j )"t = 0. Therefore we the results that 0 = 1 + 2; for j = 0 1 = 0; for j = 1 and j = j 1; for j > 1: That is 0 = 0 + 2 = 2 1 2 : Beside 0, we need rst moment ( 1) to solve 0. 2.2 The Second-Order Autoregressive Process A stochastic process fYt; t 2 Tg is said to be a second order autoregressive process (AR(2)) if it can be expressed in the form Yt = c + 1Yt 1 + 2Yt 2 + "t; where c, 1 and 2 are constants and "t is a white-noise process. 2.2.1 Check Stationarity and Ergodicity Write the AR(2) process is lag operator form: Yt = c + 1LYt + 2L2Yt + "t; then (1 1L 2L2)Yt = c + "t: 11 In the case that all the roots of the polynomial (1 1L 2L2) = 0 lies outside the unit circle, we know from the properties of lag operator in last chapter that there exist a polynomial ’(L) such that ’(L) = (1 1L 2L2) 1 = ’0 + ’1L + ’2L2 + ::::; with 1X j=0 j’jj < 1: Proof: ’j here is equal to c1 j1 + c2 j2, where c1 + c2 = 1 and 1; 2 are the reciprocal of the roots of the polynomial (1 1L 2L2) = 0. Therefore, 1 and 2 lie inside the unit circle. See Hamilton, p. 33, [2.3.23]. 1X j=0 j’jj = 1X j=0 jc1 j1 + c2 j2j 1X j=0 jc1 j1j+ j 1X j=0 jc2 j2j jc1j 1X j=0 j j1j + jc2jj 1X j=0 j j2j < 1: Thus Yt = (c + "t) (1 + ’1L + ’2L2 + ::::) = (c + ’1Lc + ’22L2c + :::) + ("t + ’1L"t + ’2L2"t + :::) = (c + ’1c + ’22c + :::) + ("t + ’1"t 1 + ’2"t 2 + :::) = c(1 + ’1 + ’22 + :::) + ("t + ’1"t 1 + ’2"t 2 + :::) = c1 1 2 + "t + ’1"t 1 + ’2"t 2 + :::; where the constant term is from the fact that substituting 1 into the identity (1 1L 2L2) 1 = 1 + ’1L + ’2L2 + :::: This can be viewed as an MA(1) process with absolute summable coe cient Therefore, the AR(2) process is stationary and ergodic provided that all the roots of (1 1L 2L2) = 0 lies outside the unit circle. 12 2.2.2 The Dependence Structure Assume that the AR(2) process under consideration is weakly-stationary, then taking expectation on both side we have E(Yt) = c + 1E(Yt 1) + 2E(Yt 2) + E("t): Since by assumption that the process is stationary, E(Yt) = E(Yt 1) = E(Yt 2) = : Therefore, = c + 1 + + 2 + 0 or = c1 1 2 : To nd the higher moment of Yt in an analogous manner, we rewrite this AR(2) as Yt = (1 1 2) + 1Yt 1 + + 2Yt 2 + "t or (Yt ) = 1(Yt 1 ) + 2(Yt 2 ) + "t: (4) For j 0, multiply (Yt j ) on both side of (4) and take expectation: j = E[(Yt )(Yt j )] = 1E[(Yt 1 )(Yt j )] + 2E[(Yt 2 )(Yt j )] + E(Yt j )"t = 1 j 1 + 2 j 2 + E(Yt j )"t: 13 Next we consider the term E(Yt j )"t. When j = 0, multiply "t on both side of (4) and take expectation: E(Yt )"t = E[ 1(Yt 1 )"t] + E[ 2(Yt 2 )"t] + E("2t ): Recall that Yt 1 is a linear function of "t 1; "t 2; ::: :, we have E[ 1(Yt 1 )"t] = 0 and obviously E[ 2(Yt 2 )"t] = 0 also. Therefore, E(Yt )"t = E("2t ) = 2; and when j > 0, it is obvious that E(Yt j )"t = 0. Therefore we the results that 0 = 1 1 + 2 2 + 2; for j = 0; 1 = 1 0 + 2 1; for j = 1; 2 = 1 1 + 2 0; for j = 2; and j = 1 j 1 + 2 j 2; for j > 2: That is 1 = 11 2 0; (5) 2 = 2 1 1 2 0 + 2 0 (6) and therefore 0 = 2 1 1 2 + 2 21 1 2 + 2 2 + 2 or 0 = (1 2) 2 (1 + 2)[(1 2)2 21]: Substituting this result to (5) and (6), we obtains 1 and 2. Beside 0, we need rst two moments ( 1 and 2) to solve 0. 14 2.3 The pth-Order Autoregressive Process A stochastic process fYt; t 2 Tg is said to be a pth order autoregressive process (AR(p)) if it can be expressed in the form Yt = c + 1Yt 1 + 2Yt 2 + ::: + pYt p + "t; where c, 1, 2,..., and p are constants and "t is a white-noise process. 2.3.1 Check Stationarity and Ergodicity Write the AR(p) process is lag operator form: Yt = c + 1LYt + 2L2Yt + ::: + pLpYt + "t; then (1 1L 2L2 ::: pLp)Yt = c + "t: In the case all the roots of the polynomial (1 1L 2L2 ::: pLp) = 0 lies outside the unit circle, we know from the properties of lag operator in last chapter that there exist a polynomial ’(L) such that ’(L) = (1 1L 2L2 ::: pLp) 1 = ’0 + ’1L + ’2L2 + ::::; with 1X j=0 j’jj < 1: Thus Yt = (c + "t) (1 + ’1L + ’2L2 + ::::) = (c + ’1Lc + ’22L2c + :::) + ("t + ’1L"t + ’2L2"t + :::) = (c + ’1c + ’22c + :::) + ("t + ’1"t 1 + ’2"t 2 + :::) = c(1 + ’1 + ’22 + :::) + ("t + ’1"t 1 + ’2"t 2 + :::) = c1 1 2 ::: p + "t + ’1"t 1 + ’2"t 2 + :::; where the constant term is from the fact that substituting 1 into the identity (1 1L 2L2 ::: pLp) 1 = 1 + ’1L + ’2L2 + :::: 15 This can be viewed as an MA(1) process with absolute summable coe cient Therefore, the AR(p) process is stationary and ergodic provided that all the roots of (1 1L 2L2 ::: pLp) = 0 lies outside the unit circle. 2.3.2 The Dependence Structure Assume that the AR(p) process under consideration is weakly-stationary, then taking expectation on both side we have E(Yt) = c + 1E(Yt 1) + 2E(Yt 2) + ::: + pE(Yt p) + E("t): Since by assumption that the process is stationary, E(Yt) = E(Yt 1) = E(Yt 2) = ::: = E(Yt p) = : Therefore, = c + 1 + 2 + ::: + p + 0 or = c1 1 2 ::: p : To nd the higher moment of Yt in an analogous manner, we rewrite this AR(p) as Yt = (1 1 2 ::: p) + 1Yt 1 + 2Yt 2 + ::: + pYt p + "t or (Yt ) = 1(Yt 1 ) + 2(Yt 2 ) + ::: + p(Yt p ) + "t: (7) For j 0, multiply (Yt j ) on both side of (7) and take expectation: j = E[(Yt )(Yt j )] = 1E[(Yt 1 )(Yt j )] + 2E[(Yt 2 )(Yt j )] + ::: + pE[(Yt p )(Yt j )] + E(Yt j )"t = 1 j 1 + 2 j 2 + ::: + p j p + E(Yt j )"t: 16 Next we consider the term E(Yt j )"t. When j = 0, multiply "t on both side of (7) and take expectation: E(Yt )"t = E[ 1(Yt 1 )"t] + E[ 2(Yt 2 )"t] + ::: + E[ p(Yt p )"t] + E("2t ): Recall that Yt 1 is a linear function of "t 1; "t 2; ::: :, we have E[ 1(Yt 1 )"t] = 0 and obviously, E[ i(Yt i )"t] = 0, i = 2; :::; p also. Therefore, E(Yt )"t = E("2t ) = 2; and when j > 0, it is obvious that E(Yt j )"t = 0. Therefore we the results that 0 = 1 1 + 2 2 + +::: + p p + 2; for j = 0; and j = 1 j 1 + 2 j 2 + ::: + p j p; for j = 1; 2; ::: (8) Divided (8) by 0 produce the Y ule Waker equation: rj = 1rj 1 + 2rj 2 + ::: + prj p; for j = 1; 2; ::: (9) Intuitively, beside 0, we need rst p moments ( 1, 2,..., p) to solve 0. Exercise: Use the same realization of "’s to simulate and plot the following Gaussian process Yt (set 2 = 1) in a sample of size T=500: (1). Yt = "t, (2). Yt = "t + 0:8"t 1, (3). Yt = "t 0:8"t 1, (4). Yt = 0:8Yt 1 + "t, (5). Yt = 0:8Yt 1 + "t, (6). ~Yt = ~"t + 1:25~"t 1, where V ar(~"t) = 0:64. 17 3 Mixed Autoregressive Moving Average Pro- cess The dependence structure described by a MA(q) process is truncated after the rst q period, meanwhile it is geometrically decaying in an AR(p) process, de- pending on it AR coe cients. A richer exibility in the dependence structure in the rst few lags model is called for to meet the real phenomena. An ARMA(p; q) model meets this requirement. A stochastic process fYt; t 2 Tg is said to be a autoregressive moving average process of order (p; q) (ARMA(p; q)) if it can be expressed in the form Yt = c + 1Yt 1 + 2Yt 2 + ::: + pYt p + "t + 1"t 1 + 2"t 2 + ::: + q"t q; where c, 1, 2,..., p, and 1; :::; q are constants and "t is a white-noise process. 3.1 Check Stationarity and Ergodicity Write the ARMA(p; q) process is lag operator form: (1 1L 2L2 ::: pLp)Yt = c + (1 + 1L + 2L2 + ::: + qLq)"t: In the case all the roots of the polynomial (1 1L 2L2 ::: pLp) = 0 lies outside the unit circle, we know from the properties of lag operator in last chapter that there exist a polynomial ’(L) such that ’(L) = (1 1L 2L2 ::: pLp) 1(1 + 1L + 2L2 + ::: + qLq) = ’0 + ’1L + ’2L2 + ::::; with 1X j=0 j’jj < 1: Thus Yt = + ’(L)"t; 18 where ’(L) = (1 + 1L + 2L 2 + ::: + qLq) (1 1L 2L2 ::: pLp);P 1 j=0 j’jj < 1; = c1 1 2 ::: p : This can be viewed as an MA(1) process with absolute summable coe cient Therefore, the ARMA(p; q) process is stationary and ergodic provided that all the roots of (1 1L 2L2 ::: pLp) = 0 lies outside the unit circle. Thus, the stationarity of an ARMA(p; q) process depends entirely on the autoregressive pa- rameters ( 1; 2; :::; p) and not on the moving average parameters ( 1; 2; :::; q). 3.2 The Dependence Structure Assume that the ARMA(p; q) process under consideration is weakly-stationary, then taking expectation on both side we have E(Yt) = c + 1E(Yt 1) + 2E(Yt 2) + ::: + pE(Yt p) + E("t) + 1E("t 1) + ::: + qE("t q): Since by assumption that the process is stationary, E(Yt) = E(Yt 1) = E(Yt 2) = ::: = E(Yt p) = : Therefore, = c + 1 + 2 + ::: + p + 0 + 0 + ::: + 0 or = c1 1 2 ::: p : To nd the higher moment of Yt in an analogous manner, we rewrite this ARMA(p; q) as Yt = (1 1 2 ::: p) + 1Yt 1 + 2Yt 2 + ::: + pYt p + "t + 1"t 1 + ::: + q"t q 19 or (Yt ) = 1(Yt 1 ) + 2(Yt 2 ) + ::: + p(Yt p ) + "t + 1"t 1 + ::: + q"t q:(10) For j 0, multiply Yt j on both side of (10) and take expectation: j = E[(Yt )(Yt j )] = 1E[(Yt 1 )(Yt j )] + 2E[(Yt 2 )(Yt j )] + ::: + pE[(Yt p )(Yt j )] + (Yt j )"t + 1E(Yt j )("t 1) +:::: + qE(Yt j )("t q) = 1 j 1 + 2 j 2 + ::: + p j p +E(Yt j )"t + 1E(Yt j )("t 1) + :::: + qE(Yt j )("t q): It is obvious that the term E(Yt j )"t+ 1E(Yt j )("t 1)+::::+ qE(Yt j )("t q) = 0 when j > q. Therefore we the results that j = 1 j 1 + 2 j 2 + ::: + p j p; for j = q + 1; q + 2; ::: (11) . Thus, after q lags the autocovariance function j follow the pth order di er- ence equation governed by the autoregressive coe cients. However, (11) does not hold for j q, owing to correlation between j"t j and Yt j. Hence, an ARMA(p; q) process will have more complicated autocovariance function from lag 1 through q than would the corresponding AR(p) process. 3.3 Common Factor Therefore is a potential for redundant parameterizations with ARMA process. Consider factoring the lag polynomial operator in an ARMA(p; q) process: (1 1L)(1 2L):::(1 pL)Yt = (1 1L)(1 2L)::::(1 qL)"t: (12) 20 We assume that j ij < 1 for all i, so that the process is covariance-stationary. If the autoregressive operator (1 1L 2L2 ::: pLp) and the moving average operator (1 + 1L + 2L2 ::: + qLq) have any roots in common, say, i = j for some i and j, then both side of (12) can be divided by (1 iL): pY k=1;k6=i (1 k)Yt = qY k=1;k6=j (1 j)"t; or (1 1L 2L2 ::: p 1Lp 1)Yt = (1 + 1L + 2L2 ::: + q 1Lq 1)"t; (13) where (1 1L 2L2 ::: p 1Lp 1) = (1 1L)(1 2L):::(1 i 1L)(1 i+1L):::(1 pL); and (1 + 1L + 2L2 ::: + q 1Lq 1) = (1 1L)(1 2L):::(1 j 1L)(1 j+1L):::(1 qL): The stationary ARMA(p; q) process satisfying (12) is clearly identical to the stationary ARMA(p 1; q 1) process satisfying (13). 21 4 The Autocovariance-Generating Function If fYt; t 2 Tg is a stationary process with autocovariance function j, then its autocovariance generating function is de ned by gY (z) = 1X j= 1 jzj: If two di erent process share the same autocovariance-generating function, then the two processes exhibit the identical sequence of autocovariance. As an example of calculating an autocovariance-generating function, consider the MA(1) process. Its autocovariance-generating function is: gY (z) = [ 2]z 1 + [(1 + 2) 2]z0 + [ 2]z1 = 2 [ z 1 + (1 + 2) + z](14) = 2(1 + z)(1 + z 1): (15) The form of expression (15) suggests that for the MA(q) process, its autocovariance- generating function might be calculated as gY (z) = 2(1 + 1z + 2z2 + ::: + qzq)(1 + 1z 1 + 2z 2 + ::: + qz q): (16) This conjecture can be veri ed by carrying out the multiplication in (16) and collecting terms by power of z: 2(1 + 1z + 2z2 + ::: + qzq)(1 + 1z 1 + 2z 2 + ::: + qz q) = ( q)zq + ( q 1 + q 1)zq 1 + ( q 2 + q 1 1 + q 2)zq 2 +:::: + ( 1 + 2 1 + 3 2 + :::: + q q 1)z1 +(1 + 21 + 22 + ::: + 2q)z0 +( 1 + 2 1 + 3 2 + ::: + q q 1)z 1 + :::: + ( q)z q: The coe cient on zj is indeed the jth autocovariance in an MA(q) process. The method for nding gY (z) extends to the MA(1) case. If Yt = + ’(L)"t with ’(L) = ’0 + ’1L + ’2L2 + ::: 22 and 1X j=0 j’jj < 1; then gY (z) = 2’(z)’(z 1): Example: An AR(1) process: Yt = (1 L) 1"t; is in this form with ’(L) = 1=(1 L). Thus, the autocovariance-generating function can be calculated from gY (z) = 2 (1 z)(1 z 1): Example: The autocovariance-generating function of an ARMA(p; q) process is therefore be written: gY (z) = 2(1 + 1z + 2z2 + ::: + qzq)(1 + 1z 1 + 2z 2 + ::: + qz q) (1 1z 2z2 ::: pzp)(1 1z 1 2z 2 ::: pz p): 4.1 Filters Sometimes the data are filtered, or treated in a particular way before they are analyzed, and we would like to summarize the e ect of this treatment on the autocovariance. This calculation is particularly simple using the autocovariance- generating function. For example, suppose that the original data, Yt were gener- ated from an MA(1) process: Yt = (1 + L)"t; 23 which has autocovariance-generating function gY (z) = 2 (1 + z)(1 + z 1): Let the data be analyzed is Xt which is taking rst di erence of Yt: Xt = Yt Yt 1 = (1 L)Yt = (1 L)(1 + L)"t: Regarding this Xt as an MA(2) process, then it has the autocovariance-generating function as gX(z) = 2 [(1 z)(1 + z)][(1 z 1)(1 + z 1)] = 2 [(1 z)(1 z 1)][(1 + z)(1 + z 1)] = [(1 z)(1 z 1)]gY (z): Therefore, applying the lter (1 L) to Yt thus resulting in multiplying its autocovariance-generating function by (1 z)(1 z 1). This principle readily generalizes. Let the original data series be Yt and it is ltered according to Xt = h(L)Yt; with h(L) = 1X j= 1 hjLj; and 1X j= 1 jhjj < 1: The autocovariance-generating function of Xt can according be calculated as gX(z) = h(z)h(z 1)gY (z): 24 5 Invertibility 5.1 Invertibility for the MA(q) Process Consider an MA(1) process, Yt = (1 + L)"t; (17) with E("t"s) = 2 for t = s 0 otherwise: Provided that j j < 1 both side of (17) can be multiplied by (1 + L) 1 to obtain (1 L + 2L2 3L3 + :::)(Yt ) = "t; (18) which could be viewed as an AR(1) representation. If a moving average repre- sentation such as (17) can be rewritten as an AR(1) representation such as (18) simply by inverting the moving average operator (1+ L), then the moving aver- age representation is said to be invertible. For an MA(1) process, invertibility requires j j < 1; if j j 1. then the in nite sequence in (18) would not be well de ned.1 Let us investigate what invertibility means in terms of the rst and second moments. Recall that the MA(1) process (17) has mean and autocovariance- generating function gY (z) = 2(1 + z)(1 + z 1): Now consider a seemingly di erent MA(1) process, ~Yt = (1 + ~ L)~"t; (19) with ~"t a white noise sequence having di erent variance E(~"t~"s) = ~ 2 for t = s 0 otherwise: 1When j j 1, (1 + L) 1 = 1L 1 1+ 1L 1 = 1L 1(1 1L 1 + 2L 2 3L 3 + ::::::) 25 Note that ~Yt has the same mean ( ) as Yt. Its autocovariance-generating function is g~Y (z) = ~ 2(1 + ~ z)(1 + ~ z 1) = ~ 2f(~ 1z 1 + 1)(~ z)gf(~ 1z + 1)(~ z 1)g = (~ 2~ 2)(1 + ~ 1z)(1 + ~ 1z 1): Suppose that the parameters of (19), (~ ; ~ 2) are related to those of (17) by the following equations: = ~ 1; (20) 2 = ~ 2~ 2: (21) Then the autocovariance-generating function gY (z) and g~Y (z) would be the same, meaning that Yt and ~Yt would have identical rst and second moments. Notice from (20) that if j j < 1, then j~ j > 1. In other words, for any invertible MA(1) representation (17), we have found a noninvertible MA(1) representation (19) with the same rst and second moments as the invertible representation. Conversely, given any noninvertible representation with j~ j > 1, there exist an invertible representation with = (1=~ ) that has the same rst and second mo- ments as the noninvertible representation. Not only do the invertible and noninvertible representation share the same mo- ments, either representation (17) or (19) could be used as an equally valid de- scription of any given MA(1) process. Suppose that a data series ~Yt is generated from a noninvertible MA(1) process: ~Yt = (1 + ~ L)~"t with j~ j > 1. In what sense could these same data be associated with a invertible MA(1) representation ? Imagine calculating a series f"tg1t= 1 de ned by " (1 + L) 1(~Yt ) (22) = (~Yt ) (~Yt 1 ) + 2(~Yt 2 ) 3(~Yt 3 ) + ::::; (23) where = (1=~ ). 26 The autocovariance-generating function of "t is g"(z) = (1 + z) 1(1 + z 1) 1g~Y (z) = (1 + z) 1(1 + z 1) 1(~ 2~ 2)(1 + ~ 1z)(1 + ~ 1z 1) = (~ 2~ 2); where the last equality follows from the fact that ~ 1 = . Since the autocovariance- generating function is a constant, it follows that "t is a white noise process with variance 2 = ~ 2~ 2. Multiplying both side of (22) by (1 + L), ~Yt = (1 + L)"t is a perfectly valid invertible MA(1) representation of data that were actually generated from the noninvertible representation (19). The converse proposition is also true{suppose that the data were generated really from (17) with j j < 1, an invertible representation. Then there exists a noninvertible representation with ~ = 1= that describes these data with equal validity. Either the invertible or the noninvertible representation could characterize any given data equally well, though there is a practical reason for preferring the invertible representation. To nd the value of " for date t associated with the invertible representation as in (17), we need to know current and past value of Y . By contrast, to nd the value of ~" for date t associated with the noninvertible representation. we need to use all of the future value of Y . If the intention is to calculate the current value of "t using real-world data, it will be feasible only to work with the invertible representation. 5.2 Invertibility for the MA(q) Process Consider the MA(q) process, Yt = (1 + 1L + 2L2 + ::: + qLq)"t E("t"s) = 2 for t = s 0 otherwise: 27 Provided that all the roots of (1 + 1L + 2L2 + :::: + qLq) = 0 lie outside the unit circle, this MA(q) process can be written as an AR(1) simply by inverting the MA operator, (1 + 1L + 2L2 + ::::)(Yt ) = "t; where (1 + 1L + 2L2 + ::::) = (1 + 1L + 2L2 + :::: + qLq) 1: where this is the case, the MA(q) representation is invertible. 28