16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Page 1 of 8
Lecture 12
Non-zero power at zero frequency
Non-zero power at non-zero frequency
If ()
xx
R τ includes a sinusoidal component corresponding to the component
0
( ) sin( )xt A tω θ=+
where θ is uniformly distributed over 2π , A is random, independent of θ , that
component will be
() ()
()()
00
00
2
0
2
0
2
2
2
00
1
() cos
2
1
() cos
2
11
22
11
22
1
2
xx
j
xx
jjj
jj
RA
SAed
A eeed
Ae e d
A
ωτ
ωτ ωτ ωτ
ωωτ ωωτ
τωτ
ωωττ
τ
τ
πδωωδωω
∞
?
?∞
∞
? ?
?∞
∞
??+
?∞
=
=
??=+
??
??
=+
??
=?++
∫
∫
∫
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Page 2 of 8
Units of
xx
S
Mean squared value per unit frequency interval.
Usually:
2
() ()
1
()
2
() , where
2
j
xx xx
xx
xx
SRed
xSd
Sfdf f
ωτ
ωττ
ωω
π
ω
π
∞
?
∞
∞
?∞
∞
?∞
=
=
==
∫
∫
∫
In this case,
2
~
xx
q
S
Hz
Next most common:
2
1
() ()
2
()
j
xx xx
xx
SRed
xSd
ωτ
ω ττ
π
ωω
∞
?
?∞
∞
?∞
=
=
∫
∫
In this case,
2
2
~sec
rad / sec
xx
q
Sq=
There is an alternate form of the power spectral density function.
Since ()
xx
S ω is a measure of the power density of the harmonic components of
()x t , one should be able to get ()
xx
S ω also from the Fourier Transform of ()x t
which is a direct decomposition of ()x t into its infinitesimal harmonic
components.
This is true, and is the approach taken in the text. One difficulty is that the
Fourier Transform does not converge for members of stationary ensembles. The
mathematics are handled by a limiting process.
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Page 3 of 8
Define
(), ( )
()
0, elsewhere
T
x tTtT
xt
?<<
?
=
?
?
() ()
()
j
TT
T
jt
T
X xte d
x te dt
ωτ
ω
ω τ
∞
?
?∞
?
?
=
=
∫
∫
Then
*
2
() ()
() lim
2
()
lim
2
TT
xx
T
T
T
XX
S
T
X
T
ω ω
ω
ω
→∞
→∞
=
=
Notice that the operations of transforming and averaging and product are done
in opposite order here than if the transform of the autocorrelation function is
calculated.
If one has only a finite record of a single random function, and ()
xx
S ω is to be so
calculated approximately under the ergodic hypothesis, it can be done either
way.
max
0
0
*
2
() ()( )
() 2 ()cos
() ()
() ()
()
2
T
xx
xx xx
T
jt
T
xx
R xtxt d
T
SRd
xxtedt
XX
S
T
τ
τ
ω
τ ττ
τ
ω τωττ
ω
ωω
ω
?
?
?
=+
?
=
=
=
∫
∫
∫
Standard deviation of
xx
S measured this way is approximately equal to mean.
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Page 4 of 8
The second approach is faster.
In fact, with the advent of the Fast Fourier Transform (initiated by Cooley and
Tukey), even if one wanted to calculate ()
xx
R τ from ()x t , it is faster to transform
()x t to ()X ω , form ()
xx
S ω , and transform to get ()
xx
R τ than to integrate
() ( )xtxt τ+ directly for all desired values of τ .
The Fast Fourier Transform is an amazingly efficient procedure for digital
calculation of finite Fourier Transforms.
References:
Full issue – IEEE Transactions on Audio and Electroacoustics, Vol. AU-15, No.2;
June 1967.
Tutorial article: Brighton, E.O. and Morrow, R.E.: The Fast Fourier Transform,
IEEE Spectrum; Dec. 1967.
Cross spectral density
In dealing with more than one random process, the cross power spectral density
arises naturally. For example, if
() () ()zt xt yt=+
where ()x t and ()yt are members of random ensembles, then we found before
that
() () () () ()
zz xx xy yx yy
RRRRRτ ττττ=+++
so that
() ()
() () () ()
j
zz zz
xx xy yx yy
SRed
SSSS
ωτ
ωττ
ω ωωω
∞
?
?∞
=
=+++
∫
where we have defined cross spectral density functions:
() ()
() ()
j
xy xy
j
yx yx
SRed
SRed
ωτ
ωτ
ω ττ
ω ττ
∞
?
?∞
∞
?
?∞
=
=
∫
∫
This is equivalent to the definition
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Page 5 of 8
*
*
() ()
( ) lim
2
() ()
( ) lim
2
TT
xy
T
TT
yx
T
XY
S
T
YX
S
T
ω ω
ω
ω ω
ω
→∞
→∞
=
=
We note from this that:
*
() ()
yx xy
SSω ω=
so that the sum of these two as they appear in ()
zz
S ω is real.
Also note that
*
() () ()
xy xy yx
SSSω ωω?= = .
Examples of Random Processes Analytically Defined
Example: Random step function
Amplitude
n
a independent, random
Change points
n
t , Poisson-distributed with average density λ (points per
second)
()
1
()
!
(0)
k
Pk e
k
Pe
λτ
λτ
λτ
?
?
=
=
[ ]
()
22
22
22
() ()( )
(at least one change point in ) (no change point in )
1
xx
a
R Extxt
PaPa
eaea
ea
λτ λτ
λτ
ττ
ττ
σ
??
?
=+
=? +
=+
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Page 6 of 8
() ()
22
0
22 2
0
0
22
2
0
22
2
2
22
()
2()
2()
11
2()
2
2()
j
xx a
jj
aa
jj
aa
a
a
Saeed
aeeded
ae e
a
jj
a
λτ ωτ
ωτ λτ ωτ λτ
λωτ λωτ
ωσ τ
πδω σ τ σ τ
σσ
πδω
λω λω
πδωσ
λωλω
σλ
πδω
λω
∞
? ?
?∞
∞
???
?∞
∞
??+
?∞
??
=+
??
=+ +
=+ +
???
??
=+ +
??
?+
??
=+
+
∫
∫∫
Example: A signal random process
Reference: Newton, G.C, L.A. Gould and J.F. Kaiser. Design of Linear Feedback
Controls. John Wiley, 1961. p.100. Papoulis calls this the semirandom
telegraph signal; p.288.
A signal takes the values plus and minus
0
x only. It switches from one level
to the other at “event points” which are Poisson distributed in time with
constant average frequency λ . This is sometimes called a telegraph signal.
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Page 7 of 8
Average rate of occurrence of change points λ= points per second
() ()
()()
2
0
2
0
22
00
0,2,4,... 1,3,5,...
2
0
() ()( )
(even number of change points in the interval )
(odd number of change points in the interval )( )
11
!!
1
1
!
xx
kk
kk
k
k
k
RExtxt
Px
xexe
ex
k
λτ λτ
λτ
ττ
τ
τ
λτ λτ
λτ
??
==
?
=+??
??
=
+?
=?
=?
∑∑
()
0,1,2,...
2
0
0
22
0
!
k
k
xe
k
xe
λτ
λτ
λτ
=
∞
?
=
?
?
=
=
∑
∑
You are generating higher harmonics in this case, as at each change point the
amplitude changes sign. In the previous example, changes in amplitude at
change point may be far smaller.
()
2
0
2
2
2(2)
()
2
xx
x
S
λ
ω
ω λ
=
+
Example: Binary function with an arbitrary amplitude distribution
The problem considers a binary function with a more general amplitude
distribution. The distribution of a is restricted to 1± with equal probability.
That gives the pseudo-random binary code used by GPS.
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Page 8 of 8
0
t is uniformly distributed over (0, )T
Change points are periodic with period T
Amplitudes independent with
2
,aa
[ ]
22
22
22
2
() ()( )
(1 or more change points in ) (No change point in )
1 ,
1 ,
()
,
xx
a
xx
R Extxt
PaPa
aaT
TT
aT
TR
aT
ττ
ττ
ττ
τ
τ
στ
τ
τ
=+
??
=+? ≤
??
??
?
??
?+ ≤
? ??
=
? ??
?
>
?