3.1,
(1)
() ()
()329291π(1))2( π
32310
32310
32031
001
100π(0))1(π
1
1
=?=
=
=?=
P
P∵
923323922911)(
2
=×+×+×=∴ XE
()32310)1(π =∵
38323312)3(
37323311)2(
)1(
3
1
312
3
1
212
12
=×+×=?==
=×+×=?==
=∴
∑
∑
=
=
i
i
i
i
piXXE
piXXE
XXE没有意义
()329291)2(π =∵
1)1(
3
1
123 ∑
=
=?=∴
i
i
piXXE＝
根据马氏性,38)3(,37)2(
2323
==== XXEXXE
(2)
0)31(
310
==== pXTP
()
3
0210
2
3
21 10
2
3
31
2
(2 3) ( 1,3)
(1 )( 3)
13
13 23 19
0
i
i
ii
i
PT X PX X iX
PX X i PX iX
pp
=
=
=
=== ===
===?==
=?=?=
∑
∑
∑
()
0
33
321 0
22
33
31
22
(3 3)
(1,,3)
023 13
13 23 227
13 23 0
ij
jjii
ij
PT X
PX X iX jX
ppp
==
==
==
===
=
==
∑∑
∑∑
2722272911)34(
0
===≥∴ XTP
27100272242723912)34(
0
=×+×+×==∧∴ XTE
(3)
0)1(
11
==TP
设}3,2{
0
=S,当2≥k时,
()
11
00
2
21
()
(1,,0 1)
023 13
13 23
23 0 13
23
kl
k
kk
PT k
PX X S l kX
=
==∈<<=
=
=
4
232
232
32
)(
1
1
1
2
12
11
=
+=
+?=
=
∑
∑
∑
∞
=
∞
=
+
∞
=
k
kk
k
kk
k
kk
k
k
TE
3.7,
(1)
因为有限状态的马氏链平稳分布一定存在,
Pππ =∴,
解方程组得
)319,6223,6212(π =,
=∴
∞→
31962236212
31962236212
31962236212
lim
n
n
P,
(2)
当Pπ(0)π(0) =时,即)319,6223,6212(π(0) =时,马氏链是平稳序列,
)319,6223,6212()0π()π( ==n∵
() 38442409
6227531996223462211
6212131936223262211
22
2
=?=
=×+×+×=
=×+×+×=∴
nnn
n
n
EXEXDX
EX
EX
3.11
(1)
( )
()
()()
n
n
nn
nnn
n
aX
N
a
aNaX
kN
N
k
aNaX
k
NaXakk
N
N
a
N
k
kN
X
k
X
k
N
ak
N
k
kX
ak
N
k
nn
ak
N
k
nn
ak
n
aX
e
e
ee
eeeeC
e
Cepe
XkXPe
XXXkXPe
XXXeE
2
2
22
0
2222
2
0
2
0
2
0
1
2
0
101
2
10
2
1
)1(
1
1
1
)π1(π
)(
),,,(
,,,
1
=
=
=
=
+
=
+
=
=
=
==
==
==
∑
∑∑
∑
∑
+
null
null
(2)
定义
}},,0{,0:min{
0
kXNXnnT
n
=∈≥=,
},,0:min{
0
kXNXnnT
nN
==≥=,
)()(
0
NXPkXTPV
TNN
===+∞<=,
可知
N
TT和是关于}0,{ ≥nX
n
的停时,且
NN
VkP =)(,
因为}0,{
2
≥
ne
n
aX
是鞅,且
①因为两边带有吸收壁的有限状态马氏链的中间状态为瞬时态,所以1)( =∞<TP,
②∞<=≤
1
02
eEeE
T
aX
③0)(limlim
}{
2
=>≤?
∞→>
∞→
nTPIeE
nnT
aX
n
T
满足停时定理的条件,所以
akaXaX
eeEeE
T
222
)()(
0
==,
NTNT
VXPVNXP?==== 1)0(,)(∵
aN
ak
N
ak
NN
aN
e
e
V
eVVe
2
2
22
1
1
)1(
=∴
=?+?∴
3.14
(1) 用停时定理证明,
定义随机变量
0
0,,1
nnn
ZZXY= =? ≥,可知
n
Z的分布律为
12
12 1 2
21
(1)( 1,0)(1)
(0)( 0,0)(1,1)
(1 )(1 )
(1)(0,1)(1)
nnn
nn
nnn
PZ PX Y p p
PZ PX Y PX Y
pp p p
PZ PX Y p p
== = ==?
== = =+ = =
=+
=? = = = =?
设
0
1
0,,1
n
nk
k
MM Zn
=
== ≥
∑
,易知{,0}
n
Mn≥是马氏链,
定义
min{,0,}
Mn
NnnMM
=≥=?,
可知
M
NN
和是{,0}
n
Zn≥的停时,观察所证结果,想到验证
{}
12
21
(1 )
,0,
(1 )
n
M
n
p p
Vn
p p
λλ
=≥ =
其中
关于{,0}
n
Zn≥是鞅,如下,
11
10
0
1
1212 1 2 21
(,,)
,)()
( (1) (1)(1) (1)
nn n n
n
n
nn
MZ M Z
n
M
M
n
EV Z Z
EZZE
p ppp p p p p
V
λλ λ λ
λλ λ
λ
++
+
=? =
=+ ++?
==
null
null
并且,
①可以将M M?和看成吸收态,所以()1PN<∞ =,
②max(,)
N
M MM
E λλλ
≤<∞
③
{}
lim max(,) lim ( ) 0
n
M MM
nNn n
EI PNnλλλ
→∞ > →∞
≤? >=
满足停时定理的条件,所以
0
()()1
T
MM
EEλλ
==
() ( )+(1( )
11
()
1
N
M MM
TT
M
N
MM M
E PM M PM M
PM M
λλ λ
λ
λλ λ
==?=?
∴=?= =
+
∵
()
T
PM M=?即为误判概率,得证,
(2)
① 方法1,用Wald等式
因为{,1}
n
Zn≥独立同分布,
12n
EZ p p=?<∞,N关于
{,1}
n
Zn≥是停时,且N符合PH分布,
0
()EN I P eα=? <∞,
其中P为瞬时态集的转移矩阵,根据书中P52中的Wald等式,
得
1
()()()
N
nn
n
E ZEZEN
=
=
∑
12n
EZpp=?∵
1
()()
11
(1 )
(1)
1
N
nN
n
MM
M
M
EZEM
MM
M
λ λ
λ
λ
=
=
=
++
=
+
∑
12
(1)
()(1)
M
M
M
EN
pp
λ
λ
∴=
+
得证,
②方法2,用PH分布(略)
3.18,
观察状态图,可以知道
0
1X =时,首次到达0
n
X =需要经过奇数步,而首次到达3
n
X =需要经过偶数步,所以需要分奇偶数讨论,
()
10
2 0
1
11
12 21 12 23
(2 1)
(3,2,1,2,0,1)
klm
k
kk
PT kX
PX X X k l m X
pp pp pq
+?
==
= == => > =
==
为奇数为偶数
()
10
2 0
1
1
12 21 10
(21 1)
(0,2,1,2,01)
klm
k
kk
PT k X
PX X X k l m X
pp p pq
=+ =
= ===> >=
==
为奇数为偶数
(1)/2(1)/2 /21 /21
10 {} {}
(1)
kk kk
kk
PT kX p q I p q I
+ +?
∴= == +
为奇数为偶数
1
010
1
211
1
2
(31) (2 1)
(1 )
T
k
kk
k
PX X PT kX
ppq
ppq
∞
=
∞
=
= == = =
=
=?
∑
∑
3.20,
(1)
58 14 18
13 12 16
34 14 0
P
=
因为有限状态的马氏链平稳分布一定存在,
Pππ =∴,
解方程组得
)8110,31,8144(π =
又因为非周期不可约链的极限分布就是它的平稳分布,
j
n
ijn
p πlim
)(
=∴
∞→
811288110331281441)1(lim
0
=×+×+×==∴
∞→
XXE
n
n
(2)
}1,1,,11,1,:min{
}1,:min{
},11,1,0:min{
},0:min{
1011
11
0
01
=?≤≤∈?≤≤=>=
=>=
∈?≤≤=>=
∈>=
nji
n
ni
n
XnjTSXTiXTnn
XTnn
SXniXnn
SXnnT
τ
因此
}{}{
11
nnT
II
== τ
和均只是
n
XX,,
1
null的函数,即
11
τ和T 都是关于}0,{ ≥nX
n
的停时,
(3)
通过观察状态图,可以得到,
()()
1
00
1
1
11 12 13
1
()
(1,1 1,1)
()58418
35 8
lk
k
k
kk
PT k
PX l k X S X
ppp
=
==≤≤?∈=
=?+=?+
=?
3885
853)(
1
11
1
1
1
==
=∴
∑
∑
∞
=
∞
=
k
kk
k
kk
kTE
由上面(2)可知'
111
ττ +=T,其中'
1
τ为PH分布,其对应的状态空间为''
0
SSS ∪=,}3,2{'=S为瞬时态集,}1{'
0
=S为吸收态,
一步转移概率矩阵为
=
858141
43041
316121
~
P
可以把上面的一步转移概率矩阵看成
=
100
43041
316121
'
~
P,
而不影响对PH分布的计算结果,其中,
=
=
43
31
,
041
6121
0
PP,
确定这个过程的初始分布()αα,)0(π
0
=,由上可知
32)()2(
131212
1
=+== pppXP
T
,
31)()3(
131213
1
=+== pppXP
T
,
所以()31,32,0)0(π =,其中)31,32(=α,
0
1
1
)'( PPkP
k?
==∴ ατ
11
10
1
1
(') ( )
12 16 1
(2 3 1 3)
14 1 1
74 33
k
k
E kP P I P eταα
∞
=
∴= =?
=
=
∑
111
() ( ') 837433 5411EETττ=+=+ =
(4)
()
1230
3
((3) 0)
(1,1,1 1)
58 125512
PN
PX X X X
=
=====
==
11 3
302 100
21
12 13 12 13
31
((3) 2)
(1,2,3)
(,1,1)
()()
13
(1 4 1 8) 3 8
34
17 256
PN
PT T
PX S X X S X
p
pp pp
p
τ
=
====
=∈ =∈=
=+
=
=
((3) 2) 0
( (3) 1) 1 125 512 17 256 353 512
PN
PN
>=
== =
同理,
11 2 11 2
112 112
( (4) 2)
(1,2,3)(1,2,4)
(1,3,4)(2,3,4)
185 1024
PN
PT T PT T
PT T PT T
ττ
=
====+===
+===+===
=
(1)
() ()
()329291π(1))2( π
32310
32310
32031
001
100π(0))1(π
1
1
=?=
=
=?=
P
P∵
923323922911)(
2
=×+×+×=∴ XE
()32310)1(π =∵
38323312)3(
37323311)2(
)1(
3
1
312
3
1
212
12
=×+×=?==
=×+×=?==
=∴
∑
∑
=
=
i
i
i
i
piXXE
piXXE
XXE没有意义
()329291)2(π =∵
1)1(
3
1
123 ∑
=
=?=∴
i
i
piXXE＝
根据马氏性,38)3(,37)2(
2323
==== XXEXXE
(2)
0)31(
310
==== pXTP
()
3
0210
2
3
21 10
2
3
31
2
(2 3) ( 1,3)
(1 )( 3)
13
13 23 19
0
i
i
ii
i
PT X PX X iX
PX X i PX iX
pp
=
=
=
=== ===
===?==
=?=?=
∑
∑
∑
()
0
33
321 0
22
33
31
22
(3 3)
(1,,3)
023 13
13 23 227
13 23 0
ij
jjii
ij
PT X
PX X iX jX
ppp
==
==
==
===
=
==
∑∑
∑∑
2722272911)34(
0
===≥∴ XTP
27100272242723912)34(
0
=×+×+×==∧∴ XTE
(3)
0)1(
11
==TP
设}3,2{
0
=S,当2≥k时,
()
11
00
2
21
()
(1,,0 1)
023 13
13 23
23 0 13
23
kl
k
kk
PT k
PX X S l kX
=
==∈<<=
=
=
4
232
232
32
)(
1
1
1
2
12
11
=
+=
+?=
=
∑
∑
∑
∞
=
∞
=
+
∞
=
k
kk
k
kk
k
kk
k
k
TE
3.7,
(1)
因为有限状态的马氏链平稳分布一定存在,
Pππ =∴,
解方程组得
)319,6223,6212(π =,
=∴
∞→
31962236212
31962236212
31962236212
lim
n
n
P,
(2)
当Pπ(0)π(0) =时,即)319,6223,6212(π(0) =时,马氏链是平稳序列,
)319,6223,6212()0π()π( ==n∵
() 38442409
6227531996223462211
6212131936223262211
22
2
=?=
=×+×+×=
=×+×+×=∴
nnn
n
n
EXEXDX
EX
EX
3.11
(1)
( )
()
()()
n
n
nn
nnn
n
aX
N
a
aNaX
kN
N
k
aNaX
k
NaXakk
N
N
a
N
k
kN
X
k
X
k
N
ak
N
k
kX
ak
N
k
nn
ak
N
k
nn
ak
n
aX
e
e
ee
eeeeC
e
Cepe
XkXPe
XXXkXPe
XXXeE
2
2
22
0
2222
2
0
2
0
2
0
1
2
0
101
2
10
2
1
)1(
1
1
1
)π1(π
)(
),,,(
,,,
1
=
=
=
=
+
=
+
=
=
=
==
==
==
∑
∑∑
∑
∑
+
null
null
(2)
定义
}},,0{,0:min{
0
kXNXnnT
n
=∈≥=,
},,0:min{
0
kXNXnnT
nN
==≥=,
)()(
0
NXPkXTPV
TNN
===+∞<=,
可知
N
TT和是关于}0,{ ≥nX
n
的停时,且
NN
VkP =)(,
因为}0,{
2
≥
ne
n
aX
是鞅,且
①因为两边带有吸收壁的有限状态马氏链的中间状态为瞬时态,所以1)( =∞<TP,
②∞<=≤
1
02
eEeE
T
aX
③0)(limlim
}{
2
=>≤?
∞→>
∞→
nTPIeE
nnT
aX
n
T
满足停时定理的条件,所以
akaXaX
eeEeE
T
222
)()(
0
==,
NTNT
VXPVNXP?==== 1)0(,)(∵
aN
ak
N
ak
NN
aN
e
e
V
eVVe
2
2
22
1
1
)1(
=∴
=?+?∴
3.14
(1) 用停时定理证明,
定义随机变量
0
0,,1
nnn
ZZXY= =? ≥,可知
n
Z的分布律为
12
12 1 2
21
(1)( 1,0)(1)
(0)( 0,0)(1,1)
(1 )(1 )
(1)(0,1)(1)
nnn
nn
nnn
PZ PX Y p p
PZ PX Y PX Y
pp p p
PZ PX Y p p
== = ==?
== = =+ = =
=+
=? = = = =?
设
0
1
0,,1
n
nk
k
MM Zn
=
== ≥
∑
,易知{,0}
n
Mn≥是马氏链,
定义
min{,0,}
Mn
NnnMM
=≥=?,
可知
M
NN
和是{,0}
n
Zn≥的停时,观察所证结果,想到验证
{}
12
21
(1 )
,0,
(1 )
n
M
n
p p
Vn
p p
λλ
=≥ =
其中
关于{,0}
n
Zn≥是鞅,如下,
11
10
0
1
1212 1 2 21
(,,)
,)()
( (1) (1)(1) (1)
nn n n
n
n
nn
MZ M Z
n
M
M
n
EV Z Z
EZZE
p ppp p p p p
V
λλ λ λ
λλ λ
λ
++
+
=? =
=+ ++?
==
null
null
并且,
①可以将M M?和看成吸收态,所以()1PN<∞ =,
②max(,)
N
M MM
E λλλ
≤<∞
③
{}
lim max(,) lim ( ) 0
n
M MM
nNn n
EI PNnλλλ
→∞ > →∞
≤? >=
满足停时定理的条件,所以
0
()()1
T
MM
EEλλ
==
() ( )+(1( )
11
()
1
N
M MM
TT
M
N
MM M
E PM M PM M
PM M
λλ λ
λ
λλ λ
==?=?
∴=?= =
+
∵
()
T
PM M=?即为误判概率,得证,
(2)
① 方法1,用Wald等式
因为{,1}
n
Zn≥独立同分布,
12n
EZ p p=?<∞,N关于
{,1}
n
Zn≥是停时,且N符合PH分布,
0
()EN I P eα=? <∞,
其中P为瞬时态集的转移矩阵,根据书中P52中的Wald等式,
得
1
()()()
N
nn
n
E ZEZEN
=
=
∑
12n
EZpp=?∵
1
()()
11
(1 )
(1)
1
N
nN
n
MM
M
M
EZEM
MM
M
λ λ
λ
λ
=
=
=
++
=
+
∑
12
(1)
()(1)
M
M
M
EN
pp
λ
λ
∴=
+
得证,
②方法2,用PH分布(略)
3.18,
观察状态图,可以知道
0
1X =时,首次到达0
n
X =需要经过奇数步,而首次到达3
n
X =需要经过偶数步,所以需要分奇偶数讨论,
()
10
2 0
1
11
12 21 12 23
(2 1)
(3,2,1,2,0,1)
klm
k
kk
PT kX
PX X X k l m X
pp pp pq
+?
==
= == => > =
==
为奇数为偶数
()
10
2 0
1
1
12 21 10
(21 1)
(0,2,1,2,01)
klm
k
kk
PT k X
PX X X k l m X
pp p pq
=+ =
= ===> >=
==
为奇数为偶数
(1)/2(1)/2 /21 /21
10 {} {}
(1)
kk kk
kk
PT kX p q I p q I
+ +?
∴= == +
为奇数为偶数
1
010
1
211
1
2
(31) (2 1)
(1 )
T
k
kk
k
PX X PT kX
ppq
ppq
∞
=
∞
=
= == = =
=
=?
∑
∑
3.20,
(1)
58 14 18
13 12 16
34 14 0
P
=
因为有限状态的马氏链平稳分布一定存在,
Pππ =∴,
解方程组得
)8110,31,8144(π =
又因为非周期不可约链的极限分布就是它的平稳分布,
j
n
ijn
p πlim
)(
=∴
∞→
811288110331281441)1(lim
0
=×+×+×==∴
∞→
XXE
n
n
(2)
}1,1,,11,1,:min{
}1,:min{
},11,1,0:min{
},0:min{
1011
11
0
01
=?≤≤∈?≤≤=>=
=>=
∈?≤≤=>=
∈>=
nji
n
ni
n
XnjTSXTiXTnn
XTnn
SXniXnn
SXnnT
τ
因此
}{}{
11
nnT
II
== τ
和均只是
n
XX,,
1
null的函数,即
11
τ和T 都是关于}0,{ ≥nX
n
的停时,
(3)
通过观察状态图,可以得到,
()()
1
00
1
1
11 12 13
1
()
(1,1 1,1)
()58418
35 8
lk
k
k
kk
PT k
PX l k X S X
ppp
=
==≤≤?∈=
=?+=?+
=?
3885
853)(
1
11
1
1
1
==
=∴
∑
∑
∞
=
∞
=
k
kk
k
kk
kTE
由上面(2)可知'
111
ττ +=T,其中'
1
τ为PH分布,其对应的状态空间为''
0
SSS ∪=,}3,2{'=S为瞬时态集,}1{'
0
=S为吸收态,
一步转移概率矩阵为
=
858141
43041
316121
~
P
可以把上面的一步转移概率矩阵看成
=
100
43041
316121
'
~
P,
而不影响对PH分布的计算结果,其中,
=
=
43
31
,
041
6121
0
PP,
确定这个过程的初始分布()αα,)0(π
0
=,由上可知
32)()2(
131212
1
=+== pppXP
T
,
31)()3(
131213
1
=+== pppXP
T
,
所以()31,32,0)0(π =,其中)31,32(=α,
0
1
1
)'( PPkP
k?
==∴ ατ
11
10
1
1
(') ( )
12 16 1
(2 3 1 3)
14 1 1
74 33
k
k
E kP P I P eταα
∞
=
∴= =?
=
=
∑
111
() ( ') 837433 5411EETττ=+=+ =
(4)
()
1230
3
((3) 0)
(1,1,1 1)
58 125512
PN
PX X X X
=
=====
==
11 3
302 100
21
12 13 12 13
31
((3) 2)
(1,2,3)
(,1,1)
()()
13
(1 4 1 8) 3 8
34
17 256
PN
PT T
PX S X X S X
p
pp pp
p
τ
=
====
=∈ =∈=
=+
=
=
((3) 2) 0
( (3) 1) 1 125 512 17 256 353 512
PN
PN
>=
== =
同理,
11 2 11 2
112 112
( (4) 2)
(1,2,3)(1,2,4)
(1,3,4)(2,3,4)
185 1024
PN
PT T PT T
PT T PT T
ττ
=
====+===
+===+===
=
