EXPECTATIONS AND CONDITIONAL EXPECTATIONS 1
Expectations and Conditional Expectations
Definition 1 Discrete Random Variable
A random variable is discrete if the set of outcomes is either finite in number or countably
infinite.
For a discrete random variable,
f (x) = Prob(X = x).
Example 1 Binomial Distribution
P (X = x|n,p) =
parenleftBiggn
x
parenrightBigg
px (1?p)n?x ; x = 0,1,2,...,n; 0 ≤ p ≤ 1
Definition 2 Continuous Random Variable
The random variable is continuous if the set of outcomes is infinitely divisible and, hence,
not countable.
For a continuous random variable,
Prob(a ≤ x ≤ b) =
integraldisplay b
a
f (x)dx ≥ 0.
Example 2 Normal Distribution
f
parenleftBig
x|μ,σ2
parenrightBig
= 1√2piσ exp
braceleftBigg
?12
parenleftbiggx?μ
σ
parenrightbigg2bracerightBigg
, ?∞ < x < ∞, ?∞ < μ < ∞, σ > 0
Axioms of probability require that
1. 0 ≤Prob(X = x) ≤ 1.
2. summationtextx f (x) = 1.
Definition 3 Expectation of a Random Variable
The mean, or expected value, of a random variable is
E [x] =
braceleftBigg summationtext
x xf (x) if x is discreteintegraltext
x xf (x)dx if x is continuous
Remark 1 Linearity of expectation
E [a + bx] = a + bE [x]
Remark 2
E
bracketleftBig
x2
bracketrightBig
negationslash= (E [x])2
and
E
bracketleftBig
x2
bracketrightBig
?(E [x])2 = σ2
EXPECTATIONS AND CONDITIONAL EXPECTATIONS 2
Example 3
x ～ N (0,1)
E [1 + 5x] = 1 + 5E [x] = 1 + 5 × 0 = 1
E
bracketleftBig
1 + 5x2
bracketrightBig
= 1 + 5E
bracketleftBig
x2
bracketrightBig
= 1 + 5 ×1 = 6
Definition 4 For two random variables, x and y, we say that the conditional distribution
of y given x is
f (y|x) = f (x,y)f
x (x)
Remark 3 If x and y are independent,
f (y|x) = fy (y)
Definition 5 A conditional mean (or conditional expectation) is the mean of the condi-
tional distribution and is defined by
E [y|x] =
braceleftBigg integraltext
y yf (y|x)dy if y is continuoussummationtext
y yf (y|x) if y is discrete
Note that
E [y|x] = E [y]
if x and y are independent.
Example 4
y = bx + ε
ε ～ N (0,1)
ε and x are independent.
E [y|x] = E [bx + ε|x]
= E [bx|x] + E [ε|x]
Since ε and x are independent, E [ε|x] = E [ε] = 0,
E [y|x] = bx
(Reference: See Greene Appendix B, P. 845—865)
MATRIX ALGEBRA 3
Matrix Algebra
A matrix is an array of numbers,
A = [aik] = [A]ik =
?
??
??
a11 a12 · · · a1K
a21 a22 · · · a2K
· · ·
an1 an2 · · · anK
?
??
??, i = 1,...,n, k = 1,...,K
where i denotes the row number and k denotes the column number.
Column vector: A matrix with only one column.
Example 5
vi =
?
??
??
?
a1
a2
...
an
?
??
??
?
Row vector: A matrix with only one row
Example 6
vk =
bracketleftBig
a1 a2 · · · aK
bracketrightBig
Symmetric matrix (aik = aki)
Example 7
A =
?
?? 1 2 32 1 2
3 2 1
?
??
Diagonal matrix (aik = 0, i negationslash= k)
Example 8
A =
?
?? 1 0 00 2 0
0 0 3
?
??
Scalar matrix (aik = 0, aii = akk, i negationslash= k)
Example 9
A =
?
?? 3 0 00 3 0
0 0 3
?
??
Identity matrix (aik = 0, aii = akk = 1, i negationslash= k)
MATRIX ALGEBRA 4
Example 10
A =
?
?? 1 0 00 1 0
0 0 1
?
??
Upper triangular matrix (aik = 0, i > k)
Example 11
A =
?
?? 1 2 30 1 2
0 0 1
?
??
Lower triangular matrix (aik = 0, i < k)
Example 12
A =
?
?? 1 0 02 1 0
3 2 1
?
??
Matrix Manupulations
Transposition
For two matrices, A and B, B is the transpose of A implies that
bik = aki
Example 13
A =
bracketleftBigg 1 2 3
4 5 6
bracketrightBigg
, B = Aprime =
?
?? 1 42 5
3 6
?
??
Matrix Addition and Subtraction
For two matrices, A and B, matrix addition and subtraction are defined by
A+B = [aik + bik]
A?B = [aik ?bik]
Example 14
A =
bracketleftBigg 1 2 3
4 5 6
bracketrightBigg
, B =
bracketleftBigg 0 1 0
1 0 1
bracketrightBigg
A+B =
bracketleftBigg 1 3 3
5 5 7
bracketrightBigg
, A?B =
bracketleftBigg 1 1 3
3 5 5
bracketrightBigg
MATRIX ALGEBRA 5
Inner Product (Dot Product)
For two column vectors a and b, their inner product (or dot product) is
aprimeb = bprimea = a1b1 + a2b2 + ··· + anbn
Example 15
a =
?
?? 12
3
?
??, b =
?
?? 32
1
?
??
aprimeb = bprimea = 1 × 3 + 2 × 2 + 3 × 1
= 10
Matrix Multiplication
Example 16
A =
bracketleftBigg 1 2 3
4 5 6
bracketrightBigg
2×3
, B =
?
?? 0 11 0
0 1
?
??
3×2
AB =
bracketleftBigg 1 × 0 + 2 × 1 + 3 × 0 1 × 1 + 2 × 0 + 3 × 1
4 × 0 + 5 × 1 + 6 × 0 4 × 1 + 5 × 0 + 6 × 1
bracketrightBigg
=
bracketleftBigg 2 4
5 10
bracketrightBigg
2×2
BA =
?
?? 0 × 1 + 1 × 4 0 × 2 + 1 × 5 0 × 3 + 1 × 61 × 1 + 0 × 4 1 × 2 + 0 × 5 1 × 3 + 0 × 6
0 × 1 + 1 × 4 0 × 2 + 1 × 5 0 × 3 + 1 × 6
?
?? =
?
?? 4 5 61 2 3
4 5 6
?
??
3×3
In general AB negationslash= BA.
? Associative law: (AB)C = A(BC)
? Distributive law: A(B+C) = AB+AC
? Transpose of a product: (AB)prime = BprimeAprime
? Transpose of an extended product: (ABC)prime = CprimeBprimeAprime
Sum of Values
Denote i a vector that contains a column of ones. Then,
nsummationdisplay
i=1
xi = x1 + ···xn = iprimex
Example 17
x =
?
?? 12
3
?
??, i =
?
?? 11
1
?
??, iprimex = 1 × 1 + 1 × 2 + 1 × 3 = 6
MATRIX ALGEBRA 6
Sum of Squares
The sum of squares of the elements in a column vector x is
nsummationdisplay
i=1
x2i = xprimex
Example 18
x =
?
?? 12
3
?
??, xprimex = 1×1+2×2+3×3 = 14
Sum of Products
The sum of the products of two column vectors x and y is
nsummationdisplay
i=1
xiyi = xprimey = yprimex
(Reference: See Greene Appendix A, P. 803—845)
