CHAPTER 2
A REVIEW OF BASIC
STATISTICAL CONCEPTS
2.1 SOME NOTATION
? 1,The Summation Notation
can be abbreviated as,or
? 2,Properties of the Summation Operator
??? ????nii ni XXXX1 21 ?
???nii iX1 ?iX ?
X X
?? ?ni nkk1
? ?? ii XkkX
? ?? ??? iiii YXYX )(
? ???? ii XbnabXa )(
2.2 EXPERIMENT,SAMPLE SPACE,SAMPLE
POINT,AND EVENTS
1,Experiment
A statistical/random experiment,a process
leading to at least two possible outcomes with
uncertainty as to which will occur.
2.Sample space or population
The population or sample space,the set of all
possible outcomes of an experiment
3,Sample Point
Sample Point, each member,or outcome,of the
sample space (or population)
2.2 EXPERIMENT,SAMPLE SPACE,
SAMPLE POINT,AND EVENTS
? 4,Events
An event,a collection of the possible outcomes of an
experiment; that is,it is a subset of the sample space.
Mutually exclusive events,the occurrence of one
event prevents the occurrence of another event at the
same time.
Equally likely events,one event is as likely to occur as
the other event.
Collectively exhaustive events,events that exhaust all
possible outcomes of an experiment
2.3 RANDOM VARIABLES
? A random/stochastic variable（ r.v.,for short）, a
variable whose (numerical) value is determined
by the outcome of an experiment.
? （ 1） A discrete random variable—— an r.v,that
takes on only a finite (or accountably
infinite) number of values,
? （ 2） A continuous random variable—— an r.v,
that can take on any value in some interval
of values.
2.4 PROBABILITY
? 1,The Classical or A Priori Definition,if an
experiment can result in n mutually exclusive
and equally likely outcomes,and if m of
these outcomes are favorable to event A,then
P(A),the probability that A occurs,is m/n
Two features of the probability:
（ 1） The outcomes must be mutually exclusive;
（ 2） Each outcome must have an equal chance of
occurring.
o u t c o m es ofn u m b e r t o t a lt h e
A t of a v o r ab l e o u t c o m es ofn u m b e r t h e
)(
?
? nmAP
2.4 PROBABILITY
? 2.Relative Frequency or Empirical Definition
Frequency distribution,how an r.v,are distributed.
Absolute frequencies,the number of occurrence of a
given event.
Relative frequencies,the absolute frequencies divided by
the total number of occurrence.
Empirical Definition of Probability,if in n trials（ or
observations）,m of them are favorable to event A,then
P（ A）,the probability of event A,is simply the ration m/n,
（ that is,relative frequency） provided n,the number of
trials,is sufficiently large
In this definition,we do not need to insist that the outcome be
mutually exclusive and equally likely.
2.4 PROBABILITY
? 3,Properties of probabilities
(1) 0≤P(A)≤1
(2) If A,B,C,..,are mutually exclusive events,
then,P(A+B+C+...)=P(A)+P(B)+P(C)+...
(3) If A,B,C,..,are mutually exclusive and
collectively exhaustive set of events,
P(A+B+C+...)=P(A)+P(B)+P(C)+...=1
2.4 PROBABILITY
Rules of probability:
? 1) If A,B,C,...are any events,they are said to be
statistically independent events if:
P(ABC...)=P(A)P(B)P(C)
? 2) If events A,B,C,..,are not mutually exclusive,then
P(A+B)=P(A)+P(B)－ P(AB)
Conditional probability of A,given B
Conditional probability of B,given A
)( )()|( BP ABPBAP ?
)( )()|( AP ABPABP ?
2.5 RANDOM VARIABLES AND PROBABILITY
DISTRIBUTION FUNCTION (PDF)
? The probability distribution function or probability density function
(PDF) of a random variable X,the values taken by that random
variable and their associated probabilities.
? 1.PDF of a Discrete Random Variable
Discrete r.v.（ X） takes only a finite（ or countably infinite）
number of values.
? Probability distribution or probability density function (PDF)—it
shows how the probabilities are spread over or distributed over the
various values of the random variable X.
? PDF of a discrete r.v.(X)
f(X)= P(X=Xi) for i=1,2,3...,n And
0 for X≠Xi
2.5 RANDOM VARIABLES AND PROBABILITY
DISTRIBUTION FUNCTION (PDF)
? 2,PDF of a Continuous Random Variable
The probability for a continuous r.v,is always measured over
an interval
For a continuous r.v,the probability that such an r.v,takes a
particular numerical value is always zero.
? 3,Cumulative Distribution Function (CDF)
F(X)=P(X ≤x)
P(X ≤x),the probability that the r.v,X takes a value of less
than or equal to x,where x is given.
In fact,a CDF is merely an,accumulation” or simply the sum of
the PDF for the values of X less than or equal to a given x.
CDF of a discrete r.v.,step function;
CDF of a continuous r.v.,continuous curve
2.6 MULTIVARIATE PROBABILITY DENSITY
FUNCTIONS
? 1,Two-variate/Bivariate PDF
Single/Univariate probability distribution functions
Multivariate probability distribution functions
Two-variate/Bivariate PDF
joint probability,the prob,that the r.v,X takes a
given value and Y takes a given value,
Bivariate/joint PDF～ f(X,Y)
? （ 1） Discrete joint PDF,
f(X,Y)=P(X=x,Y=y)
=0 when X≠x,Y≠y
? （ 2） Continuous joint PDF,（ omitted）
2.6 MULTIVARIATE PROBABILITY DENSITY
FUNCTIONS
?2,Marginal Probability Density Function
The relationship between the univariate PDFs,f(X) or f(Y),
and the bivariate joint PDF,
The marginal probability of X,the probability that X
assumes a given value regardless of the values taken by Y.
The marginal PDF of X,the distribution of the marginal
probabilities.——sum the joint prob,corresponding to the
given value of X regardless of the values taken by Y.
The marginal PDF of Y,sum the joint prob,corresponding
to the given value of Y regardless of the values taken by X.
2.6 MULTIVARIATE PROBABILITY DENSITY
FUNCTIONS
? 3,Conditional Probability Density Function
（ 1） discrete conditional PDFS
conditional PDF of Y,f(Y|X)=P(Y=y|X=x),
conditional PDF of X,f(X|Y)=P(X=x|Y=y)
The conditional PDF of one variable,given the value of the
other variable,is simply the ratio of the joint probability of the
two variables divided by the marginal or unconditional PDF of
the other(i.e.,the conditioning ) variable.
Xofprobabilitymarginalthe
YandXofprobabilityjoint the
)( X
),()|(
?
?
f
YXfXYF
Yofprobabilitymarginalthe
YandXofprobabilityjoint the
)(
),()|(
?
?
Yf
YXfYXF
2.6 MULTIVARIATE PROBABILITY
DENSITY FUNCTIONS
?4,Statistical Independence
Independent random variables:
Two variables X and Y are statistically
independent if and only if their joint PDF can be
expressed as the product of their individual,or
marginal,PDFs for all combinations of X and Y
values,
f(X,Y)=f(X)f(Y)
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
? 1,Expected Value,A Measure of Central Tendency
（ 1） Definition
Expected Value,a measure of central tendency,gives
the center of gravity.The expected value of a discrete
r.v,is the sum of products of the values taken by the r.v,
and their corresponding probabilities.
（ 2） Properties of expected value.
If a and b are constants,then
E(b)=b
E(X+Y)=E(X)+E(Y) E(XY)=E(X)E(Y),if X,Y are independent r.v.
E(aX)=aE(X)
E(aX+b)=aE(X)+E(b)=aE(X)+b
?? X XXfxE )()(
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
? 2,Variance,A Measure of Dispersion
（ 1） Definition of the variance of X,the expected value of
the squared difference between an individual X value and
its expected or mean value.
（ 2） Compute the variance
Discrete:
（ 3） Properties of variance
If a and b are constants var(a)=0
If X,Y are two independent r.v.,then
var(X+Y)=var(X)+var(Y)
var(X-Y)=var(X)+var(Y)
var(X+b)=var(X)
var(aX)=a2var(X)
var(aX+b)=a2var(X)
var(aX+bY)=a2var(X)+b2var(Y)
22 )()v ar ( xx XEX ?? ???
)()()v ar ( 2 XfXX X x? ?? ?
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
? 3,Covariance—— characteristics of multivariate PDFs
（ 1） Definition,Covariance is a measure of how two variables vary or
move together.
If the two r.v,move in the same direction,then the cov
cov(X,Y)=E[(X-μx)(Y-μy)] =E(XY)-μxμy
Discrete:
（ 2） Properties of covariance
If X and Y are independent r.v.,then cov(X,Y) =0
If a,b,c,d are constants,then
cov(a+bX,c+dY)=bdcov(X,Y)
cov(X,X)=var(X)
),())((),co v ( YXfYXYX yX Y x ?? ??? ? ? yX Y xYXX Y f ??? ? ?? ),(
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
?4,Correlation Coefficient
（ 1） Definition of the correlation coefficient,a measure of
linear association between two variables,i.e.,how strongly
the two variables are linearly related.
（ 2） Properties of correlation coefficient
－ 1 ≤ρ≤1 If 0≤ρ≤1 two variables are positively correlated
－ 1 ≤ρ≤0 negatively correlated
（ 3） Variances of correlated variables
If X,Y are not independent,that is,they are correlated,then
var(X+Y)=var(X)+var(Y)+2cov(X,Y)
var(X-Y)=var(X)+var(Y)-2cov(X,Y)
yx
YX
??r
),cov(?
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
? 5,Conditional Expectation
unconditional expectation.
conditional expectation
6,The Skewness and the Kurtosis of a PDF
moments of the PDF of a r.v,X
first moment,E(X)=μx second moment, E(X-μx)2
third moment,E(X-μx)3 n-th moment,E(X-μx)n
（ 1） Skewness (S) is a measure of asymmetry
If s=0 the PDF is symmetrical,
s>0 the PDF is positively skewed,
s<0 the PDF is negatively skewed
? ??? X yYXXfyYXE )()|(
? ??? Y xXYYfxXYE )|()|(
2/32
3
)]([ )]([ x xXE XES ? ?? ??
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
6,The Skewness and the Kurtosis of a PDF
（ 2） kurtosis (K) is a measure of tallness or
flatness of a PDF
K<3,the PDF is platykurtic（ fat or short-tailed）
K>3,the PDF is leptokurtic （ slim or long-tailed）
K=3,the PDF is mesokurtic.
Computation of the Third moment,E(X-μx)3f(X)
Computation of the Fourth moment,E(X-μx)4f(X)
momentSquare of second
momentfourth
)]([
)]([
22
4
?
?
??
x
x
XE
XES
?
?
2.8 From Population to the Sample
? The sample Mean:
? The Sample Variance:
? The Sample Covariance:
? The Sample Correlation Coefficient
? Sample Skewness:
? Sample Kurtosis:
Note,the sample value is the estimator of the population
one,that is,a rule or formula that tells us how to go about
estimating a population quantity．
??? ni inXX 1
?? ??? ni ix n XXS 1 22 1 )(
1-n )Y-) ( YX-(XY)co v ( X,S am p le ii??
).(.).(.
),(c o v s am p le
)1/())((
1
YdsXds
YX
SS
nYYXX
r
yx
n
i
ii
?
???
?
?
?
1 )(
3
??? n XX
1 )(
4
??? n XX
A REVIEW OF BASIC
STATISTICAL CONCEPTS
2.1 SOME NOTATION
? 1,The Summation Notation
can be abbreviated as,or
? 2,Properties of the Summation Operator
??? ????nii ni XXXX1 21 ?
???nii iX1 ?iX ?
X X
?? ?ni nkk1
? ?? ii XkkX
? ?? ??? iiii YXYX )(
? ???? ii XbnabXa )(
2.2 EXPERIMENT,SAMPLE SPACE,SAMPLE
POINT,AND EVENTS
1,Experiment
A statistical/random experiment,a process
leading to at least two possible outcomes with
uncertainty as to which will occur.
2.Sample space or population
The population or sample space,the set of all
possible outcomes of an experiment
3,Sample Point
Sample Point, each member,or outcome,of the
sample space (or population)
2.2 EXPERIMENT,SAMPLE SPACE,
SAMPLE POINT,AND EVENTS
? 4,Events
An event,a collection of the possible outcomes of an
experiment; that is,it is a subset of the sample space.
Mutually exclusive events,the occurrence of one
event prevents the occurrence of another event at the
same time.
Equally likely events,one event is as likely to occur as
the other event.
Collectively exhaustive events,events that exhaust all
possible outcomes of an experiment
2.3 RANDOM VARIABLES
? A random/stochastic variable（ r.v.,for short）, a
variable whose (numerical) value is determined
by the outcome of an experiment.
? （ 1） A discrete random variable—— an r.v,that
takes on only a finite (or accountably
infinite) number of values,
? （ 2） A continuous random variable—— an r.v,
that can take on any value in some interval
of values.
2.4 PROBABILITY
? 1,The Classical or A Priori Definition,if an
experiment can result in n mutually exclusive
and equally likely outcomes,and if m of
these outcomes are favorable to event A,then
P(A),the probability that A occurs,is m/n
Two features of the probability:
（ 1） The outcomes must be mutually exclusive;
（ 2） Each outcome must have an equal chance of
occurring.
o u t c o m es ofn u m b e r t o t a lt h e
A t of a v o r ab l e o u t c o m es ofn u m b e r t h e
)(
?
? nmAP
2.4 PROBABILITY
? 2.Relative Frequency or Empirical Definition
Frequency distribution,how an r.v,are distributed.
Absolute frequencies,the number of occurrence of a
given event.
Relative frequencies,the absolute frequencies divided by
the total number of occurrence.
Empirical Definition of Probability,if in n trials（ or
observations）,m of them are favorable to event A,then
P（ A）,the probability of event A,is simply the ration m/n,
（ that is,relative frequency） provided n,the number of
trials,is sufficiently large
In this definition,we do not need to insist that the outcome be
mutually exclusive and equally likely.
2.4 PROBABILITY
? 3,Properties of probabilities
(1) 0≤P(A)≤1
(2) If A,B,C,..,are mutually exclusive events,
then,P(A+B+C+...)=P(A)+P(B)+P(C)+...
(3) If A,B,C,..,are mutually exclusive and
collectively exhaustive set of events,
P(A+B+C+...)=P(A)+P(B)+P(C)+...=1
2.4 PROBABILITY
Rules of probability:
? 1) If A,B,C,...are any events,they are said to be
statistically independent events if:
P(ABC...)=P(A)P(B)P(C)
? 2) If events A,B,C,..,are not mutually exclusive,then
P(A+B)=P(A)+P(B)－ P(AB)
Conditional probability of A,given B
Conditional probability of B,given A
)( )()|( BP ABPBAP ?
)( )()|( AP ABPABP ?
2.5 RANDOM VARIABLES AND PROBABILITY
DISTRIBUTION FUNCTION (PDF)
? The probability distribution function or probability density function
(PDF) of a random variable X,the values taken by that random
variable and their associated probabilities.
? 1.PDF of a Discrete Random Variable
Discrete r.v.（ X） takes only a finite（ or countably infinite）
number of values.
? Probability distribution or probability density function (PDF)—it
shows how the probabilities are spread over or distributed over the
various values of the random variable X.
? PDF of a discrete r.v.(X)
f(X)= P(X=Xi) for i=1,2,3...,n And
0 for X≠Xi
2.5 RANDOM VARIABLES AND PROBABILITY
DISTRIBUTION FUNCTION (PDF)
? 2,PDF of a Continuous Random Variable
The probability for a continuous r.v,is always measured over
an interval
For a continuous r.v,the probability that such an r.v,takes a
particular numerical value is always zero.
? 3,Cumulative Distribution Function (CDF)
F(X)=P(X ≤x)
P(X ≤x),the probability that the r.v,X takes a value of less
than or equal to x,where x is given.
In fact,a CDF is merely an,accumulation” or simply the sum of
the PDF for the values of X less than or equal to a given x.
CDF of a discrete r.v.,step function;
CDF of a continuous r.v.,continuous curve
2.6 MULTIVARIATE PROBABILITY DENSITY
FUNCTIONS
? 1,Two-variate/Bivariate PDF
Single/Univariate probability distribution functions
Multivariate probability distribution functions
Two-variate/Bivariate PDF
joint probability,the prob,that the r.v,X takes a
given value and Y takes a given value,
Bivariate/joint PDF～ f(X,Y)
? （ 1） Discrete joint PDF,
f(X,Y)=P(X=x,Y=y)
=0 when X≠x,Y≠y
? （ 2） Continuous joint PDF,（ omitted）
2.6 MULTIVARIATE PROBABILITY DENSITY
FUNCTIONS
?2,Marginal Probability Density Function
The relationship between the univariate PDFs,f(X) or f(Y),
and the bivariate joint PDF,
The marginal probability of X,the probability that X
assumes a given value regardless of the values taken by Y.
The marginal PDF of X,the distribution of the marginal
probabilities.——sum the joint prob,corresponding to the
given value of X regardless of the values taken by Y.
The marginal PDF of Y,sum the joint prob,corresponding
to the given value of Y regardless of the values taken by X.
2.6 MULTIVARIATE PROBABILITY DENSITY
FUNCTIONS
? 3,Conditional Probability Density Function
（ 1） discrete conditional PDFS
conditional PDF of Y,f(Y|X)=P(Y=y|X=x),
conditional PDF of X,f(X|Y)=P(X=x|Y=y)
The conditional PDF of one variable,given the value of the
other variable,is simply the ratio of the joint probability of the
two variables divided by the marginal or unconditional PDF of
the other(i.e.,the conditioning ) variable.
Xofprobabilitymarginalthe
YandXofprobabilityjoint the
)( X
),()|(
?
?
f
YXfXYF
Yofprobabilitymarginalthe
YandXofprobabilityjoint the
)(
),()|(
?
?
Yf
YXfYXF
2.6 MULTIVARIATE PROBABILITY
DENSITY FUNCTIONS
?4,Statistical Independence
Independent random variables:
Two variables X and Y are statistically
independent if and only if their joint PDF can be
expressed as the product of their individual,or
marginal,PDFs for all combinations of X and Y
values,
f(X,Y)=f(X)f(Y)
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
? 1,Expected Value,A Measure of Central Tendency
（ 1） Definition
Expected Value,a measure of central tendency,gives
the center of gravity.The expected value of a discrete
r.v,is the sum of products of the values taken by the r.v,
and their corresponding probabilities.
（ 2） Properties of expected value.
If a and b are constants,then
E(b)=b
E(X+Y)=E(X)+E(Y) E(XY)=E(X)E(Y),if X,Y are independent r.v.
E(aX)=aE(X)
E(aX+b)=aE(X)+E(b)=aE(X)+b
?? X XXfxE )()(
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
? 2,Variance,A Measure of Dispersion
（ 1） Definition of the variance of X,the expected value of
the squared difference between an individual X value and
its expected or mean value.
（ 2） Compute the variance
Discrete:
（ 3） Properties of variance
If a and b are constants var(a)=0
If X,Y are two independent r.v.,then
var(X+Y)=var(X)+var(Y)
var(X-Y)=var(X)+var(Y)
var(X+b)=var(X)
var(aX)=a2var(X)
var(aX+b)=a2var(X)
var(aX+bY)=a2var(X)+b2var(Y)
22 )()v ar ( xx XEX ?? ???
)()()v ar ( 2 XfXX X x? ?? ?
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
? 3,Covariance—— characteristics of multivariate PDFs
（ 1） Definition,Covariance is a measure of how two variables vary or
move together.
If the two r.v,move in the same direction,then the cov
cov(X,Y)=E[(X-μx)(Y-μy)] =E(XY)-μxμy
Discrete:
（ 2） Properties of covariance
If X and Y are independent r.v.,then cov(X,Y) =0
If a,b,c,d are constants,then
cov(a+bX,c+dY)=bdcov(X,Y)
cov(X,X)=var(X)
),())((),co v ( YXfYXYX yX Y x ?? ??? ? ? yX Y xYXX Y f ??? ? ?? ),(
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
?4,Correlation Coefficient
（ 1） Definition of the correlation coefficient,a measure of
linear association between two variables,i.e.,how strongly
the two variables are linearly related.
（ 2） Properties of correlation coefficient
－ 1 ≤ρ≤1 If 0≤ρ≤1 two variables are positively correlated
－ 1 ≤ρ≤0 negatively correlated
（ 3） Variances of correlated variables
If X,Y are not independent,that is,they are correlated,then
var(X+Y)=var(X)+var(Y)+2cov(X,Y)
var(X-Y)=var(X)+var(Y)-2cov(X,Y)
yx
YX
??r
),cov(?
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
? 5,Conditional Expectation
unconditional expectation.
conditional expectation
6,The Skewness and the Kurtosis of a PDF
moments of the PDF of a r.v,X
first moment,E(X)=μx second moment, E(X-μx)2
third moment,E(X-μx)3 n-th moment,E(X-μx)n
（ 1） Skewness (S) is a measure of asymmetry
If s=0 the PDF is symmetrical,
s>0 the PDF is positively skewed,
s<0 the PDF is negatively skewed
? ??? X yYXXfyYXE )()|(
? ??? Y xXYYfxXYE )|()|(
2/32
3
)]([ )]([ x xXE XES ? ?? ??
2.7 CHARACTERISTICS/MOMENTS OF
PROBABILITY DISTRIBUTIONS
6,The Skewness and the Kurtosis of a PDF
（ 2） kurtosis (K) is a measure of tallness or
flatness of a PDF
K<3,the PDF is platykurtic（ fat or short-tailed）
K>3,the PDF is leptokurtic （ slim or long-tailed）
K=3,the PDF is mesokurtic.
Computation of the Third moment,E(X-μx)3f(X)
Computation of the Fourth moment,E(X-μx)4f(X)
momentSquare of second
momentfourth
)]([
)]([
22
4
?
?
??
x
x
XE
XES
?
?
2.8 From Population to the Sample
? The sample Mean:
? The Sample Variance:
? The Sample Covariance:
? The Sample Correlation Coefficient
? Sample Skewness:
? Sample Kurtosis:
Note,the sample value is the estimator of the population
one,that is,a rule or formula that tells us how to go about
estimating a population quantity．
??? ni inXX 1
?? ??? ni ix n XXS 1 22 1 )(
1-n )Y-) ( YX-(XY)co v ( X,S am p le ii??
).(.).(.
),(c o v s am p le
)1/())((
1
YdsXds
YX
SS
nYYXX
r
yx
n
i
ii
?
???
?
?
?
1 )(
3
??? n XX
1 )(
4
??? n XX
