Random variables ∑

Random variables M. Veeraraghavan A random variable is a rule that assigns a numerical value to each possible outcome of an experiment. Outcomes of an experiment form the sample space S . Definition: A random variable X on a sample space S is a function X : S → ℜ that assigns a real number X ( s ) to each sample point s ∈ S .

We define the event space A x to be the subset of S to which the random variable X assigns the value x . Ax = { s ∈ S X ( s ) = x }

(1)

Image of a rv X is the set of all values taken by X . Inverse image is A x . Example: Take 3 Bernoulli trials. Sample space has 8 possible outcomes: 000, 001, 010, 100, 011, 110, 101, 111. But if we only interested in the number of successes, then 001, 100, 010 sample points map to the value 1 (for number of 1s within the three trials). Similarly, 011, 110, 101 map to 2. The event space has only 4 outcomes 0, 1, 2, 3 instead of the original sample space which has 8 points.

Typically, we are only interested in the event space A x rather than in the whole sample space S if our interest lies only in the experimental values of the rv X . Discrete random variables: A random variable is discrete if it can take on values from a discrete set of numbers. Probability mass function (pmf): P ( Ax ) =

∑

P ( s ) = P [ X = x ] = pX ( x )

X(s) = x

Probabiltiy distribution function or Cumulative distribution function (cdf):

1

(2)

FX ( t ) = P ( X ≤ t ) =

∑ pX ( x )

(3)

x≤t

Often we will see the statement, “X is a discrete rv with pmf P X ( x ) “ with no hint or mention of the sample space. Know the pmf, cdf, mean (expected value) and variance for the following types of discrete random variables: 1. Bernoulli - one parameter p (of success) 2. Binomial - count number of successes in n independent trials with p as probability of success of each trial - two parameters n and p . 3. Geometric - count number of trials upto 1st success (while binomial counts number of successes) - one parameter p pX ( i ) = p ( 1 – p )

i–1

FX ( t ) = 1 – ( 1 – p )

for i = 1, 2, … and t

for ( t ≥ 0 )

(4) (5)

4. Negative binomial - count number of trials until r th success - two parameters p and r . 5. Poisson - approximation of binomial if n is large and p is small. One parameter α = np . –α k

e α p X ( k ) = -------------k!

for k = 0, 1, 2, …

(6)

6. Hypergeometric - sampling without replacement while binomial is sampling with replacement; N components of which d are defective. In a sample of n components what is the probability that k are defective. 7. Uniform - one parameter - range N . 8. Constant - one parameter c 9. Indicator - one parameter p . Continuous random variables: Probability density function (pdf): dF X ( x ) f X ( x ) = ----------------dx

2

(7)

Probabiltiy distribution function or Cumulative distribution function (cdf): t

∫ fX ( r ) dr

FX ( t ) = P ( X ≤ t ) =

(8)

–∞

Know the pdf, cdf, mean (expected value) and variance for the following types of random variables: 1. Exponential - 1 parameter λ  1 – e – λx F(x) =  0 

if ( 0 ≤ x < ∞ ) otherwise

 λe –λx f( x) =   0

if ( x > 0 ) otherwise

(9)

(10)

1 E [ X ] = --λ

(11)

1 Var [ X ] = ----2λ

(12)

2. Gamma, chi-square, Student-t distributions 3. Erlang distribution 4. Hypoexponential distribution 5. Hyperexponential distribution 6. Normal (Gaussian) distribution 7. Pareto distribution 8. Weibull distribution Relationships between random variables 1. Mutually exclusive events P(A ∪ B) = P(A) + P(B ) ; P(A ∩ B) = 0 .

(13)

In general P ( A ∪ B ) = P ( A ) + P ( B ) – P ( A ∩ B )

(14)

2. Independent events P ( A ∩ B ) = P ( A )P ( B ) 3. Bayes rule

3

(15)

P ( A B j )P ( B j ) P ( B j A ) = ----------------------------------------∑ P ( A Bi )P ( Bi )

(16)

i

4. Theorem of total probability n

P(A) =

∑ P ( A Bi )P ( Bi )

(17)

i=1

5. Correlation of two random variables r XY = E [ XY ] E [ XY ] =

∑ ∑

(18)

xyP X, Y ( x, y ) for a discrete r.v.

(19)

x ∈ SX y ∈ SY

If X and Y are independent random variables then because P X, Y ( x, y ) = P X ( x )P Y ( y ) , 2

E [ XY ] = E [ X ]E [ Y ] (see [2, page 107]). Second moment of a single random variable is E [ X ] . 6. Covariance of two random variables cov ( X, Y ) = E [ X – µ X ] [ Y – µ Y ] = E [ XY ] – µ X µ Y , where µ X = E [ X ] and µ Y = E [ Y ] . Relate this to variance of a single random variable 2

2

Var [ X ] = E [ ( X – µ X ) ] = E [ X 2 ] – µ X

(20)

cov ( X, Y ) ρ XY = --------------------------------------var ( X )var ( Y )

(21)

7. Correlation coefficient:

Memoryless property: Two distributions, the exponential continuous r.v. distribution and geometric discrete r.v. distribution enjoy this property. Let X be the lifetime of a component. Suppose we have observed that the system has been operational for time t . We would like to know the probability that it will be operational for y more hours. Say Y = X – t . We will show that

4

P ( Y ≤ y X > t ) = P ( Y ≤ y ) , in other words, how long it lasts beyond some time t is independent of how long the component has been operational so far, i.e. t . P(Y ≤ y X > t) = P(X – t ≤ y X > t) = P(X ≤ y + t X > t)

(22)

P ( X ≤ y + t and X > t ) P(t < X ≤ y + t) = ----------------------------------------------------- = ------------------------------------P(X > t) P(X > t) Exponential distribution: (y + t)

∫ P( Y ≤ y X > t) =

λe

– λx

dx

t ------------------------------∞ – λx

∫ λe

dx

– λt

– λy

e ( 1 – e -) – λy = -------------------------------= (1 – e ) = P(Y ≤ y) – λt e

(23)

t

Geometric distribution: Let X be a rv with a geometric distribution and Y = X – t . We will show that P( Y ≤ y X > t) = P( Y ≤ y ) . y+t

t

t

y

1 – (1 – p) – ( 1 – ( 1 – p ) )( 1 – p ) ( 1 – ( 1 – p ) )P ( Y ≤ y X > t ) = --------------------------------------------------------------------------------= -----------------------------------------------------------= P(Y ≤ y) t t (1 – p) (1 – p) Note: Y = X – t has the same distribution as X because t is a constant. See the stat lecture to see distributions of functions of random variables. Relation between exponential distribution (for continuous rv) and geometric distribution (for discrete rv): [2] If X is an exponential random variable with parameter a , then K =

X is a geometric random

–a

variable with parameter p = 1 – e . References [1] [2]

K. S. Trivedi, “Probability, Statistics with Reliability, Queueing and Computer Science Applications,” Second Edition, Wiley, 2002, ISBN 0-471-33341-7. R. Yates and D. Goodman, “Probability and Stochastic Processes,” Wiley, ISBN 0-471-17837-3.

5