fbpx
Wikipedia

Independence (probability theory)

Independence is a fundamental notion in probability theory, as in statistics and the theory of stochastic processes.

Two events are independent, statistically independent, or stochastically independent if the occurrence of one does not affect the probability of occurrence of the other (equivalently, does not affect the odds). Similarly, two random variables are independent if the realization of one does not affect the probability distribution of the other.

When dealing with collections of more than two events, a weak and a strong notion of independence need to be distinguished. The events are called pairwise independent if any two events in the collection are independent of each other, while saying that the events are mutually independent (or collectively independent) intuitively means that each event is independent of any combination of other events in the collection. A similar notion exists for collections of random variables.

The name "mutual independence" (same as "collective independence") seems the outcome of a educational choice, merely to distinguish the stronger notion from "pairwise independence" which is a weaker notion. In the advanced literature of probability theory, statistics, and stochastic processes, the stronger notion is simply named independence with no modifier. It is stronger since independence implies pairwise independence, but not the other way around.

Contents

For events

Two events

Two events A {\displaystyle A} and B {\displaystyle B} are independent (often written as A B {\displaystyle A\perp B} or A B {\displaystyle A\perp \!\!\!\perp B} ) if and only if their joint probability equals the product of their probabilities:: p. 29: p. 10

P ( A B ) = P ( A ) P ( B ) {\displaystyle \mathrm {P} (A\cap B)=\mathrm {P} (A)\mathrm {P} (B)}

(Eq.1)

Why this defines independence is made clear by rewriting with conditional probabilities:

P ( A B ) = P ( A ) P ( B ) P ( A ) = P ( A B ) P ( B ) = P ( A B ) . {\displaystyle \mathrm {P} (A\cap B)=\mathrm {P} (A)\mathrm {P} (B)\iff \mathrm {P} (A)={\frac {\mathrm {P} (A\cap B)}{\mathrm {P} (B)}}=\mathrm {P} (A\mid B).}

and similarly

P ( A B ) = P ( A ) P ( B ) P ( B ) = P ( B A ) . {\displaystyle \mathrm {P} (A\cap B)=\mathrm {P} (A)\mathrm {P} (B)\iff \mathrm {P} (B)=\mathrm {P} (B\mid A).}

Thus, the occurrence of B {\displaystyle B} does not affect the probability of A {\displaystyle A} , and vice versa. Although the derived expressions may seem more intuitive, they are not the preferred definition, as the conditional probabilities may be undefined if P ( A ) {\displaystyle \mathrm {P} (A)} or P ( B ) {\displaystyle \mathrm {P} (B)} are 0. Furthermore, the preferred definition makes clear by symmetry that when A {\displaystyle A} is independent of B {\displaystyle B} , B {\displaystyle B} is also independent of A {\displaystyle A} .

Log probability and information content

Stated in terms of log probability, two events are independent if and only if the log probability of the joint event is the sum of the log probability of the individual events:

log P ( A B ) = log P ( A ) + log P ( B ) {\displaystyle \log \mathrm {P} (A\cap B)=\log \mathrm {P} (A)+\log \mathrm {P} (B)}

In information theory, negative log probability is interpreted as information content, and thus two events are independent if and only if the information content of the combined event equals the sum of information content of the individual events:

I ( A B ) = I ( A ) + I ( B ) {\displaystyle \mathrm {I} (A\cap B)=\mathrm {I} (A)+\mathrm {I} (B)}

See Information content § Additivity of independent events for details.

Odds

Stated in terms of odds, two events are independent if and only if the odds ratio of A {\displaystyle A} and B {\displaystyle B} is unity (1). Analogously with probability, this is equivalent to the conditional odds being equal to the unconditional odds:

O ( A B ) = O ( A ) and O ( B A ) = O ( B ) , {\displaystyle O(A\mid B)=O(A){\text{ and }}O(B\mid A)=O(B),}

or to the odds of one event, given the other event, being the same as the odds of the event, given the other event not occurring:

O ( A B ) = O ( A ¬ B ) and O ( B A ) = O ( B ¬ A ) . {\displaystyle O(A\mid B)=O(A\mid \neg B){\text{ and }}O(B\mid A)=O(B\mid \neg A).}

The odds ratio can be defined as

O ( A B ) : O ( A ¬ B ) , {\displaystyle O(A\mid B):O(A\mid \neg B),}

or symmetrically for odds of B {\displaystyle B} given A {\displaystyle A} , and thus is 1 if and only if the events are independent.

More than two events

A finite set of events { A i } i = 1 n {\displaystyle \{A_{i}\}_{i=1}^{n}} is pairwise independent if every pair of events is independent—that is, if and only if for all distinct pairs of indices m , k {\displaystyle m,k} ,

P ( A m A k ) = P ( A m ) P ( A k ) {\displaystyle \mathrm {P} (A_{m}\cap A_{k})=\mathrm {P} (A_{m})\mathrm {P} (A_{k})}

(Eq.2)

A finite set of events is mutually independent if every event is independent of any intersection of the other events: p. 11—that is, if and only if for every k n {\displaystyle k\leq n} and for every k {\displaystyle k} -element subset of events { B i } i = 1 k {\displaystyle \{B_{i}\}_{i=1}^{k}} of { A i } i = 1 n {\displaystyle \{A_{i}\}_{i=1}^{n}} ,

P ( i = 1 k B i ) = i = 1 k P ( B i ) {\displaystyle \mathrm {P} \left(\bigcap _{i=1}^{k}B_{i}\right)=\prod _{i=1}^{k}\mathrm {P} (B_{i})}

(Eq.3)

This is called the multiplication rule for independent events. Note that it is not a single condition involving only the product of all the probabilities of all single events; it must hold true for all subsets of events.

For more than two events, a mutually independent set of events is (by definition) pairwise independent; but the converse is not necessarily true.: p. 30

For real valued random variables

Two random variables

Two random variables X {\displaystyle X} and Y {\displaystyle Y} are independent if and only if (iff) the elements of the π-system generated by them are independent; that is to say, for every x {\displaystyle x} and y {\displaystyle y} , the events { X x } {\displaystyle \{X\leq x\}} and { Y y } {\displaystyle \{Y\leq y\}} are independent events (as defined above in Eq.1). That is, X {\displaystyle X} and Y {\displaystyle Y} with cumulative distribution functions F X ( x ) {\displaystyle F_{X}(x)} and F Y ( y ) {\displaystyle F_{Y}(y)} , are independent iff the combined random variable ( X , Y ) {\displaystyle (X,Y)} has a joint cumulative distribution function: p. 15

F X , Y ( x , y ) = F X ( x ) F Y ( y ) for all x , y {\displaystyle F_{X,Y}(x,y)=F_{X}(x)F_{Y}(y)\quad {\text{for all }}x,y}

(Eq.4)

or equivalently, if the probability densities f X ( x ) {\displaystyle f_{X}(x)} and f Y ( y ) {\displaystyle f_{Y}(y)} and the joint probability density f X , Y ( x , y ) {\displaystyle f_{X,Y}(x,y)} exist,

f X , Y ( x , y ) = f X ( x ) f Y ( y ) for all x , y . {\displaystyle f_{X,Y}(x,y)=f_{X}(x)f_{Y}(y)\quad {\text{for all }}x,y.}

More than two random variables

A finite set of n {\displaystyle n} random variables { X 1 , , X n } {\displaystyle \{X_{1},\ldots ,X_{n}\}} is pairwise independent if and only if every pair of random variables is independent. Even if the set of random variables is pairwise independent, it is not necessarily mutually independent as defined next.

A finite set of n {\displaystyle n} random variables { X 1 , , X n } {\displaystyle \{X_{1},\ldots ,X_{n}\}} is mutually independent if and only if for any sequence of numbers { x 1 , , x n } {\displaystyle \{x_{1},\ldots ,x_{n}\}} , the events { X 1 x 1 } , , { X n x n } {\displaystyle \{X_{1}\leq x_{1}\},\ldots ,\{X_{n}\leq x_{n}\}} are mutually independent events (as defined above in Eq.3). This is equivalent to the following condition on the joint cumulative distribution function F X 1 , , X n ( x 1 , , x n ) {\displaystyle F_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})} . A finite set of n {\displaystyle n} random variables { X 1 , , X n } {\displaystyle \{X_{1},\ldots ,X_{n}\}} is mutually independent if and only if: p. 16

F X 1 , , X n ( x 1 , , x n ) = F X 1 ( x 1 ) F X n ( x n ) for all x 1 , , x n {\displaystyle F_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})=F_{X_{1}}(x_{1})\cdot \ldots \cdot F_{X_{n}}(x_{n})\quad {\text{for all }}x_{1},\ldots ,x_{n}}

(Eq.5)

Notice that it is not necessary here to require that the probability distribution factorizes for all possible k {\displaystyle k} -element subsets as in the case for n {\displaystyle n} events. This is not required because e.g. F X 1 , X 2 , X 3 ( x 1 , x 2 , x 3 ) = F X 1 ( x 1 ) F X 2 ( x 2 ) F X 3 ( x 3 ) {\displaystyle F_{X_{1},X_{2},X_{3}}(x_{1},x_{2},x_{3})=F_{X_{1}}(x_{1})\cdot F_{X_{2}}(x_{2})\cdot F_{X_{3}}(x_{3})} implies F X 1 , X 3 ( x 1 , x 3 ) = F X 1 ( x 1 ) F X 3 ( x 3 ) {\displaystyle F_{X_{1},X_{3}}(x_{1},x_{3})=F_{X_{1}}(x_{1})\cdot F_{X_{3}}(x_{3})} .

The measure-theoretically inclined may prefer to substitute events { X A } {\displaystyle \{X\in A\}} for events { X x } {\displaystyle \{X\leq x\}} in the above definition, where A {\displaystyle A} is any Borel set. That definition is exactly equivalent to the one above when the values of the random variables are real numbers. It has the advantage of working also for complex-valued random variables or for random variables taking values in any measurable space (which includes topological spaces endowed by appropriate σ-algebras).

For real valued random vectors

Two random vectors X = ( X 1 , , X m ) T {\displaystyle \mathbf {X} =(X_{1},\ldots ,X_{m})^{\mathrm {T} }} and Y = ( Y 1 , , Y n ) T {\displaystyle \mathbf {Y} =(Y_{1},\ldots ,Y_{n})^{\mathrm {T} }} are called independent if: p. 187

F X , Y ( x , y ) = F X ( x ) F Y ( y ) for all x , y {\displaystyle F_{\mathbf {X,Y} }(\mathbf {x,y} )=F_{\mathbf {X} }(\mathbf {x} )\cdot F_{\mathbf {Y} }(\mathbf {y} )\quad {\text{for all }}\mathbf {x} ,\mathbf {y} }

(Eq.6)

where F X ( x ) {\displaystyle F_{\mathbf {X} }(\mathbf {x} )} and F Y ( y ) {\displaystyle F_{\mathbf {Y} }(\mathbf {y} )} denote the cumulative distribution functions of X {\displaystyle \mathbf {X} } and Y {\displaystyle \mathbf {Y} } and F X , Y ( x , y ) {\displaystyle F_{\mathbf {X,Y} }(\mathbf {x,y} )} denotes their joint cumulative distribution function. Independence of X {\displaystyle \mathbf {X} } and Y {\displaystyle \mathbf {Y} } is often denoted by X Y {\displaystyle \mathbf {X} \perp \!\!\!\perp \mathbf {Y} } . Written component-wise, X {\displaystyle \mathbf {X} } and Y {\displaystyle \mathbf {Y} } are called independent if

F X 1 , , X m , Y 1 , , Y n ( x 1 , , x m , y 1 , , y n ) = F X 1 , , X m ( x 1 , , x m ) F Y 1 , , Y n ( y 1 , , y n ) for all x 1 , , x m , y 1 , , y n . {\displaystyle F_{X_{1},\ldots ,X_{m},Y_{1},\ldots ,Y_{n}}(x_{1},\ldots ,x_{m},y_{1},\ldots ,y_{n})=F_{X_{1},\ldots ,X_{m}}(x_{1},\ldots ,x_{m})\cdot F_{Y_{1},\ldots ,Y_{n}}(y_{1},\ldots ,y_{n})\quad {\text{for all }}x_{1},\ldots ,x_{m},y_{1},\ldots ,y_{n}.}

For stochastic processes

For one stochastic process

The definition of independence may be extended from random vectors to a stochastic process. Therefore, it is required for an independent stochastic process that the random variables obtained by sampling the process at any n {\displaystyle n} times t 1 , , t n {\displaystyle t_{1},\ldots ,t_{n}} are independent random variables for any n {\displaystyle n} .: p. 163

Formally, a stochastic process { X t } t T {\displaystyle \left\{X_{t}\right\}_{t\in {\mathcal {T}}}} is called independent, if and only if for all n N {\displaystyle n\in \mathbb {N} } and for all t 1 , , t n T {\displaystyle t_{1},\ldots ,t_{n}\in {\mathcal {T}}}

F X t 1 , , X t n ( x 1 , , x n ) = F X t 1 ( x 1 ) F X t n ( x n ) for all x 1 , , x n {\displaystyle F_{X_{t_{1}},\ldots ,X_{t_{n}}}(x_{1},\ldots ,x_{n})=F_{X_{t_{1}}}(x_{1})\cdot \ldots \cdot F_{X_{t_{n}}}(x_{n})\quad {\text{for all }}x_{1},\ldots ,x_{n}}

(Eq.7)

where F X t 1 , , X t n ( x 1 , , x n ) = P ( X ( t 1 ) x 1 , , X ( t n ) x n ) {\displaystyle F_{X_{t_{1}},\ldots ,X_{t_{n}}}(x_{1},\ldots ,x_{n})=\mathrm {P} (X(t_{1})\leq x_{1},\ldots ,X(t_{n})\leq x_{n})} . Independence of a stochastic process is a property within a stochastic process, not between two stochastic processes.

For two stochastic processes

Independence of two stochastic processes is a property between two stochastic processes { X t } t T {\displaystyle \left\{X_{t}\right\}_{t\in {\mathcal {T}}}} and { Y t } t T {\displaystyle \left\{Y_{t}\right\}_{t\in {\mathcal {T}}}} that are defined on the same probability space ( Ω , F , P ) {\displaystyle (\Omega ,{\mathcal {F}},P)} . Formally, two stochastic processes { X t } t T {\displaystyle \left\{X_{t}\right\}_{t\in {\mathcal {T}}}} and { Y t } t T {\displaystyle \left\{Y_{t}\right\}_{t\in {\mathcal {T}}}} are said to be independent if for all n N {\displaystyle n\in \mathbb {N} } and for all t 1 , , t n T {\displaystyle t_{1},\ldots ,t_{n}\in {\mathcal {T}}} , the random vectors ( X ( t 1 ) , , X ( t n ) ) {\displaystyle (X(t_{1}),\ldots ,X(t_{n}))} and ( Y ( t 1 ) , , Y ( t n ) ) {\displaystyle (Y(t_{1}),\ldots ,Y(t_{n}))} are independent,: p. 515 i.e. if

F X t 1 , , X t n , Y t 1 , , Y t n ( x 1 , , x n , y 1 , , y n ) = F X t 1 , , X t n ( x 1 , , x n ) F Y t 1 , , Y t n ( y 1 , , y n ) for all x 1 , , x n {\displaystyle F_{X_{t_{1}},\ldots ,X_{t_{n}},Y_{t_{1}},\ldots ,Y_{t_{n}}}(x_{1},\ldots ,x_{n},y_{1},\ldots ,y_{n})=F_{X_{t_{1}},\ldots ,X_{t_{n}}}(x_{1},\ldots ,x_{n})\cdot F_{Y_{t_{1}},\ldots ,Y_{t_{n}}}(y_{1},\ldots ,y_{n})\quad {\text{for all }}x_{1},\ldots ,x_{n}}

(Eq.8)

Independent σ-algebras

The definitions above (Eq.1 and Eq.2) are both generalized by the following definition of independence for σ-algebras. Let ( Ω , Σ , P ) {\displaystyle (\Omega ,\Sigma ,\mathrm {P} )} be a probability space and let A {\displaystyle {\mathcal {A}}} and B {\displaystyle {\mathcal {B}}} be two sub-σ-algebras of Σ {\displaystyle \Sigma } . A {\displaystyle {\mathcal {A}}} and B {\displaystyle {\mathcal {B}}} are said to be independent if, whenever A A {\displaystyle A\in {\mathcal {A}}} and B B {\displaystyle B\in {\mathcal {B}}} ,

P ( A B ) = P ( A ) P ( B ) . {\displaystyle \mathrm {P} (A\cap B)=\mathrm {P} (A)\mathrm {P} (B).}

Likewise, a finite family of σ-algebras ( τ i ) i I {\displaystyle (\tau _{i})_{i\in I}} , where I {\displaystyle I} is an index set, is said to be independent if and only if

( A i ) i I i I τ i : P ( i I A i ) = i I P ( A i ) {\displaystyle \forall \left(A_{i}\right)_{i\in I}\in \prod \nolimits _{i\in I}\tau _{i}\ :\ \mathrm {P} \left(\bigcap \nolimits _{i\in I}A_{i}\right)=\prod \nolimits _{i\in I}\mathrm {P} \left(A_{i}\right)}

and an infinite family of σ-algebras is said to be independent if all its finite subfamilies are independent.

The new definition relates to the previous ones very directly:

  • Two events are independent (in the old sense) if and only if the σ-algebras that they generate are independent (in the new sense). The σ-algebra generated by an event E Σ {\displaystyle E\in \Sigma } is, by definition,
σ ( { E } ) = { , E , Ω E , Ω } . {\displaystyle \sigma (\{E\})=\{\emptyset ,E,\Omega \setminus E,\Omega \}.}
  • Two random variables X {\displaystyle X} and Y {\displaystyle Y} defined over Ω {\displaystyle \Omega } are independent (in the old sense) if and only if the σ-algebras that they generate are independent (in the new sense). The σ-algebra generated by a random variable X {\displaystyle X} taking values in some measurable space S {\displaystyle S} consists, by definition, of all subsets of Ω {\displaystyle \Omega } of the form X 1 ( U ) {\displaystyle X^{-1}(U)} , where U {\displaystyle U} is any measurable subset of S {\displaystyle S} .

Using this definition, it is easy to show that if X {\displaystyle X} and Y {\displaystyle Y} are random variables and Y {\displaystyle Y} is constant, then X {\displaystyle X} and Y {\displaystyle Y} are independent, since the σ-algebra generated by a constant random variable is the trivial σ-algebra { , Ω } {\displaystyle \{\varnothing ,\Omega \}} . Probability zero events cannot affect independence so independence also holds if Y {\displaystyle Y} is only Pr-almost surely constant.

Self-independence

Note that an event is independent of itself if and only if

P ( A ) = P ( A A ) = P ( A ) P ( A ) P ( A ) = 0 or P ( A ) = 1. {\displaystyle \mathrm {P} (A)=\mathrm {P} (A\cap A)=\mathrm {P} (A)\cdot \mathrm {P} (A)\iff \mathrm {P} (A)=0{\text{ or }}\mathrm {P} (A)=1.}

Thus an event is independent of itself if and only if it almost surely occurs or its complement almost surely occurs; this fact is useful when proving zero–one laws.

Expectation and covariance

If X {\displaystyle X} and Y {\displaystyle Y} are independent random variables, then the expectation operator E {\displaystyle \operatorname {E} } has the property

E [ X Y ] = E [ X ] E [ Y ] , {\displaystyle \operatorname {E} [XY]=\operatorname {E} [X]\operatorname {E} [Y],}

and the covariance cov [ X , Y ] {\displaystyle \operatorname {cov} [X,Y]} is zero, as follows from

cov [ X , Y ] = E [ X Y ] E [ X ] E [ Y ] . {\displaystyle \operatorname {cov} [X,Y]=\operatorname {E} [XY]-\operatorname {E} [X]\operatorname {E} [Y].}

The converse does not hold: if two random variables have a covariance of 0 they still may be not independent. See uncorrelated.

Similarly for two stochastic processes { X t } t T {\displaystyle \left\{X_{t}\right\}_{t\in {\mathcal {T}}}} and { Y t } t T {\displaystyle \left\{Y_{t}\right\}_{t\in {\mathcal {T}}}} : If they are independent, then they are uncorrelated.: p. 151

Characteristic function

Two random variables X {\displaystyle X} and Y {\displaystyle Y} are independent if and only if the characteristic function of the random vector ( X , Y ) {\displaystyle (X,Y)} satisfies

φ ( X , Y ) ( t , s ) = φ X ( t ) φ Y ( s ) . {\displaystyle \varphi _{(X,Y)}(t,s)=\varphi _{X}(t)\cdot \varphi _{Y}(s).}

In particular the characteristic function of their sum is the product of their marginal characteristic functions:

φ X + Y ( t ) = φ X ( t ) φ Y ( t ) , {\displaystyle \varphi _{X+Y}(t)=\varphi _{X}(t)\cdot \varphi _{Y}(t),}

though the reverse implication is not true. Random variables that satisfy the latter condition are called subindependent.

Rolling dice

The event of getting a 6 the first time a die is rolled and the event of getting a 6 the second time are independent. By contrast, the event of getting a 6 the first time a die is rolled and the event that the sum of the numbers seen on the first and second trial is 8 are not independent.

Drawing cards

If two cards are drawn with replacement from a deck of cards, the event of drawing a red card on the first trial and that of drawing a red card on the second trial are independent. By contrast, if two cards are drawn without replacement from a deck of cards, the event of drawing a red card on the first trial and that of drawing a red card on the second trial are not independent, because a deck that has had a red card removed has proportionately fewer red cards.

Pairwise and mutual independence

Pairwise independent, but not mutually independent, events.
Mutually independent events.

Consider the two probability spaces shown. In both cases, P ( A ) = P ( B ) = 1 / 2 {\displaystyle \mathrm {P} (A)=\mathrm {P} (B)=1/2} and P ( C ) = 1 / 4 {\displaystyle \mathrm {P} (C)=1/4} . The random variables in the first space are pairwise independent because P ( A | B ) = P ( A | C ) = 1 / 2 = P ( A ) {\displaystyle \mathrm {P} (A|B)=\mathrm {P} (A|C)=1/2=\mathrm {P} (A)} , P ( B | A ) = P ( B | C ) = 1 / 2 = P ( B ) {\displaystyle \mathrm {P} (B|A)=\mathrm {P} (B|C)=1/2=\mathrm {P} (B)} , and P ( C | A ) = P ( C | B ) = 1 / 4 = P ( C ) {\displaystyle \mathrm {P} (C|A)=\mathrm {P} (C|B)=1/4=\mathrm {P} (C)} ; but the three random variables are not mutually independent. The random variables in the second space are both pairwise independent and mutually independent. To illustrate the difference, consider conditioning on two events. In the pairwise independent case, although any one event is independent of each of the other two individually, it is not independent of the intersection of the other two:

P ( A | B C ) = 4 40 4 40 + 1 40 = 4 5 P ( A ) {\displaystyle \mathrm {P} (A|BC)={\frac {\frac {4}{40}}{{\frac {4}{40}}+{\frac {1}{40}}}}={\tfrac {4}{5}}\neq \mathrm {P} (A)}
P ( B | A C ) = 4 40 4 40 + 1 40 = 4 5 P ( B ) {\displaystyle \mathrm {P} (B|AC)={\frac {\frac {4}{40}}{{\frac {4}{40}}+{\frac {1}{40}}}}={\tfrac {4}{5}}\neq \mathrm {P} (B)}
P ( C | A B ) = 4 40 4 40 + 6 40 = 2 5 P ( C ) {\displaystyle \mathrm {P} (C|AB)={\frac {\frac {4}{40}}{{\frac {4}{40}}+{\frac {6}{40}}}}={\tfrac {2}{5}}\neq \mathrm {P} (C)}

In the mutually independent case, however,

P ( A | B C ) = 1 16 1 16 + 1 16 = 1 2 = P ( A ) {\displaystyle \mathrm {P} (A|BC)={\frac {\frac {1}{16}}{{\frac {1}{16}}+{\frac {1}{16}}}}={\tfrac {1}{2}}=\mathrm {P} (A)}
P ( B | A C ) = 1 16 1 16 + 1 16 = 1 2 = P ( B ) {\displaystyle \mathrm {P} (B|AC)={\frac {\frac {1}{16}}{{\frac {1}{16}}+{\frac {1}{16}}}}={\tfrac {1}{2}}=\mathrm {P} (B)}
P ( C | A B ) = 1 16 1 16 + 3 16 = 1 4 = P ( C ) {\displaystyle \mathrm {P} (C|AB)={\frac {\frac {1}{16}}{{\frac {1}{16}}+{\frac {3}{16}}}}={\tfrac {1}{4}}=\mathrm {P} (C)}

Mutual independence

It is possible to create a three-event example in which

P ( A B C ) = P ( A ) P ( B ) P ( C ) , {\displaystyle \mathrm {P} (A\cap B\cap C)=\mathrm {P} (A)\mathrm {P} (B)\mathrm {P} (C),}

and yet no two of the three events are pairwise independent (and hence the set of events are not mutually independent). This example shows that mutual independence involves requirements on the products of probabilities of all combinations of events, not just the single events as in this example.

For events

The events A {\displaystyle A} and B {\displaystyle B} are conditionally independent given an event C {\displaystyle C} when

P ( A B C ) = P ( A C ) P ( B C ) {\displaystyle \mathrm {P} (A\cap B\mid C)=\mathrm {P} (A\mid C)\cdot \mathrm {P} (B\mid C)} .

For random variables

Intuitively, two random variables X {\displaystyle X} and Y {\displaystyle Y} are conditionally independent given Z {\displaystyle Z} if, once Z {\displaystyle Z} is known, the value of Y {\displaystyle Y} does not add any additional information about X {\displaystyle X} . For instance, two measurements X {\displaystyle X} and Y {\displaystyle Y} of the same underlying quantity Z {\displaystyle Z} are not independent, but they are conditionally independent given Z {\displaystyle Z} (unless the errors in the two measurements are somehow connected).

The formal definition of conditional independence is based on the idea of conditional distributions. If X {\displaystyle X} , Y {\displaystyle Y} , and Z {\displaystyle Z} are discrete random variables, then we define X {\displaystyle X} and Y {\displaystyle Y} to be conditionally independent given Z {\displaystyle Z} if

P ( X x , Y y | Z = z ) = P ( X x | Z = z ) P ( Y y | Z = z ) {\displaystyle \mathrm {P} (X\leq x,Y\leq y\;|\;Z=z)=\mathrm {P} (X\leq x\;|\;Z=z)\cdot \mathrm {P} (Y\leq y\;|\;Z=z)}

for all x {\displaystyle x} , y {\displaystyle y} and z {\displaystyle z} such that P ( Z = z ) > 0 {\displaystyle \mathrm {P} (Z=z)>0} . On the other hand, if the random variables are continuous and have a joint probability density function f X Y Z ( x , y , z ) {\displaystyle f_{XYZ}(x,y,z)} , then X {\displaystyle X} and Y {\displaystyle Y} are conditionally independent given Z {\displaystyle Z} if

f X Y | Z ( x , y | z ) = f X | Z ( x | z ) f Y | Z ( y | z ) {\displaystyle f_{XY|Z}(x,y|z)=f_{X|Z}(x|z)\cdot f_{Y|Z}(y|z)}

for all real numbers x {\displaystyle x} , y {\displaystyle y} and z {\displaystyle z} such that f Z ( z ) > 0 {\displaystyle f_{Z}(z)>0} .

If discrete X {\displaystyle X} and Y {\displaystyle Y} are conditionally independent given Z {\displaystyle Z} , then

P ( X = x | Y = y , Z = z ) = P ( X = x | Z = z ) {\displaystyle \mathrm {P} (X=x|Y=y,Z=z)=\mathrm {P} (X=x|Z=z)}

for any x {\displaystyle x} , y {\displaystyle y} and z {\displaystyle z} with P ( Z = z ) > 0 {\displaystyle \mathrm {P} (Z=z)>0} . That is, the conditional distribution for X {\displaystyle X} given Y {\displaystyle Y} and Z {\displaystyle Z} is the same as that given Z {\displaystyle Z} alone. A similar equation holds for the conditional probability density functions in the continuous case.

Independence can be seen as a special kind of conditional independence, since probability can be seen as a kind of conditional probability given no events.

  1. Russell, Stuart; Norvig, Peter (2002).Artificial Intelligence: A Modern Approach. Prentice Hall. p. 478. ISBN 0-13-790395-2.
  2. Florescu, Ionut (2014). Probability and Stochastic Processes. Wiley. ISBN 978-0-470-62455-5.
  3. Gallager, Robert G. (2013). Stochastic Processes Theory for Applications. Cambridge University Press. ISBN 978-1-107-03975-9.
  4. Feller, W (1971). "Stochastic Independence". An Introduction to Probability Theory and Its Applications. Wiley.
  5. Papoulis, Athanasios (1991). Probability, Random Variables and Stochastic Processes. MCGraw Hill. ISBN 0-07-048477-5.
  6. Hwei, Piao (1997).Theory and Problems of Probability, Random Variables, and Random Processes. McGraw-Hill. ISBN 0-07-030644-3.
  7. Amos Lapidoth (8 February 2017). A Foundation in Digital Communication. Cambridge University Press. ISBN 978-1-107-17732-1.
  8. Durrett, Richard (1996). Probability: theory and examples (Second ed.). page 62
  9. Park,Kun Il (2018). Fundamentals of Probability and Stochastic Processes with Applications to Communications. Springer. ISBN 978-3-319-68074-3.
  10. George, Glyn, "Testing for the independence of three events," Mathematical Gazette 88, November 2004, 568. PDF

Independence (probability theory)
Independence probability theory Language Watch Edit 160 160 Redirected from Stochastic dependence Independence is a fundamental notion in probability theory as in statistics and the theory of stochastic processes Two events are independent statistically independent or stochastically independent 1 if the occurrence of one does not affect the probability of occurrence of the other equivalently does not affect the odds Similarly two random variables are independent if the realization of one does not affect the probability distribution of the other When dealing with collections of more than two events a weak and a strong notion of independence need to be distinguished The events are called pairwise independent if any two events in the collection are independent of each other while saying that the events are mutually independent or collectively independent intuitively means that each event is independent of any combination of other events in the collection A similar notion exists for collections of random variables The name mutual independence same as collective independence seems the outcome of a educational choice merely to distinguish the stronger notion from pairwise independence which is a weaker notion In the advanced literature of probability theory statistics and stochastic processes the stronger notion is simply named independence with no modifier It is stronger since independence implies pairwise independence but not the other way around Contents 1 Definition 1 1 For events 1 1 1 Two events 1 1 2 Log probability and information content 1 1 3 Odds 1 1 4 More than two events 1 2 For real valued random variables 1 2 1 Two random variables 1 2 2 More than two random variables 1 3 For real valued random vectors 1 4 For stochastic processes 1 4 1 For one stochastic process 1 4 2 For two stochastic processes 1 5 Independent s algebras 2 Properties 2 1 Self independence 2 2 Expectation and covariance 2 3 Characteristic function 3 Examples 3 1 Rolling dice 3 2 Drawing cards 3 3 Pairwise and mutual independence 3 4 Mutual independence 4 Conditional independence 4 1 For events 4 2 For random variables 5 See also 6 References 7 External linksDefinition EditFor events Edit Two events Edit Two events A displaystyle A and B displaystyle B are independent often written as A B displaystyle A perp B or A B displaystyle A perp perp B if and only if their joint probability equals the product of their probabilities 2 p 29 3 p 10 P A B P A P B displaystyle mathrm P A cap B mathrm P A mathrm P B Eq 1 Why this defines independence is made clear by rewriting with conditional probabilities P A B P A P B P A P A B P B P A B displaystyle mathrm P A cap B mathrm P A mathrm P B iff mathrm P A frac mathrm P A cap B mathrm P B mathrm P A mid B and similarly P A B P A P B P B P B A displaystyle mathrm P A cap B mathrm P A mathrm P B iff mathrm P B mathrm P B mid A Thus the occurrence of B displaystyle B does not affect the probability of A displaystyle A and vice versa Although the derived expressions may seem more intuitive they are not the preferred definition as the conditional probabilities may be undefined if P A displaystyle mathrm P A or P B displaystyle mathrm P B are 0 Furthermore the preferred definition makes clear by symmetry that when A displaystyle A is independent of B displaystyle B B displaystyle B is also independent of A displaystyle A Log probability and information content Edit Stated in terms of log probability two events are independent if and only if the log probability of the joint event is the sum of the log probability of the individual events log P A B log P A log P B displaystyle log mathrm P A cap B log mathrm P A log mathrm P B In information theory negative log probability is interpreted as information content and thus two events are independent if and only if the information content of the combined event equals the sum of information content of the individual events I A B I A I B displaystyle mathrm I A cap B mathrm I A mathrm I B See Information content Additivity of independent events for details Odds Edit Stated in terms of odds two events are independent if and only if the odds ratio of A displaystyle A and B displaystyle B is unity 1 Analogously with probability this is equivalent to the conditional odds being equal to the unconditional odds O A B O A and O B A O B displaystyle O A mid B O A text and O B mid A O B or to the odds of one event given the other event being the same as the odds of the event given the other event not occurring O A B O A B and O B A O B A displaystyle O A mid B O A mid neg B text and O B mid A O B mid neg A The odds ratio can be defined as O A B O A B displaystyle O A mid B O A mid neg B or symmetrically for odds of B displaystyle B given A displaystyle A and thus is 1 if and only if the events are independent More than two events Edit A finite set of events A i i 1 n displaystyle A i i 1 n is pairwise independent if every pair of events is independent 4 that is if and only if for all distinct pairs of indices m k displaystyle m k P A m A k P A m P A k displaystyle mathrm P A m cap A k mathrm P A m mathrm P A k Eq 2 A finite set of events is mutually independent if every event is independent of any intersection of the other events 4 3 p 11 that is if and only if for every k n displaystyle k leq n and for every k displaystyle k element subset of events B i i 1 k displaystyle B i i 1 k of A i i 1 n displaystyle A i i 1 n P i 1 k B i i 1 k P B i displaystyle mathrm P left bigcap i 1 k B i right prod i 1 k mathrm P B i Eq 3 This is called the multiplication rule for independent events Note that it is not a single condition involving only the product of all the probabilities of all single events it must hold true for all subsets of events For more than two events a mutually independent set of events is by definition pairwise independent but the converse is not necessarily true 2 p 30 For real valued random variables Edit Two random variables Edit Two random variables X displaystyle X and Y displaystyle Y are independent if and only if iff the elements of the p system generated by them are independent that is to say for every x displaystyle x and y displaystyle y the events X x displaystyle X leq x and Y y displaystyle Y leq y are independent events as defined above in Eq 1 That is X displaystyle X and Y displaystyle Y with cumulative distribution functions F X x displaystyle F X x and F Y y displaystyle F Y y are independent iff the combined random variable X Y displaystyle X Y has a joint cumulative distribution function 3 p 15 F X Y x y F X x F Y y for all x y displaystyle F X Y x y F X x F Y y quad text for all x y Eq 4 or equivalently if the probability densities f X x displaystyle f X x and f Y y displaystyle f Y y and the joint probability density f X Y x y displaystyle f X Y x y exist f X Y x y f X x f Y y for all x y displaystyle f X Y x y f X x f Y y quad text for all x y More than two random variables Edit A finite set of n displaystyle n random variables X 1 X n displaystyle X 1 ldots X n is pairwise independent if and only if every pair of random variables is independent Even if the set of random variables is pairwise independent it is not necessarily mutually independent as defined next A finite set of n displaystyle n random variables X 1 X n displaystyle X 1 ldots X n is mutually independent if and only if for any sequence of numbers x 1 x n displaystyle x 1 ldots x n the events X 1 x 1 X n x n displaystyle X 1 leq x 1 ldots X n leq x n are mutually independent events as defined above in Eq 3 This is equivalent to the following condition on the joint cumulative distribution function F X 1 X n x 1 x n displaystyle F X 1 ldots X n x 1 ldots x n A finite set of n displaystyle n random variables X 1 X n displaystyle X 1 ldots X n is mutually independent if and only if 3 p 16 F X 1 X n x 1 x n F X 1 x 1 F X n x n for all x 1 x n displaystyle F X 1 ldots X n x 1 ldots x n F X 1 x 1 cdot ldots cdot F X n x n quad text for all x 1 ldots x n Eq 5 Notice that it is not necessary here to require that the probability distribution factorizes for all possible k displaystyle k element subsets as in the case for n displaystyle n events This is not required because e g F X 1 X 2 X 3 x 1 x 2 x 3 F X 1 x 1 F X 2 x 2 F X 3 x 3 displaystyle F X 1 X 2 X 3 x 1 x 2 x 3 F X 1 x 1 cdot F X 2 x 2 cdot F X 3 x 3 implies F X 1 X 3 x 1 x 3 F X 1 x 1 F X 3 x 3 displaystyle F X 1 X 3 x 1 x 3 F X 1 x 1 cdot F X 3 x 3 The measure theoretically inclined may prefer to substitute events X A displaystyle X in A for events X x displaystyle X leq x in the above definition where A displaystyle A is any Borel set That definition is exactly equivalent to the one above when the values of the random variables are real numbers It has the advantage of working also for complex valued random variables or for random variables taking values in any measurable space which includes topological spaces endowed by appropriate s algebras For real valued random vectors Edit Two random vectors X X 1 X m T displaystyle mathbf X X 1 ldots X m mathrm T and Y Y 1 Y n T displaystyle mathbf Y Y 1 ldots Y n mathrm T are called independent if 5 p 187 F X Y x y F X x F Y y for all x y displaystyle F mathbf X Y mathbf x y F mathbf X mathbf x cdot F mathbf Y mathbf y quad text for all mathbf x mathbf y Eq 6 where F X x displaystyle F mathbf X mathbf x and F Y y displaystyle F mathbf Y mathbf y denote the cumulative distribution functions of X displaystyle mathbf X and Y displaystyle mathbf Y and F X Y x y displaystyle F mathbf X Y mathbf x y denotes their joint cumulative distribution function Independence of X displaystyle mathbf X and Y displaystyle mathbf Y is often denoted by X Y displaystyle mathbf X perp perp mathbf Y Written component wise X displaystyle mathbf X and Y displaystyle mathbf Y are called independent if F X 1 X m Y 1 Y n x 1 x m y 1 y n F X 1 X m x 1 x m F Y 1 Y n y 1 y n for all x 1 x m y 1 y n displaystyle F X 1 ldots X m Y 1 ldots Y n x 1 ldots x m y 1 ldots y n F X 1 ldots X m x 1 ldots x m cdot F Y 1 ldots Y n y 1 ldots y n quad text for all x 1 ldots x m y 1 ldots y n For stochastic processes Edit For one stochastic process Edit The definition of independence may be extended from random vectors to a stochastic process Therefore it is required for an independent stochastic process that the random variables obtained by sampling the process at any n displaystyle n times t 1 t n displaystyle t 1 ldots t n are independent random variables for any n displaystyle n 6 p 163 Formally a stochastic process X t t T displaystyle left X t right t in mathcal T is called independent if and only if for all n N displaystyle n in mathbb N and for all t 1 t n T displaystyle t 1 ldots t n in mathcal T F X t 1 X t n x 1 x n F X t 1 x 1 F X t n x n for all x 1 x n displaystyle F X t 1 ldots X t n x 1 ldots x n F X t 1 x 1 cdot ldots cdot F X t n x n quad text for all x 1 ldots x n Eq 7 where F X t 1 X t n x 1 x n P X t 1 x 1 X t n x n displaystyle F X t 1 ldots X t n x 1 ldots x n mathrm P X t 1 leq x 1 ldots X t n leq x n Independence of a stochastic process is a property within a stochastic process not between two stochastic processes For two stochastic processes Edit Independence of two stochastic processes is a property between two stochastic processes X t t T displaystyle left X t right t in mathcal T and Y t t T displaystyle left Y t right t in mathcal T that are defined on the same probability space W F P displaystyle Omega mathcal F P Formally two stochastic processes X t t T displaystyle left X t right t in mathcal T and Y t t T displaystyle left Y t right t in mathcal T are said to be independent if for all n N displaystyle n in mathbb N and for all t 1 t n T displaystyle t 1 ldots t n in mathcal T the random vectors X t 1 X t n displaystyle X t 1 ldots X t n and Y t 1 Y t n displaystyle Y t 1 ldots Y t n are independent 7 p 515 i e if F X t 1 X t n Y t 1 Y t n x 1 x n y 1 y n F X t 1 X t n x 1 x n F Y t 1 Y t n y 1 y n for all x 1 x n displaystyle F X t 1 ldots X t n Y t 1 ldots Y t n x 1 ldots x n y 1 ldots y n F X t 1 ldots X t n x 1 ldots x n cdot F Y t 1 ldots Y t n y 1 ldots y n quad text for all x 1 ldots x n Eq 8 Independent s algebras Edit The definitions above Eq 1 and Eq 2 are both generalized by the following definition of independence for s algebras Let W S P displaystyle Omega Sigma mathrm P be a probability space and let A displaystyle mathcal A and B displaystyle mathcal B be two sub s algebras of S displaystyle Sigma A displaystyle mathcal A and B displaystyle mathcal B are said to be independent if whenever A A displaystyle A in mathcal A and B B displaystyle B in mathcal B P A B P A P B displaystyle mathrm P A cap B mathrm P A mathrm P B Likewise a finite family of s algebras t i i I displaystyle tau i i in I where I displaystyle I is an index set is said to be independent if and only if A i i I i I t i P i I A i i I P A i displaystyle forall left A i right i in I in prod nolimits i in I tau i mathrm P left bigcap nolimits i in I A i right prod nolimits i in I mathrm P left A i right and an infinite family of s algebras is said to be independent if all its finite subfamilies are independent The new definition relates to the previous ones very directly Two events are independent in the old sense if and only if the s algebras that they generate are independent in the new sense The s algebra generated by an event E S displaystyle E in Sigma is by definition s E E W E W displaystyle sigma E emptyset E Omega setminus E Omega dd Two random variables X displaystyle X and Y displaystyle Y defined over W displaystyle Omega are independent in the old sense if and only if the s algebras that they generate are independent in the new sense The s algebra generated by a random variable X displaystyle X taking values in some measurable space S displaystyle S consists by definition of all subsets of W displaystyle Omega of the form X 1 U displaystyle X 1 U where U displaystyle U is any measurable subset of S displaystyle S Using this definition it is easy to show that if X displaystyle X and Y displaystyle Y are random variables and Y displaystyle Y is constant then X displaystyle X and Y displaystyle Y are independent since the s algebra generated by a constant random variable is the trivial s algebra W displaystyle varnothing Omega Probability zero events cannot affect independence so independence also holds if Y displaystyle Y is only Pr almost surely constant Properties EditSelf independence Edit Note that an event is independent of itself if and only if P A P A A P A P A P A 0 or P A 1 displaystyle mathrm P A mathrm P A cap A mathrm P A cdot mathrm P A iff mathrm P A 0 text or mathrm P A 1 Thus an event is independent of itself if and only if it almost surely occurs or its complement almost surely occurs this fact is useful when proving zero one laws 8 Expectation and covariance Edit Main article Correlation and dependence If X displaystyle X and Y displaystyle Y are independent random variables then the expectation operator E displaystyle operatorname E has the property E X Y E X E Y displaystyle operatorname E XY operatorname E X operatorname E Y and the covariance cov X Y displaystyle operatorname cov X Y is zero as follows from cov X Y E X Y E X E Y displaystyle operatorname cov X Y operatorname E XY operatorname E X operatorname E Y The converse does not hold if two random variables have a covariance of 0 they still may be not independent See uncorrelated Similarly for two stochastic processes X t t T displaystyle left X t right t in mathcal T and Y t t T displaystyle left Y t right t in mathcal T If they are independent then they are uncorrelated 9 p 151 Characteristic function Edit Two random variables X displaystyle X and Y displaystyle Y are independent if and only if the characteristic function of the random vector X Y displaystyle X Y satisfies f X Y t s f X t f Y s displaystyle varphi X Y t s varphi X t cdot varphi Y s In particular the characteristic function of their sum is the product of their marginal characteristic functions f X Y t f X t f Y t displaystyle varphi X Y t varphi X t cdot varphi Y t though the reverse implication is not true Random variables that satisfy the latter condition are called subindependent Examples EditRolling dice Edit The event of getting a 6 the first time a die is rolled and the event of getting a 6 the second time are independent By contrast the event of getting a 6 the first time a die is rolled and the event that the sum of the numbers seen on the first and second trial is 8 are not independent Drawing cards Edit If two cards are drawn with replacement from a deck of cards the event of drawing a red card on the first trial and that of drawing a red card on the second trial are independent By contrast if two cards are drawn without replacement from a deck of cards the event of drawing a red card on the first trial and that of drawing a red card on the second trial are not independent because a deck that has had a red card removed has proportionately fewer red cards Pairwise and mutual independence Edit Pairwise independent but not mutually independent events Mutually independent events Consider the two probability spaces shown In both cases P A P B 1 2 displaystyle mathrm P A mathrm P B 1 2 and P C 1 4 displaystyle mathrm P C 1 4 The random variables in the first space are pairwise independent because P A B P A C 1 2 P A displaystyle mathrm P A B mathrm P A C 1 2 mathrm P A P B A P B C 1 2 P B displaystyle mathrm P B A mathrm P B C 1 2 mathrm P B and P C A P C B 1 4 P C displaystyle mathrm P C A mathrm P C B 1 4 mathrm P C but the three random variables are not mutually independent The random variables in the second space are both pairwise independent and mutually independent To illustrate the difference consider conditioning on two events In the pairwise independent case although any one event is independent of each of the other two individually it is not independent of the intersection of the other two P A B C 4 40 4 40 1 40 4 5 P A displaystyle mathrm P A BC frac frac 4 40 frac 4 40 frac 1 40 tfrac 4 5 neq mathrm P A P B A C 4 40 4 40 1 40 4 5 P B displaystyle mathrm P B AC frac frac 4 40 frac 4 40 frac 1 40 tfrac 4 5 neq mathrm P B P C A B 4 40 4 40 6 40 2 5 P C displaystyle mathrm P C AB frac frac 4 40 frac 4 40 frac 6 40 tfrac 2 5 neq mathrm P C In the mutually independent case however P A B C 1 16 1 16 1 16 1 2 P A displaystyle mathrm P A BC frac frac 1 16 frac 1 16 frac 1 16 tfrac 1 2 mathrm P A P B A C 1 16 1 16 1 16 1 2 P B displaystyle mathrm P B AC frac frac 1 16 frac 1 16 frac 1 16 tfrac 1 2 mathrm P B P C A B 1 16 1 16 3 16 1 4 P C displaystyle mathrm P C AB frac frac 1 16 frac 1 16 frac 3 16 tfrac 1 4 mathrm P C Mutual independence Edit It is possible to create a three event example in which P A B C P A P B P C displaystyle mathrm P A cap B cap C mathrm P A mathrm P B mathrm P C and yet no two of the three events are pairwise independent and hence the set of events are not mutually independent 10 This example shows that mutual independence involves requirements on the products of probabilities of all combinations of events not just the single events as in this example Conditional independence EditMain article Conditional independence For events Edit The events A displaystyle A and B displaystyle B are conditionally independent given an event C displaystyle C when P A B C P A C P B C displaystyle mathrm P A cap B mid C mathrm P A mid C cdot mathrm P B mid C For random variables Edit Intuitively two random variables X displaystyle X and Y displaystyle Y are conditionally independent given Z displaystyle Z if once Z displaystyle Z is known the value of Y displaystyle Y does not add any additional information about X displaystyle X For instance two measurements X displaystyle X and Y displaystyle Y of the same underlying quantity Z displaystyle Z are not independent but they are conditionally independent given Z displaystyle Z unless the errors in the two measurements are somehow connected The formal definition of conditional independence is based on the idea of conditional distributions If X displaystyle X Y displaystyle Y and Z displaystyle Z are discrete random variables then we define X displaystyle X and Y displaystyle Y to be conditionally independent given Z displaystyle Z if P X x Y y Z z P X x Z z P Y y Z z displaystyle mathrm P X leq x Y leq y Z z mathrm P X leq x Z z cdot mathrm P Y leq y Z z for all x displaystyle x y displaystyle y and z displaystyle z such that P Z z gt 0 displaystyle mathrm P Z z gt 0 On the other hand if the random variables are continuous and have a joint probability density function f X Y Z x y z displaystyle f XYZ x y z then X displaystyle X and Y displaystyle Y are conditionally independent given Z displaystyle Z if f X Y Z x y z f X Z x z f Y Z y z displaystyle f XY Z x y z f X Z x z cdot f Y Z y z for all real numbers x displaystyle x y displaystyle y and z displaystyle z such that f Z z gt 0 displaystyle f Z z gt 0 If discrete X displaystyle X and Y displaystyle Y are conditionally independent given Z displaystyle Z then P X x Y y Z z P X x Z z displaystyle mathrm P X x Y y Z z mathrm P X x Z z for any x displaystyle x y displaystyle y and z displaystyle z with P Z z gt 0 displaystyle mathrm P Z z gt 0 That is the conditional distribution for X displaystyle X given Y displaystyle Y and Z displaystyle Z is the same as that given Z displaystyle Z alone A similar equation holds for the conditional probability density functions in the continuous case Independence can be seen as a special kind of conditional independence since probability can be seen as a kind of conditional probability given no events See also EditCopula statistics Independent and identically distributed random variables Mutually exclusive events Pairwise independent events Subindependence Conditional independence Normally distributed and uncorrelated does not imply independent Mean dependenceReferences Edit Russell Stuart Norvig Peter 2002 Artificial Intelligence A Modern Approach Prentice Hall p 478 ISBN 0 13 790395 2 a b Florescu Ionut 2014 Probability and Stochastic Processes Wiley ISBN 978 0 470 62455 5 a b c d Gallager Robert G 2013 Stochastic Processes Theory for Applications Cambridge University Press ISBN 978 1 107 03975 9 a b Feller W 1971 Stochastic Independence An Introduction to Probability Theory and Its Applications Wiley Papoulis Athanasios 1991 Probability Random Variables and Stochastic Processes MCGraw Hill ISBN 0 07 048477 5 Hwei Piao 1997 Theory and Problems of Probability Random Variables and Random Processes McGraw Hill ISBN 0 07 030644 3 Amos Lapidoth 8 February 2017 A Foundation in Digital Communication Cambridge University Press ISBN 978 1 107 17732 1 Durrett Richard 1996 Probability theory and examples Second ed page 62 Park Kun Il 2018 Fundamentals of Probability and Stochastic Processes with Applications to Communications Springer ISBN 978 3 319 68074 3 George Glyn Testing for the independence of three events Mathematical Gazette 88 November 2004 568 PDFExternal links Edit Media related to Statistical dependence at Wikimedia Commons Retrieved from https en wikipedia org w index php title Independence probability theory amp oldid 1043161137, wikipedia, wiki, book,

books

, library,

article

, read, download, free, free download, mp3, video, mp4, 3gp, jpg, jpeg, gif, png, picture, music, song, movie, book, game, games.