A procedure consisting of all possible simple events, in which it consist of all possible outcomes.

5.3 The Sample Space

The sample space is a set of all possible outcomes in given experiment.

Definition 5.3.1

Experiment

An experiment is the method by which an observation or a measurement is obtained.

Some examples of experiments are

making a measurement of the day's highest temperature,

rolling a die and observing the number that appears on the top,

recording the effectiveness of a certain drug administered to a patient in a hospital,

recording the weight of a newborn baby,

recording a grade on a certain examination, and

ascertaining the preference of a voter before an election.

Thus, by an experiment we mean the following of a specific procedure at the completion of which we observe certain results.

Definition 5.3.2

Sample Point

A possible outcome of an experiment is called a sample point; the actual observations of an experiment constitute a sample.

Example 5.3.1

Tossing a Fair Coin × 5

Conducting the experiment of tossing a coin five times and recording the outcome each time will constitute a sample. The outcome of each individual toss is a sample point.

HHHHH,HHHHT,etc.

Definition 5.3.3

Population

A group of individual persons, objects, or items from which samples are taken for statistical measurement constitutes a population.

Definition 5.3.4

Event

An event is a set of sample points.

We shall denote events by capital letters and sample points by lowercase.

Example 5.3.2

Tossing a Fair Coin × 2

Consider the experiment of flipping a coin twice. Some of the possible events are as follows:

Event A: You observe two heads.

Event B: You observe two tails.

Event C: You observe one head and one tail.

Definition 5.3.5

Simple Event

An event that consists of only one sample point is called a simple or elementary event.

Example 5.3.3

Two Sample Points

Events A and B in Example 5.3.2 are simple or elementary events. Event C is not, because it can be thought of as consisting of two sample points; namely, HT and TH.

Definition 5.3.6

Compound Event

An event that consists of more than one sample point is called a compound event.

Example 5.3.4

Rolling a Fair Die

Consider the experiment of rolling a fair die and the event an odd number is observed.

There are six possible outcomes; namely,

Event A: a 1 is observed,

Event B: a 2 is observed,

Event C: a 3 is observed,

Event D: a 4 is observed,

Event E: a 5 is observed, and

Event F: a 6 is observed.

Each of the preceding events is an elementary event. With events A, C, or E we observe an odd number; that is, we observe a l, 3, or 5. Thus, this event of an odd number is a compound event.

In view of the preceding definitions, we can say that an event that cannot be decomposed is a simple or elementary event. Similarly an event that can be decomposed into two or more simple events is a compound event. It is clear that in Example 5.3.4 events A, B, C, D, E, and F cannot be decomposed.

However, the event of observing an odd number is decomposable.

Definition 5.3.7

Sample Space

A set of all possible outcomes of an experiment is called a sample space. We shall denote the sample space by S.

Hence, that the sample space is simply the set of all possible sample points of a given experiment.

Example 5.3.5

Tossing a Fair Coin

Consider the experiment of tossing a fair coin. The sample space S consists of the two possible outcomes

H=aheadwasobservedandT=atailwasobserved.

Note that outcomes H and T are sample points. Using set theory we symbolize the sample space by S=H,T.

Example 5.3.6

Tossing a Fair Coin × 2

Consider the experiment of tossing a coin twice. The possible outcomes are as follows:

Event A=H,H= headinthefirsttoss, headinthesecondtoss,

Event B=H,T=headinthefirsttoss,tailinthesecondtoss,

Event C=T,H=tailinthefirsttoss,headinthesecondtoss, and

Event D=T,T=tail inthefirsttoss,tailinthesecondtoss.

Thus, the sample space S consists of the simple events (sample points) A, B, C, and D. Again, using set notation we can write the sample space as follows:

S=H,H,H,T, T,H,T, T

or

S=A,B,C,D.

The sample space may be viewed symbolically in terms of the Venn diagram. The sample space of Example 5.3.6 is shown by Figure 5.1.

Having constructed the sample space, we can formulate additional events by grouping the necessary sample points. That is, in Example 5.3.6 the event of obtaining either two heads or two tails will consist of the simple events A and D. This event is represented on the Venn diagram by circling the sample points A and D, as shown by the enclosed` portion of S in Figure 5.1.

Definition 5.3.8

Mutually Exclusive

Two events, A and B, of the sample space S are said to be mutually exclusive if A∩B=∅.

That is, events A and B do not have any elements in common.

Example 5.3.7

Mutually Exclusive Events

In Example 5.3.6, the events A and D are mutually exclusive; the definition of mutually exclusive events can similarly be extended to a larger number of events. For example, the events A, B, and C are mutually exclusive if A∩B=∅, A∩C=∅, B∩C=∅, and

A∩ B∩C=∅.

Outline of the Neyman-Pearson justification

According to the Neyman-Pearson theory, we devise hypotheses test so as to minimize two errors. The main error we want to minimize, called error of type I, is to reject H0, when in fact H0 is true. If we have achieved a reasonable minimization of this type of error, we also try to minimize the error, called error of type II, of accepting H0, when it is false, i.e., when H1 is true. I do not consider, however, the minimization of type II error as a justification for the tests, but as we saw at the end of Section 4 in the previous chapter, only as a convenient feature that may be available.

The Neyman-Pearson tests are usually presented in the following way: The working hypothesis H0 and the alternative hypothesis H1 serve to partition the parameter space Ω. Under H0, θ lies in a subspace Ω′; under H1, θ lies in the complementary subspace Ω - Ω′. H0:θ∈Ω′andH1:θ∈Ω−Ω ′.

The sample spaceS of possible values of X1,…,Xn, is partitioned by the test procedure into two complementary subspaces S0 and S1: the critical region and the noncritical region, respectively. The test is expressed by a sentence φ = [X ∈ S0], of the language with X, such that if x ∈ S0 occurs in reality, then φ is true, and if x ∈ S1 obtains, then φ is false. The test proceeds on the policy:

reject H0, if x ∈ S0; accept H0, if x ∈ S1.

The usual justification of hypotheses tests, by Neyman and his school can be summarized as follows. The choice of test {S0,S1} is mainly guided by the desire to safeguard the incorrect rejection of H0, by assigning a significance level α, to be an upper limit to the probability of incorrectly rejecting H0. That is, prθ [X ∈ S0] ≤ α, for all θ ∈ Ω′. Thus, the probability assigned by Prθ, for θ ∈ Ω′, to the sentence [X ∈ S0] is low. A new measure of the test is introduced, its power. The power of the test for θ is defined as 1 - β(θ), where

β(θ)=Prθ([X∈S1]), for θ∈Ω− Ω′.

In the traditional Neyman-Pearson theory, among all the tests with significance level α, the one with greatest power is chosen. In fact, since there always many tests of level α, in order to choose among them, one of the criteria is the power of the test. One of the problems is that there may not be one test which has the greatest power for all θ ∈ Ω - Ω′. So one of the important problems in math:ematical statistics for the Neyman-Pearson school, is to find conditions under which these uniformly most powerful tests exist.

Let us continue with an example. Suppose that we want to test the hypothesis H0 that a certain random variable X is nearly normally distributed with mean μ0 and variance σ02 . We suppose that the evidence consists of n independent observed values of X, x→=〈x1,x2,…, xn〉. The vector x→ is considered as a point in the samples space of possible values,S. The testing problem, as conceived by Neyman and Pearson, is that of choosing a subset S0 of S, such that, if the observed value x→∈S0, then we should reject H0. The set S0 is the critical region. As we mentioned above, Neyman and Pearson require that S0 should be chosen so as to minimize two types of errors (see, for instance, the first joint paper of Neyman and Pearson, [73, p. 3]). Type I error is the error of rejecting H0 when it is in fact true. Type II error is the error of accepting H0 when it is in fact false.

The treatment of type I error, which is the more important of the two, is the following. The hypothesis H0 induces a probability distribution over S, PrH0. We control type I error by finding a low value for PrH0 S0. If

PrH0S0=α,

we say that we are using a test of size α. The trouble, here, is that we can choose a large number of sets S0 such that PrH0 S0 = α. Of these, many will be intuitively unsatisfactory as critical regions. In order to eliminate these unsatisfactory regions, and obtain a good critical region, we must, according to Neyman and Pearson, consider type II error. Here lies the main difference with my theory. Instead of considering type II error, I propose the use of rejection sets.

It is in order to treat type II error, that Neyman and Pearson introduce a set of alternative hypotheses such that, if H0 is false, one of these alternatives hold. We, thus, begin with a set Ω of hypotheses. Each hypothesis specifies completely the probability distribution of a certain random variable, X. We assume, for simplicity, that each hypothesis is specified by values of a set of parameters. In the example, we are considering, these are the mean, μ and variance, σ2. Thus, in this case, we consider nearly normal distributions with mean μ and known variance σ2, where σ > 0. As before, we assume a sample space,S, and take the problem to be that of choosing an S0 ⊆ S for testing our hypothesis, say that the mean is μ0. It is required that S0 should have some fixed size α, i.e., PrH0 S0 = α. As we mentioned above, we reduce the type II error by making β(μ) as small as possible, for μ ≠ μ0, where 1 - β is the power or the test. Thus, we must make the power as big as possible.

Suppose that there are only two simple alternatives, μ0 and μ1. We need, then, a set S0 such that

Prμ0 S0=α,

and

Prμ1S0≥Prμ1C,

for any other set C, such that Prμ0C=α. In the case of two simple alternatives with nearly normal distributions, Neyman and Pearson have shown that it is always possible to find such a set.

The problem arises when there are many alternatives, μ to μ0. We need, then, a set S0 such that Prμ0S0=α, and PrμS0≥PrμC,for any other set C such that Prμ0 C = α, and for every μ ≠ μ0. This is what is called a uniformly most powerful test (or UMP). This does not always exists, and, then, the requirement is usually weakened to the existence of a UMP test among unbiased test, or UMPU. We shall not discuss the matter further, but present two examples due to Gillies, [37, pp. 204, 205], which I think show that the procedure is not sound.

Some definitions

A trial is a single observation on the random system, for example, one throw of a die, one measurement of a resistance in the example in Section 21.2.

The sample space is the set of all possible outcomes, for example, for the die it is the set {1, 2, 3, 4, 5, 6}, and for the resistance problem it is the set of all possible measured resistances. This set may be discrete or continuous. An event is a set of outcomes. For instance, A is the event of throwing less than 4 and B is the event of throwing a number greater than or equal to 5.

An event is a subset of the sample space S.

Notice that in the case of a continuous sample space an event is also a continuous set, represented by an interval of values. For example, C is the event that the resistance lies in the interval 2000 ± 1.5.

4.3.1 Discrete distributions: Probability mass

When the sample space consists of discrete outcomes, then we can talk about the probability of each distinct outcome. For example, the sample space of a flipped coin has two discrete outcomes, and we talk about the probability of head or tail. The sample space of a six-sided die has six discrete outcomes, and we talk about the probability of 1 dot, 2 dots, and so forth.

For continuous outcome spaces, we can discretize the space into a finite set of mutually exclusive and exhaustive “bins.” For example, although heights of people are a continuous scale, we can divide the scale into a finite number of intervals, such as < 51″, 51″ to 53″, 53″ to 55″, 55″ to 57″, …, > 83″. Then we can talk about the probability that a randomly selected person falls into any of those intervals. Suppose that we randomly sample 10,000 people and measure the heights very accurately. The top panel of Figure 4.2 shows a scatter plot of the 10,000 measurements, with vertical dashed lines marking the intervals. In particular, the number of measurements that fall within the interval 63″ to 65″ is 1,473, which means that the (estimated) probability of falling in that interval is 1,473/10, 000 = 0.1473.

The probability of a discrete outcome, such as the probability of falling into an interval on a continuous scale, is referred to as a probability mass. Loosely speaking, the term “mass” refers the amount of stuff in an object. When the stuff is probability and the object is an interval of a scale, then the mass is the proportion of the outcomes in the interval. Notice that the sum of the probability masses across the intervals must be 1.

9.3.1 Terminology

We now turn to an introduction to formal probability theory which necessarily involves some new terminology. The sample space refers to the list of all possible outcomes of an experiment. Each possible outcome is represented by a single point in the sample space, irrespective of the number of ways that outcome can occur. For example, the sample space associated with the experiment of rolling a single fair dice is formed from the outcomes

Mathematically, the sample space is a set often denoted by Ω. Recall that set theory was discussed in Chapter 1. The sample spaces in the motivating examples are then written formally as

and

Ω2=Ⓡ,Ⓑ,Ⓖ,Ⓨ

The discussion in this chapter will be entirely concerned with discrete sample spaces. That is, we will consider experiments that can result in one of any individually stated number of outcomes. However, it is also possible to have continuous sample spaces. For example, if we were looking at the lifespan of male hamsters, the outcome of the experiment of recording the time from birth until death will result in a positive real number2 and in this case Ω =R+.

Example 9.9

State the sample space associated with the following experiments.

Tossing a coin.

Drawing a single ball from a bag containing balls ⒷⒷⒼⓎ.

Drawing two balls without replacement from the bag in part b. (assume order does not matter).

The total score obtained from rolling two fair dice.

An event is defined as a collection of outcomes of an experiment that we might wish to associate a probability to. A simple example is the event that we obtain

Consider now the particular event

,

,

,

,

, or

, which is interpreted as being the event that we obtain one of either

,

,

,

,

, or

. In this case,

C=Ω

which means that C is a certain event. Technically the empty set ∅ is also a valid event and refers to an impossible event.

It is sensible to understand events as being any subset of the sample space, an empty set, or the sample space itself. There are some complications in this statement around sample spaces that are uncountably infinite, but these considerations are beyond the scope of this book.

Example 9.10

Use set notation to express the following events associated with rolling a single dice.

A is the event that we obtain

B is the event that we obtain an even score.

C is the event that we obtain a score less than 3.

D is the event that event A does not occur.

E is the event that a negative score is obtained.

1.7.1 Random Variables

Let Ω be a sample space and ℱ a σ-algebra of the subset of Ω. A function ℙ(∙) defined on ℱ and taking values in the unit interval [0, 1] is called a probability measure, if

ℙ(Ω) = 1;

ℙ(A) ≥ 0 for all A ∈ ℱ;

for an at most countable family {An, n ≥ 1} of mutually disjoint event, we have

ℙ∪n≥1A n=∑n≥1ℙAn.

The triple (Ω, ℱ, ℙ) is a probability space.

sF = (ℱt)t ≥ 0 is a family of sub-σ-algebras ℱt of σ-algebra ℱ such that ℱs ⊂ ℱt for 0 ≤ s < t < ∞. PF = (Ω, ℱ, F, ℙ) is said to be a filtered probability space. We say that a filtration ℱt satisfies the usual conditions if ℱ0 contains all ℙ-null sets of ℱ and ℱt=∩ε>0ℱt+ε for every t ≥ 0. If the last condition is satisfied, we say that a filtration ℱ is right continuous.

Let (X, ℬX) be measurable space, we mean an (ℱ, ℬX)-measurable mapping x : Ω → X, i.e., such that x− 1(A) ∈ ℱ for every A ∈ ℬX, where as usual, ℬX denotes the Borel σ-algebra on X and x− 1(A) = {ω ∈ Ω : x(ω) ∈ A}. We shall also say that x is a random variable on Ω with values at X.

The integral of an integrable random variable x is called its mean value or expectation and is denoted by

Ex=∫xwdP.

Let K and ℋ be separable Hilbert spaces, and Q be either a symmetric nonnegative definite trace-class operator on K or Q = IK, the identity operator on K. In case Q is trace-class, we will always assume that its all eigenvalues λj > 0,.. .; otherwise we can start with the Hilbert space ker(Q)⊥ instead of K. The associated eigenvectors forming an orthonormal basis (ONB) in K will be denoted by ej.

Denote ℒ(K, ℋ) by all bounded linear operators from K to ℋ. Then the space of Hilbert-Schmidt operators from K to ℋ is defined as

ℒ2Kℋ=Φ∈ℒ Kℋ:∑i=1∞Φeiℋ2<∞.

It is well known (see [214]) that ℒ2Kℋ equipped with the norm

Φℒ2Kℋ=∑i=1∞ΦeiH2

is a Hilbert space.

On the other hand, the space Q12K equipped with the scalar product

uvQ12K=∑j=1∞1λjuejKve jK

is a separable Hilbert space with an ONB λj12ej∞j=1.

Consider ℒ20=ℒ2Q12K,ℋ, the space of Hilbert-Schmidt operators from Q12 K ℋ. If e˜j∞j=1 is an ONB in ℋ, then the Hilbert-Schmidt norm of an operator Φ∈ℒ02 is given by

Φℒ20 =∑i,j=1∞Φλ j12ej,e˜jℋ2= ∑i,j=1∞ΦQ12ej,e˜iℋ2=ΦQ122ℒ2Kℋ=trΦQ12Φ Q12*.

The Sample Space: A List of What Might Happen

Each random experiment has a sample space, which is a list of all possible outcomes of the random experiment, prepared in advance without knowledge of what will happen when the experiment is run. Note that there is nothing random about the sample space. It is the (definite) list of things that might happen. This is an effective way to make a random situation more definite, and it will often help to clarify your thinking as well. Here are the sample spaces corresponding to the previous random experiments:

For the family income survey, the sample space is a list of all possible income values. Let us assume that income must be zero or a positive number, and think of the sample space as a list of all nonnegative dollar amounts:

For the focus group choosing the design of a new reading lamp, the sample space is much smaller, consisting of the seven proposed designs:

For the quality testing of frozen dinners, the sample space is the collection of all possible lists of quality numbers, one for each of the three dinners tested. This is a collection of lists of three numbers, where each number is from 1 to 2 (representing the quality of that dinner). This sample space begins with a list of all 1’s (ie, “1, 1, 1” indicating poor quality for all three dinners) and ends with a list of all 2’s (ie, “2, 2, 2” indicating good (if not gourmet) quality for all three dinners tested). Somewhere within this collection would be included all possible quality score lists (including, eg, “1, 2, 2”)5:

Problems 9.1

1.

Find the sample space, its cardinal number, and the probability of the desired event for each of the following scenarios:

Pick a letter, at random, out of the English alphabet. Desired event: choosing a vowel.

Pick a date, at random, for the Calendar Year 2008. Desired event: choosing December 7th.

Pick a US president, at random, from a list of all the presidents. Desired event: choosing Abraham Lincoln.

Pick a US president, at random, from a list of all the presidents. Desired event: choosing Grover Cleveland.

Pick a card, at random, from a well-shuffled deck of regular playing cards. Desired event: choosing the Ace of Spades.

Pick a card, at random, from a well-shuffled deck of Pinochle playing cards. Desired event: choosing the Ace of Spades.

Roll a pair of fair dice. Desired event: getting a roll of “2” (snake eyes).

Roll a pair of fair dice. Desired event: getting a roll of “12” (boxcars).

Roll a pair of fair dice. Desired event: getting a roll of “8”.

Roll a pair of fair dice. Desired event: getting a roll of “11”.

Roll a pair of fair dice. Desired event: getting a roll of an even number.

Roll a pair of fair dice. Desired event: getting a roll of a number that is a perfect square.

Roll a pair of fair dice. Desired event: getting a roll of a number that is a perfect cube.

Roll a pair of fair dice. Desired event: getting a roll of a number that is a multiple of 3.

Roll a pair of fair dice. Desired event: getting a roll of a number that is divisible by 3.

Roll a pair of fair dice. Desired event: getting a roll of “13”.

Suppose we were to roll three fair dice: a red one first, followed by a white die, followed by a blue die. Describe the sample space and find its cardinal number.

Suppose the probability for event A is known to be 0.4. Find the cardinal number of the sample space if N(A) = 36.

Suppose the probability for event B is known to be 0.65. Find the cardinal number of B, if the cardinal number of the sample space, S, is N(S) = 3000.

4.3 Properties of Probability

It is an empirical fact that if an experiment is continually repeated under the same conditions, then, for any event A, the proportion of times that the outcome is contained in A approaches some value as the number of repetitions increases. For example, if a coin is continually flipped, then the proportion of flips landing on tails will approach some value as the number of flips increases. It is this long-run proportion, or relative frequency, that we often have in mind when we speak of the probability of an event.

Consider an experiment whose sample space is S. We suppose that for each event A there is a number, denoted P(A) and called the probability of event A, that is in accord with the following three properties.

PROPERTY 1: For any event A, the probability of A is a number between 0 and 1. That is,

0≤P(A)≤1

PROPERTY 2: The probability of sample space S is 1. Symbolically,

P(S)=1

PROPERTY 3: The probability of the union of disjoint events is equal to the sum of the probabilities of these events. For instance, if A and B are disjoint, then

P(A∪B)=P(A)+P(B)

The quantity P(A) represents the probability that the outcome of the experiment is contained in event A. Property 1 states that the probability that the outcome of the experiment is contained in A is some value between 0 and 1. Property 2 states that, with probability 1, the outcome of the experiment will be an element of sample space S. Property 3 states that if events A and B cannot simultaneously occur, then the probability that the outcome of the experiment is contained in either A or B is equal to the sum of the probability that it is in A and the probability that it is in B.

If we interpret P(A) as the long-run relative frequency of event A, then the stated conditions are satisfied. The proportion of experiments in which the outcome is contained in A would certainly be a number between 0 and 1. The proportion of experiments in which the outcome is contained in S is 1 since all outcomes are contained in sample space S. Finally, if A and B have no outcomes in common, then the proportion of experiments whose outcome is in either A or B is equal to the proportion whose outcome is in A plus the proportion whose outcome is in B. For instance, if the proportion of time that a pair of rolled dice sums to 7 is 1/6 and the proportion of time that they sum to 11 is 1/18, then the proportion of time that they sum to either 7 or 11 is 1/6+1/18=2/9.

Properties 1, 2, and 3 can be used to establish some general results concerning probabilities. For instance, since A and Ac are disjoint events whose union is the entire sample space, we can write

S=A∪Ac

Using properties 2 and 3 now yields the following.

1=P(S) by property 2=P(A∪Ac)=P(A)+P(Ac)by property  3

Therefore, we see that

P(Ac)=1−P(A)

In words, the probability that the outcome of the experiment is not contained in A is 1 minus the probability that it is. For instance, if the probability of obtaining heads on the toss of a coin is 0.4, then the probability of obtaining tails is 0.6.

The odds of an event A, call it o(A), is defined by

o(A)=P(A)P(Ac )=P(A)1−P(A)

Thus the odds of an event A indicates how many times more likely it is that A occurs than that it does not occur. For instance, if P(A)=2/3, then o(A)= 2/31/3=2, and so the odds are 2 meaning that it is twice as likely that A occurs as it is that it does not occur. (Common terminology is to say that the odds are 2 to 1 in favor of A occurring.)

The following formula relates the probability of the union of events A and B, which are not necessarily disjoint, to P(A), P(B), and the probability of the intersection of A and B. It is often called the addition rule of probability.

Addition Rule

For any events A and B,

P(A∪B)=P(A)+P(B)−P(A∩B)

To see why the addition rule holds, note that P( A∪B) is the probability of all outcomes that are either in A or in B. On the other hand, P(A)+P(B) is the probability of all the outcomes that are in A plus the probability of all the outcomes that are in B. Since any outcome that is in both A and B is counted twice in P(A)+P(B) and only once in P(A∪B) (see Fig. 4.5), it follows that

P(A)+P(B)=P(A∪B)+P(A∩B)

Subtracting P(A∩B) from both sides of the preceding equation gives the addition rule.

Example 4.3 illustrates the use of the addition rule.

As an illustration of the interpretation of probability as a long-run relative frequency, we have simulated 10,000 flips of a perfectly symmetric coin. The total numbers of heads and tails that occurred in the first 10, 50, 100, 500, 2000, 6000, 8000, and 10,000 flips, along with the proportion of them that was heads, are presented in Table 4.1. Note how the proportion of the flips that lands heads becomes very close to 0.5 as the number of flips increases.

Table 4.1. 10,000 Flips of a Symmetric Coin

The results of 10,000 simulated rolls of a perfectly symmetric die are presented in Table 4.2.

Table 4.2. 10,000 Rolls of a Symmetric Die

Note: 1/6=0.166667.

1.3 Probabilities Defined on Events

Consider an experiment whose sample space is S. For each event E of the sample space S, we assume that a number P(E) is defined and satisfies the following three conditions:

0⩽P(E)⩽1.

P(S)=1.

For any sequence of events E1 ,E2,… that are mutually exclusive, that is, events for which EnEm=Ø when n≠m, then

P(⋃n=1∞E n)=∑n=1∞P(En)

We refer to P(E) as the probability of the event E.

Since the events E and Ec are always mutually exclusive and since E ∪Ec=S we have by (ii) and (iii) that

1=P(S)=P(E∪Ec)=P(E)+P(Ec)

or

(1.1)P(Ec)=1−P(E)

In words, Eq. (1.1) states that the probability that an event does not occur is one minus the probability that it does occur.

We shall now derive a formula for P(E∪F), the probability of all outcomes either in E or in F. To do so, consider P(E)+P(F), which is the probability of all outcomes in E plus the probability of all points in F. Since any outcome that is in both E and F will be counted twice in P(E)+P(F) and only once in P(E∪F), we must have

P(E)+P(F)=P(E∪F)+P(EF)

or equivalently

(1.2)P(E∪F)=P(E)+P(F)−P(EF)

Note that when E and F are mutually exclusive (that is, when EF=Ø), then Eq. (1.2) states that

P(E∪F)=P(E)+P(F)−P(Ø)=P(E)+P(F)

a result which also follows from condition (iii). (Why is P(Ø)=0?)

We may also calculate the probability that any one of the three events E or F or G occurs. This is done as follows:

P(E∪F∪G)=P((E∪F)∪G)

which by Eq. (1.2) equals

P(E∪F)+P(G)−P((E∪F)G)

Now we leave it for you to show that the events (E∪F) G and EG∪FG are equivalent, and hence the preceding equals

(1.3)P(E∪ F∪G)=P(E)+P(F) −P(EF)+P(G)−P(EG∪FG)=P(E)+P(F)−P( EF)+P(G)−P(EG)−P(FG)+P(EGFG)=P(E)+P (F)+P(G)−P(EF)−P(EG )−P(FG)+P(EFG)

In fact, it can be shown by induction that, for any n events E1,E2,E3,…,En,

(1.4)P(E1∪E2∪⋯∪En)=∑iP(Ei)−∑i<jP(EiE j)+∑i<j<kP(EiEjEk)−∑i<j<k<lP(EiEjEk El)+⋯+(−1)n+ 1P(E1E2⋯En)

In words, Eq. (1.4), known as the inclusion–exclusion identity, states that the probability of the union of n events equals the sum of the probabilities of these events taken one at a time minus the sum of the probabilities of these events taken two at a time plus the sum of the probabilities of these events taken three at a time, and so on.