1) The mathematical theory of probability assumes
that we have a well defined __repeatable__ (in principle) experiment,
which has as its outcome a set of well defined, __mutually exclusive__,
events.

Examples:

In the experiment of rolling one die, the mutually exclusive outcomes are the die landing with either the 1, 2, 3, 4, 5, or 6 face up.

2) We assume that in any particular individual trial
of the experiment, the outcome for that individual trial cannot be
predicted
or known before hand - it is controlled by chance. However, when a __very
large number__ of independent trials of the experiment are performed,
one finds that each possible outcome occurs a well defined fraction of
the time. This fraction is called the __probability__ for that
particular
outcome to occur. The probability for an outcome is always a number between
0 and 1. If the probability is zero, we say that this outcome can never
occur. If the probability is one, we say that this outcome always occurs
with complete certainty.

Examples:

When we role a die a very large number of times, we find that we get any given face 1/6 of the time. The probability for the 1 face to appear is therefore 1/6. Similarly the probability for the 2 (3, 4, 5, or 6) face to appear is also 1/6.

3) The probability for a given outcome might be calculable from some underlying assumptions. Or the probability for an outcome can be determined experimentally by doing many trials.

Example:

When we flip a coin, we assume that there is no major
difference between the two sides, and there is no exacting way we perform
the flip in order to try and influence the outcome. Because of all the
random factors beyond our control that enter the flipping process (force
with which the coin is flipped, motion of the air in the room, position
of our hand when we catch the coin...) we therefore expect a probability
of 1/2 for heads, and 1/2 for tails. Each possible outcome is __equally
likely__. However, if we did a very large number of trial flips, and
consistently found heads occurring 3/4 of the time, and tails 1/4 of the
time, we would know that our assumption of equally likely outcomes was
false - we are dealing with a loaded coin. Performing the experiment is
the way to test the assumptions!

Let's now see how these ideas work for some more complicated examples.

1) Consider the experiment of flipping of 4 coins.

Each coin flip has 2 possible outcomes, so the flipping of 4 coins has 2x2x2x2 = 16 possible outcomes. We can enumerate all possible outcomes as follows, where H indicates a head, and T a tail:

HHHH THHH

HHHT THHT

HHTH THTH

HHTT THTT

HTHH TTHH

HTHT TTHT

HTTH TTTH

HTTT TTTT

If we assume that each individual coin is equally likely to come up heads or tails, then each of the above 16 outcomes to 4 flips is equally likely. Each occurs a fraction one out of 16 times, or each has a probability of 1/16.

Alternatively, we could argue that the 1st coin has probability 1/2 to come up heads or tails, the 2nd coin has probability 1/2 to come up heads or tails, and so on for the 3rd and 4th coins, so that the probability for any one particular sequence of heads and tails is just (1/2)x(1/2)x(1/2)x(1/2)=(1/16).

Now lets ask: what is the probability that in 4
flips,
one gets N heads, where N=0, 1, 2, 3, or 4. We can get this just by
counting
the number of outcomes above which have the desired number of heads, and
dividing by the total number of possible outcomes, 16.

__N__ __# outcomes with
N heads__ __probability to get N
heads__

0 1 1/16 = 0.0625

1 4 4/16 = 1/4 = 0.25

2 6 6/16 = 3/8 = 0.375

3 4 4/16 = 1/4 = 0.25

4 1 1/16 = 0.0625

We can plot these results on a graph as shown below.

The dashed line is shown just as a guide to the
eye. Notice that the curve has a "bell" shape. The most likely outcome
is for N=2 heads, where the curve reaches its maximum value. This is just
what you would expect: if each coin is equally likely to land heads as
tails, in four flips, half should come up heads, that is N =
4x(1/2)
= 2 is the most likely outcome. Note however that an occurrence of N =
1 or N = 3 is not so unlikely - they occur 1/4 or 25% of the time. To have
an occurrence of only N = 0, or N = 4 (no heads, or all heads) is much
less likely - they occur only 1/16 or 6.25% of the time.

The above procedure is in principle the way to solve all problems in probability. Define the experiment, enumerate all possible mutually exclusive outcomes (which are usually assumed to be each equally likely), and then count the number of these outcomes which have the particular property being tested for (here for example, the number of heads). Dividing this number by the total number of possible outcomes then gives the probability of the system to have that particular property.

Often, however, the number of possible outcomes may be so large that an explicit enumeration would become very tedious. In such cases, one can resort to more subtle thinking to arrive at the desired probabilities. For example, we can deduce the probabilities to get N heads in 4 flips as follows:

N=0: There is only one possible outcome that gives 0 heads, namely when each flip results in a tail. The probability is therefore 1/16.

N=4: There is only one possible outcome that gives 4 heads, namely when each flip results in a head. The probability is therefore 1/16.

N=1: There are 4 possible outcomes which will have only one coin heads. It may be that the 1st coin is heads, and all others are tails; or it may be that the 2nd coin is heads, and all others are tails; or it may be that the 3rd (or the 4th) coin is heads, and all others are tails. Since there are 4 possible outcomes with one head only, the probability is 4/16 = 1/4.

N=3: To get 3 heads, means that one gets only one tail. This tail can be either the 1st coin, the 2nd coin, the 3rd, or the 4th coin. Thus there are only 4 outcomes which have three heads. The probability is 4/16 = 1/4.

N=2: To enumerate directly all the possible outcomes which have exactly 2 heads only, is a bit trickier than the other cases. We will come to it shortly. But we can get the desired probability for N=2 the following way: We have already enumerated all possible outcomes with either N = 0, 1, 3, or 4 heads. These account for 1 + 4 + 4 + 1 = 10 possible outcomes. The only outcomes not include in these 10 are those with exactly N=2 heads. Since there are 16 possible outcomes, and 10 do not have N=2 heads, there must therefore be exactly 16 - 10 = 6 outcomes which do have exactly N=2 heads. The probability for N=2 is therefore 6/16 = 3/8.

2) Consider the experiment of rolling 3 dice, each of which has 6 sides.

What is the probability that no two dice land with the same number side up, i.e. each of the three dice rolls a different number?

Since each die has 6 possible outcomes, the number of possible outcomes for the roll of three dice is 6x6x6 = 216. We could enumerate all these 216 possibilities, and then count the number of outcomes in which each die has a different number. This is clearly too tedious! Instead we reason as follows:

When the 1st die is rolled, it can land with anyone of its 6 faces up. When the 2nd die is rolled, there are only 5 possible sides which will be consistent with our criterion that no two die land the same. When the 3rd die is rolled, it must land with a face different from die one and two; there are 4 possibilities. Therefore the total number of outcomes which have each of the three dice with a different side up is 6x5x4 = 120.

The probability for this to happen is then 120/216 = 5/9 = .55555...

Alternatively, we can say that the probability for the 1st die to land with any side up is 1, the probability for the 2nd die to land with a side different from the 1st die is 5/6, and the probability for the 3rd die to land with a side different from both the 1st and the 2nd die is 4/6. The probability that each of the three dice has a different side up is then (1)x(5/6)x(4/6) = 20/36 = 5/9.

We learn from the above examples three general rules about probabilities:

1) The probability than any one of a given group of mutually exclusive possible outcomes will actually occur in a given experiment, is the sum of the individual probabilities for each outcome in the group.

Example:

2) The probability for two independent events to both occur is just the product of the probabilities for each individual event.

Example:

Example:

1) If we roll 4 dice, what is the probability that at least one of them lands with the "6" face on top?

Since each dice can land 6 ways, and there are 4
dice, there are 6x6x6x6
= 64 = 1,296 different possible outcomes. We could
in principle write these all down and then count the number which have
at least one "6", but this is way too tedious. Instead, lets count the
number of outcomes which do not have __any__ "6"s showing. To have no
"6" showing, the first dice can land in any one of 5 possible ways.
Similarly
for the 2nd, 3rd, and 4th die. Thus the number of outcomes with no "6"
showing on any die is 5x5x5x5
= 54 = 625. Since we must either see no "6"s, or else
we see at least one "6", it follows that the number of outcomes with at
least one "6" must be the (total number of outcomes) - (number of outcomes
with no "6") = 1,296 - 625 = 671. The probability to have at least one
"6" is therefore 671/1296 = 0.5177469...

We could also have arranged our calculation as follows:

Written this way, we see that (5/6) is the probability that any give die lands without a "6" on top. (5/6)x(5/6)x(5/6)x(5/6) is therefore (general rule #2) the probability that all 4 dice land without a "6" on top. Since the probability of all possible outcomes must sum to unity (general rule #1), we therefore conclude that 1 - (5/6)x(5/6)x(5/6)x(5/6) is the probability that the opposite of "all 4 dice land without a "6" on top" happens. But the opposite of "all 4 dice land without a "6" on top" is that "at least one die lands with a "6" on top"!

In this example we see that we can divide all our
possible outcomes into two classes. One class which has the desired
property
we are testing for (in this case, at least one "6" on top), and a second
class which does not have the desired property (no dice has a "6" on top).
Often it turns out to be hard to directly calculate the probability for
the desired property, but easy to calculate the probability __not__
to find the desired property. The first probability is then just unity
minus the second probability.

2) What is the probability that in a group of N people, we will find at least 2 people with the same birthday? For the purpose of this question, we will consider people born on February 29 as if they were born on February 28!

This is equal to 1 - (probability no 2 people have same birthday)

Probability that no 2 people have the same birthday is:

=

since the first person can have any birthday, the
second person can have only one of the remaining 364 out of 365 possible
birthdays (i.e. not the same as the first), the third person can have only
one of the remaining 363 out 365 possible birthdays (i.e. not the same
as either the first or the second), and so on, down to the last Nth person.
So the probability that *at least* 2 people in a group of N have the
same birthday is:

p(N) = 1 -

We can plot p(N) versus N on a graph, as shown below.

Note that for N=10, p(10)=0.11695, or about 12%
chance
to have two people with the same birthday. For N=50, p(50)=0.97037, or
97% chance to have two people with the same birthday. These probabilities
are in general much larger than most people's intuition would have
guessed.

__Examples from Text:__

1) Two fair dice are rolled. What is the probability p that at least one die come up a 3?

We can do this two ways:

i) The straightforward way is as follows. To get at least one 3, would be consistent with the following three mutually exclusive outcomes:

the 1st die is a 3 and the 2nd is not: prob = (1/6)x(5/6)=5/36

the 1st die is not a 3 and the 2nd is: prob = (5/6)x((1/6)=5/36

both the 1st and 2nd come up 3: prob = (1/6)x(1/6)=1/36

sum of the above three cases is prob for at least one 3, p = 11/36

ii) A faster way is as follows: prob at least one 3 = 1 - (prob no 3's)

The probability to get no 3's is (5/6)x(5/6) = 25/36.

So the probability to get at least one 3 is, p = 1 - (25/36) = 11/36

2) What is the probability that a card drawn at random from an ordinary 52 deck of playing cards is a queen or a heart?

There are 4 queens and 13 hearts, so the probability to draw a queen is

4/52 and the probability to draw a heart is 13/52. But the probability to draw a queen or a heart is NOT the sum 4/52 + 13/52. This is because drawing a queen and drawing a heart are not mutually exclusive outcomes - the queen of hearts can meet both criteria! The number of cards which meet the criteria of being either a queen or a heart is only 16 - the 4 queens and the 12 remaining hearts which are not a queen. So the probability to draw a queen or a heart is 16/52 = 4/13.

3) Five coins are tossed. What is the probability that the number of heads exceeds the number of tails?

We can divide all possible outcomes into the following two mutually exclusive groups:

i) the number of heads flipped is more than the number of tails flipped

ii) the number of tails flipped is more than the number of heads flipped

Since the probability to flip a head is the same as the probability to flip a tail, the probability of outcome (i) must be equal to the probability of outcome (ii). So both must be equal to 1/2.

Note that this answer works for any odd number of coin flips.

4) 4 boys and 3 girls are standing in a line. If the position of each child in line is random, what is the probability that the first 3 places in line are all girls?

We will solve this problem two ways:

i) Lets us count the number of all the possible orderings of the 7 children which are consistent with the desired outcome (i.e. 3 girls in front) and divide by the total number of all possible orderings.

The total number of orderings is: 7x6x5x4x3x2x1

as there are 7 children to pick to be 1st in line, only 6 remaining children to pick to be 2nd in line, then only 5 remaining children to pick to be 3rd in line, and so on.

The number of orderings consistent with 3 girls in front is: (3x2x1)x(4x3x2x1)

as there are 3 girls to pick to be 1st in line, 2 remaining girls to pick to be 2nd in line, and the last girl must go 3rd in line; then there are 4 boys to place 4th in line, 3 remaining boys to place 5th in line, and so on.

So the probability to get all 3 girls in front is

ii) Alternatively, we can argue as follows. The probability that a girl is 1st in line is (3/7) since 3 of the 7 children are girls. The probability that a girl is 2nd in line, given that the 1st is a girl, is (2/6) since 2 of the remaining 6 children are girls. The probability that a girl is 3rd in line, given that the 1st and 2nd are girls, is (1/5) since now only 1 of the remaining 5 children is a girl. So the probability that the 1st, 2nd, and 3rd positions are all girls is

(3/7)x(2/6)x(1/5)=1/35