Laws of Chance Tables

Used for testing claims of success greater than what can be attributed to random chance.

How to use the tables

The column headings along the top are the number of tries that the performer gets.  The row headings along the left side are the mathematical probability of success, expressed in percent and as an equivalent fraction.  The values within the table are the range of the number of successes that can be attributed to random chance.  If, after completing all tries, the number of successes achieved is outside the range shown in the table entry, then the success can be attributed to the performer's special skill or "magical" ability.

A table entry of the form "0-b" includes zero successes within the range of random chance.  The test is to see if the number of successes falls above b.

A table entry of the form "a-b" allows testing for both unusual success and unusual failure.  If the number of successes falls below a, you could conclude that there is something amiss with the mechanisms of the test or with the performer, or that there is "anti-magic" in the air.

A table entry of "-" means that the number of tries is insufficient to test a claim of that probability.  That is, it is within the range of random chance to be successful on all tries, and therefore a test with more tries is needed to adequately test the claim.

An example

The mysterious Jerome claims that he can name the suit of any ordinary playing card without seeing it, with a success rate significantly better than random chance.  You are called upon to test Jerome's claim.  First, you construct a preliminary test without too many tries, and you consult Table I below.  The mathematical probability of guessing a suit correctly is 1 in 4, or 25%.  You could run a test with as few as 5 tries.  The entry for 5 tries at 25% probability is "0-4", so Jerome would have to get all 5 guesses correct.  Jerome might find that a lot to ask of him.  A more friendly test would be 10 tries, which as you can see from the table would require Jerome to get more than 6 guesses correct.  To pass the preliminary test of his claim, he would need to get at least 7 successes out of 10 tries.

Jerome scores 7 successes!  Maybe this guy is for real, but 1 out of 100 preliminary tests will yield a false positive.  You now construct a final test and consult Table II below.  You decide to go with 20 tries.  The entry is "0-13", so Jerome must get at least 14 successes out of 20 tries to prove his ability conclusively.  When you run the test, Jerome names the card suit correctly only 4 times out of 20, slightly less than the most likely number of successes 5.  For now, Jerome's claim of power goes unproven.

Table I.  Range of Number of Successes Attributable to Random Chance, for Preliminary Testing.

Chance of number of successes being outside the range shown:  less than 1 in 100.  If the range is shown as simply "0", then there is less than a 1 in 100 chance of getting any successes in that number of tries.

Table II.  Range of Number of Successes Attributable to Random Chance, for Final Testing.

Chance of number of successes being outside the range shown:  less than 1 in 10,000.

Table III.  Range of Number of Successes Attributable to Random Chance, for Extremely Stringent Testing.

Chance of number of successes being outside the range shown:  less than 1 in 1,000,000.  I have heard, but have not confirmed, that this is the level of testing required for a 'formal test' in Randi's million dollar challenge.

Last update 19-Sep-2004.  Joe Czapski, Boston, Mass.  czaptournamentdesignorg
May be freely copied and distributed.