The Science of Testing

Let's try to understand these concepts with a few examples:

Problem A

Given: We are employing a new screening test designed to detect asthma in a population of 100,000 people.

Given: Ten percent of this population suffer from asthma.

Given: This new asthma screening test has a sensitivity of 98% and a specificity of 90%.

We now have enough information to evaluate the usefulness of this new test.

Our first task is to insert numbers into the nine boxes below from the information above. The total population is given as 100,000. This by definition is the total number of people and is placed in the lower right box.

Because the prevalence of asthma is 10%, we can easily conclude .10 × 100,000 = 10,000 people have asthma and 90,000 do not. These values assume the totals of the first and second columns respectively.

We now use the given values for sensitivity and specificity.

Given a sensitivity of 98% or .98, we can safely conclude that 98% of the individuals with the disease will have a positive screening test so a = .98 × 10,000 = 9,800. This is because by definition, sensitivity = a/(a + c) so .98 = a/10,000 a = 9,800
If a = 9,800, c must equal 200 because a + c =10,000 c= 10,000 – 9,800 = 200

Similarly, given a specificity of 90% or .90, we can safely conclude that 90 Percent of the individuals without the disease will test negative so d = .90 × 90,000 = 81,000. This is because by definition, specificity = d/(b + d) so .90 = d/90,000 d = .90 × 90,000 = 81,000

If d = 81,000, b must equal 9,000 because b + d = 90,000 d = 90,000 – 81,000 = 9,000

We can fill in this additional information as below.

The last two boxes are computed with simple addition to finally complete the table:

As an internal check on our work, please notice:

18,800 plus 81,200 totals 100,000 as we would expect.

We now have four groups of people with distinct characteristics:

Prevalence = 10%, Sensitivity = 98, Specificity = 90%

Third, we have two groups of people with a serious problem. The 200 people in column 1 have asthma but the test was reported negative. Furthermore, 9,000 people have a positive test for asthma but in fact are asthma free.

Now let's determine the positive and negative predictive values. (PPV and NPV)

PPV = [a/(a + b)] × 100%] so 9,800/18,800 × 100%= 52.1%

NPV = [d/(d + c)] × 100%] so 81,000/81,200 × 100% = 99.75 %

What does all this mean?

A positive predictive value of 52.1% means that only 52.1% of the people with a positive test actually have the disease. In other words, individuals with a positive test have a 52.1% chance of having asthma. Although this test correctly identified most (9,800 out of 10,000) of the people with asthma, in the process it incorrectly assigned a suspicion of asthma to an almost equal number (9,000) of individuals. This is very problematic.

A negative predictive value of 99.75% means that 99.75% of individuals with a negative test are in fact, asthma free. In other words, individuals with a negative test have a 99.75% chance of being asthma free. This provides helpful, reliable information. The small number (200 out of the 100,000 people tested) of individuals with false negative tests would have a false sense of security.

For almost all those testing negative, this test is an accurate assessment. For those testing positive, the test is almost meaningless as individuals are almost as likely to be asthma free as have asthma.

If this was an inexpensive, noninvasive, widely available test, its primary value would be identifying individuals who are asthma free. Those with a positive test would require additional, more expensive testing in order to answer the question.

Now let's look at this same test but let's change the disease prevalence to 2% and 25% and see what happens.

Prevalence = 2%, Sensitivity = 98, Specificity = 90%

PPV = 1,960/11,760 = 16.67% NPV = 88,200/88,240 = 99.95%

Prevalence = 25%, Sensitivity = 98, Specificity = 90%

PPV = 24,500/32,000 = 76.56% NPV = 67,500/68,000 = 99.26%

Summary of Data with Varying Prevalence

With this test, the lower the disease prevalence, the lower the PPV and the higher the NPV.

This example highlights the problems raised when performing mass screenings in populations where a low prevalence of disease is present. Each individual who has a positive test result but is actually disease free must endure additional unnecessary, often expensive, and sometimes painful testing, as well endure the anxiety of being diagnosed with a potentially serious condition. These screening tests establish disease-free individuals with a high degree of reliability. Individuals testing positive require further testing. As disease prevalence falls, the positive predictive value plummets.

Interpreting Home Pregnancy Tests

To correctly interpret home pregnancy tests, it is essential to know the sensitivity, specificity, and positive and negative predictive values for the test when performed by individuals without any medical or laboratory medicine training. Most home pregnancy test manufacturers have this data and will make it available upon request. Even knowing these facts and performing these tests as carefully as you can doesn’t guarantee the reliability of any individual test result.

If you suspect pregnancy, please see an appropriate physician to answer this vitally important question.