Describe sensitivity, specificity and predictive value.

Validity, Reliability and Predictive Value

• A test can be reliable but invalid — consistently giving the same wrong answer, like a weighing scale reading 2 kg heavy every time. Reliability is necessary but not sufficient for validity
• Sensitivity = the ability to correctly identify those with the disease = TP / (TP + FN)
• Specificity = the ability to correctly identify those without the disease = TN / (TN + FP)
• The mnemonics are worth carrying — SnNOut: a highly SeNsitive test, when Negative, rules the disease out; and SpPIn: a highly SPecific test, when Positive, rules the disease IN
• Sensitivity and specificity are properties of the test and do not change with the prevalence of disease — which is what distinguishes them from the predictive values
• The trade-off — for any test with a continuous variable, moving the cut-off to increase sensitivity decreases specificity, and vice versa. The ROC (receiver operating characteristic) curve plots this relationship, and the area under the curve summarises overall accuracy
• Which TO favour depends on the consequences of each error, and this is the reasoning an examiner wants:
• Favour sensitivity when missing a case is dangerous — a serious but treatable disease, an infectious disease that would spread, or screening blood donors, where a missed infection is transmitted
• Favour specificity when a false positive is costly or harmful — where confirmation is expensive or invasive, or where a false label causes serious harm, as with a diagnosis of cancer or HIV
• Hence the standard sequence in practice — a sensitive test to screen, followed by a specific test to confirm, which is exactly the ELISA-then-Western-blot strategy formerly used for HIV
• Predictive values answer the question the clinician and patient actually ask — not "how good is the test" but "given this result, what is the probability that I have the disease?"
• Positive predictive value = TP / (TP + FP) — the proportion of those testing positive who truly have the disease
• Negative predictive value = TN / (TN + FN) — the proportion of those testing negative who truly do not
• The crucial point, and the one most often examined: predictive values depend heavily ON the prevalence of disease in the population tested, whereas sensitivity and specificity do not
• As prevalence falls, the positive predictive value falls and the negative predictive value rises
• The practical consequence is dramatic — a test with excellent sensitivity and specificity applied to a low-prevalence population produces mostly false positives
• To make it concrete — with a test of 99% sensitivity and 99% specificity applied where the prevalence is 1 in 10,000, a positive result means the person is still far more likely not to have the disease than to have it, because the false positives drawn from the vast healthy majority outnumber the true positives
• This IS the single strongest argument for screening high-risk groups rather than whole populations, and for never applying a screening test to a population in which the disease is very rare
• Other measures — accuracy, the overall proportion correctly classified, which is misleading when prevalence is low; likelihood ratios, which combine sensitivity and specificity and are independent of prevalence while still usable at the bedside; and yield, the number of new cases detected per 1000 screened
• Reliability is assessed by — intra-observer and inter-observer variation, measured by the kappa statistic, which corrects for agreement occurring by chance; and test-retest repeatability
• Reliability is improved by — standardising the procedure, training observers, using more than one observer, and automating measurement
• Sensitivity and specificity belong to the test and predictive values to the population being tested
• Predictive value falls as prevalence falls, which is the point examined most often in this topic
• An excellent test in a rare disease gives mostly false positives, because the healthy majority supplies so many of them
• SnNOut and SpPIn — sensitive tests rule out and specific tests rule in, which is worth carrying verbatim
• Screen with a sensitive test and confirm with a specific one, which is the standard sequence in practice
• Favour sensitivity where missing a case is dangerous, as in blood donor screening or a treatable serious disease
• Favour specificity where a false label harms, as with a diagnosis of cancer or HIV
• Moving the cut-off trades one for the other, which the ROC curve displays and the area under it summarises
• A test can be reliable but invalid, giving the same wrong answer every time
• Kappa corrects for chance agreement, which raw percentage agreement does not
• Accuracy misleads when prevalence is low, since a test calling everyone negative appears highly accurate
• Likelihood ratios are independent of prevalence while remaining usable at the bedside, which is their advantage
• Standardise and train to improve reliability, and automate the measurement wherever it is possible