Author:

Giovanni Boniolo

Abstract

Probability and statistics are two mathematical subjects that seem intuitive. Unfortunately, it is not so, and biases and fallacies can mislead our clinical intuition. We run serious risks in our clinical decisions if we do not correctly grasp the conceptual import of probability and statistics. This is particularly true for oncologists, since if they fall into the traps of their wrong intuition, they could arrive at wrong diagnoses, wrong treatments, and wrong prognoses. Moreover, they could not have the right knowledge to communicate properly with patients.

Probabilistic Biases

In 2012, a surprising paper was published in the Annals of Internal Medicine. Its provocative title was “Do physicians understand cancer screening statistics?” (Wegwarth et al. 2012; also Gigerenzer 2013). It reported a study conducted in 2010 on 297 U.S. primary care physicians who were asked the meaning of the survival rate in the case of cancer screening. The result was shocking: more than 50% of the physicians did not understand the survival rate. Let us focus on this with the authors’ words: “The benefit of screening tests is often communicated to physicians and patients in the form of improved survival rates. These data typically show a numerically large advantage for screening (for example, survival is 90% in early-stage disease but only 20% in late-stage disease). Although such statistics are intuitively appealing, they do not provide evidence of the benefit of screening. The fundamental problem of survival statistics in the context of screening is that they are susceptible to leadtime and overdiagnosis biases. […] Imagine a group of patients in whom cancer was diagnosed because of symptoms at age 67 years, all of whom die at age 70 years. Each patient survives only 3 years, so the 5-year survival for the group is 0%. Now imagine that the same group undergoes screening. Screening tests by definition lead to earlier diagnosis. Suppose that with screening, cancer is diagnosed in all patients at age 60 years, but they nevertheless die at age 70 years. In this scenario, each patient survives 10 years, so the 5-year survival for the group is 100%. Yet, despite this dramatic improvement in survival (from 0% to 100%), nothing has changed about how many people die or when. […] Imagine a population in which cancer detected because of symptoms is diagnosed in 1000 people and that 400 are alive 5 years later, for a 5-year survival rate of 40%. Now imagine that the population had undergone screening. Screening can detect cases of cancer that are not destined to progress, a phenomenon known as overdiagnosis. Since these cases of cancer are nonprogressive, these patients will survive 5 years. The addition of the overdiagnosed cancer cases distorts the 5-year survival statistic by inflating the numerator and the denominator. Suppose 2000 cases of cancer were overdiagnosed. In this case, after 5 years, 2400 people will be alive out of the 3000 diagnosed—an 80% 5-year survival rate. But again, despite the dramatic improvement in survival (from 40% to 80%), nothing has changed about how many people die or when.”.

This passage mentions two (probabilistic) biases (the leadtime bias and the overdiagnosis bias). Generally speaking, a bias is a cognitive error in the evaluation of a situation, due to irrational prejudices, superficialities, and a not-complete comprehension of all its aspects, that impedes the rational and correct understanding of it.

Probability and statistics are two mathematical subjects that seem intuitive. Unfortunately, it is not so and biases and fallacies can mislead the clinical intuition. If we do not correctly grasp the conceptual (I would say, philosophical) import of probability and statistics, we run serious risks in our decisions. This is particularly true for oncologists, since if they fall into the traps of their wrong intuition, they could arrive at wrong diagnoses, wrong treatments, and wrong prognoses. Moreover, they could not have the right knowledge to communicate properly with patients (Boniolo, Teira 2016).

Probabilistic fallacies

The biases do not concern only the interpretation of survival rates, but they could be treacherously present in many other situations. However, there are other sources of misdiagnoses, mistreatments, and misprognoses: the probabilistic fallacies. These are non-valid or weak probabilistic reasoning in constructing a diagnostic, therapeutic, or prognostic argument.

To begin with, we should take into consideration that there is a difference between deterministic causality and probabilistic causality. In the first case, any time we have a causal event, we necessarily have the correlated effect event. Any time we have the mutation of the HTT gene (the causal event), we deterministically have Huntington's disease (the effect event). That is, the probability of having Huntington's disease given the mutation of the Huntingtin gene is 1:

In the second case, any time we have a causal event, we increase the probability of having the correlated effect event. The presence of a mutated BRAC1 (the causal event) increases the probability of having breast cancer (the effect event). That is, the probability of having breast cancer given the mutated BRAC1 is greater than the probability of having breast cancer given the non-mutated BRAC1:

Confusing deterministic causality and probabilistic causality means falling into a reasoning fallacy that could lead to wrong clinical decisions. The same happens if we confuse statistical correlations with (deterministic or probabilistic) causal correlations.

Another source of errors lies in the confusion between statistical significance and clinical significance of a test, that is, between sensitivity and specificity, on the one hand, and positive and negative predictive value, on the other hand.^[1] Gigerenzer et al. (2007) posed to 160 gynecologists the following question: given a breast cancer screening mammogram with a sensitivity of 90%, a false-positive rate of 9%, and a disease prevalence of 1%, if a woman tests positive, what are her chances of having breast cancer? The majority of the gynecologists unexpectedly grossly overestimated the probability of cancer, claiming that 8 or 9 out of 10 tested positive women would have the disease. Unfortunately, they made a probabilistic fallacy (named base rate fallacy or prosecutor fallacy): they misunderstood a statistical result with a clinical result.

Let us illustrate better this point considering the case of a test detecting a marker for colorectal cancer, i.e., the M2-PK. Let us suppose that it is a very reliable test with a sensitivity of 85% and a specificity of 95%. We know also that the cumulative risk of colorectal cancer in persons aged under 75 is 3.9% in Europe. There is only one right way of putting all this information together: using the powerful and elegant Bayes’ theorem. Abstractly, it claims that

where

• P(H/E), called the posterior probability, is the probability of the hypothesis H given the evidence E, and it gives the degree of belief in H, having taken E into account.
• P(E/H) is the probability of E given H.
• P(H), called the prior probability, is the probability of H alone and it gives the initial degree of belief in H.
• P(E) is the probability of the evidence E.

In our case,

• P(H/E) is the probability of having the disease D (the hypothesis) if the test T (the evidence) is positive, that is, it is the positive predictive value P(D/T+).
• P(E/H) is P(T+/D), that is, the sensitivity.
• P(H) is the probability of the disease, that is, the prevalence P(D).
• P(E) is (TP+FP)/N, where TP indicates the true positives, TP the false positives, and N the population under consideration.

Thus,

We have that the probability before the test of having colorectal cancer is P(D) = 3,9%. After the positive test, it is P(D/T+) = 43%.

This is what the oncologist is interested in; not in the sensitivity or specificity values. Sensitivity and specificity concern statistics, instead the positive and negative predictive values concern the clinic. And these, as seen, can be found if we apply the Bayes theorem, or if we reason in a Bayesian way (see Motulsky 2017).

Now we could also understand why the majority of the gynecologists above committed a fallacy. If we apply Bayes theorem, we see that the probability of having breast cancer given a positive mammogram is the sensitivity of the test (90%) times the disease prevalence (1%) divided by the probability of obtaining a (true or false) positive in the test, that is, they would have calculated the positive predictive value, P(D/T+), which is about 10%.

Patients should be informed

Early detection of a neoplastic disease followed by immediate treatment might be a good strategy for improving survival and reducing mortality. This is why screening programs are beneficial. Nevertheless, some warnings should be provided to the participants.

Let us think about breast cancer screenings. This means that very early and/or dormant forms of the disease are identified. However, the transition from the dormant to the active stage of the disease has not yet been completely understood, and this could lead to biases when both the research data on screening tools and clinical information are interpreted. In particular, potential biases could arise when evaluating screening effectiveness (these include leadtime/survival bias, healthy volunteer bias, and length/disease development bias).^[2]

Not only. There is, on the one hand, the just discussed problem of base-rate fallacy whenever risks associated with screening are communicated, and, on the other hand, the problem related to the misunderstanding concerning the trade-offs between extended life and quality of life. Furthermore, screening tests are more likely to detect dormant cancers, which in turn increases the risk of overdiagnosis (Brodersen et al. 2014).

Various estimates suggest that 2–2.5 lives are saved for each case of overdiagnosis (Puliti et al. 2012; Paci 2012). This means that for every overdiagnosed and unnecessarily treated woman, there are about two or three patients who will benefit from early detection and treatment. Therefore, the false-positive result and overdiagnosis should be presented to the patients as potential screening outcomes by providing comprehensive statistical information to decide whether to take the risk of being overdiagnosed in exchange for a chance of being in a group that benefits.

As seen, the major problems occurring in cancer screening are related either to the faults in the screening test, where optimal sensitivity and specificity are lacking (and thus there could be a large number of false-positive and false-negative results), or uncertainty when evaluating the further growth potential of benign and/or slow-growing cancers (overdiagnosis). The most common, but arguably not the most serious burden of any disease screening is occurrence of a false-positive result, which can cause i) negative psychological consequences such as anxiety, depression, changes in overall perception of personal health status, sense of vulnerability, which in its turn could significantly reduce quality of life; ii) negative clinical implications related to unnecessary subsequent diagnostic tests to confirm or reject the diagnosis, which expose the patient to the potential complications of additional interventions, such as biopsy or surgery; iii) negative economic burden for the patient and the healthcare system linked to the costs of the supplementary tests. Concerning overdiagnosis, in addition to the above-mentioned burdens of false-positive test results, patients might also be exposed to lengthy, burdensome, and unnecessary treatments with potentially harmful side effects (see Welch et al 2011; Welch, Passow 2014).

Communication

As seen, probability and statistics can be deceptive if not well understood. Our comprehension of them can easily be affected by cognitive biases and reasoning fallacies. This means that it is a moral duty of oncologists to understand them well because their clinical decisions can be flawed, and this would go against patients’ quality of life. Said differently, not to comprehend correctly probability and statistics is a kind of medical negligence since one should know what is necessary to operate according to the best (probabilistic) knowledge.

There is a second very important point. It is impossible to correctly communicate a probabilistic outcome to a patient if the probability in question is not conceptually well understood. Fortunately, there are several indications in the literature about how to communicate probability and statistics to a patient, and these should be known (e.g. Sutherland et al. 1991; Thorne et al. 2006; Gigerenzer et al. 2007; Wiseman 2010; Lautenbach et al. 2013; Boniolo et al. 2017; Maiga et al. 2018).

There is, therefore, a double moral obligation: one related to understanding probability well to clinically decide correctly; one related to correctly communicating what has been correctly understood. A last point to focus on, especially in the case of screening. The fact that they lead to positive results is beyond doubt, nevertheless, the participant must be aware of what false positives and false negatives are and that there may be cases of overdiagnosis. This information must be given since also this has to do with the moral duties of the oncologists: the patient has the right to be informed, and the oncologists have the duty to inform.

References

Boniolo G et al. 2017. Comprehending and Communicating Statistics in Breast Cancer Screening. Ethical Implications and Potential Solutions. In M. Gadebusch-Bondio, F. Spöring, J.-S. Gordon (eds), Medical Ethics, Prediction and Prognosis: Interdipliplinary Perspectives. New York: Routledge, pp. 30-41.

Boniolo G, Teira D. 2016. The Centrality of Probability. In G. Boniolo, V. Sanchini (eds.), Ethical counselling and medical decision-making in the era of personalized medicine. Heidelberg: Springer, pp. 49-62.

Brodersen J et al. 2014. Overdiagnosis: How cancer screening can turn indolent pathology into illness. APMIS : Acta Pathologica, Microbiologica, et Immunologica Scandinavica. 122, :683–689. Gigerenzer G 2013. Five year survival rates can mislead. BMJ 346, f548.

Gigerenzer G et al. 2007. Helping doctors and patients make sense of health statistics. Psychol Sci Public Interest 8, 53-96.

Lautenbach DM et al. 2013. Communicating genetic risk information for common disorders in the era of genomic medicine. Annual Review of Genomics and Human Genetics 14, 491-513.

Maiga AW et al. 2018. Communication About the Probability of Cancer in Indeterminate Pulmonary Nodules. JAMA Surg. 153, 353–357.

Motulsky H 2017. Intuitive Biostatistics. A Nonmathematical Guide to Statistical Thinking. Oxford: University Press, New York.

Paci E 2012. Summary of the evidence of breast cancer service screening outcomes in Europe and first estimate of the benefit and harm balance sheet. Journal of Medical Screening 19 (Suppl 1), 5–13.

Puliti D et al. 2012. Overdiagnosis in mammographic screening for breast cancer in Europe: a literature review. Journal of Medical Screening 19 (Suppl 1), 42–56.

Sutherland HJ et al. 1991. Communicating probabilistic information to cancer patients: is there 'noise' on the line? Soc Sci Med. 32, 725-731.

Thorne S et al. 2006. Hope and Probability: Patient Perspectives of the Meaning of Numerical Information in Cancer Communication. Qualitative Health Research 16, 318-336.

Wegwarth OI et al. 2012. Do Physicians Understand Cancer Screening Statistics? A National Survey Of Primary Care Physicians In The United States. Ann Intern Med 156, 340-349.

Welch HG et al. 2011. Overdiagnosis. Making People Sick in the Pursuit of Health. Boston: Beacon Press.

Welch HG, Passow HJ. 2014. Quantifying the benefits and harms of screening mammography. JAMA Internal Medicine. 174, 448–54.

Wiseman M 2010. Communicating Genetic Risk Information Within Families: A Review. Familial Cancer 9, 691-703.