I recall an encounter with a late night infomercial while I was in high school.
The CRT TV droned quietly about health. A device, if used properly, could remove “toxins.” It was a foot bath and as the toxins were removed, per the ad, the water darkened. The ad mixed in a bit of fear easily resolved at an affordable price.
Something slowing you down? Headaches? Did you forget to pay that bill? Having trouble getting up in the morning? Back pain? You might need cleansing! Watch as the water changes color. The darker it gets the more toxins you’ve drawn out. Use it every day and keep yourself clear! For only three easy payments of…
I was alarmed. Nobody had ever warned me of toxins, and here was a simple device to remove the problem. Would the water turn dark if I had a go with the device? How many uses would it take to rid me of these pesky toxins?
Nowadays, very expensive medical training is the hammer I used to crack the shell of medicalized nonsense. But how might one evaluate medical claims and avoid scams without such training?
Skepticism
The late great Carl Sagan proposed the Baloney Detection Kit in his best selling book The Demon Haunted World: Science as a Candle in the Dark. I have no chance of doing the book justice in this short essay, but the book is worth a read even 3 decades since publication. Science has no dogma, but scientists certainly do have bias, therefore the rigor of the scientific method can be used to guide healthy skepticism.
Skepticism used in this essay has a couple synonyms: doubt and incredulity. I am not referring to cynicism or distrust. I encourage the reader to believe my sincerity (trust) but take time to ascertain the veracity (doubt). In the vein of the Russian proverb popularized by Ronald Reagan, “trust, but verify.”
Skepticism folds nicely into some of the most effective elements of cognitive therapy. In other words, skepticism leads to questioning one’s own biases. This is a healthy practice. Most people would like to believe truths and reject falsehoods. Skepticism is a core tenant in the process towards veracity. There are an infinite number of incorrect ideas and only a finite number of true ideas, so we need some help to even the odds.
Quacker Detection Kit
Here are some short cuts to identifying dubious medical claims. A great number of medically adjacent fields adopt medical jargon to hijack the authority of science. Though quackery may use authoritative language, it lacks the rigor of the scientific method. Science is a method that has led to a body of knowledge. The method of science is not intuitive. Be careful not to believe your own lying eyes!
Many pseudosciences are low hanging fruit for the skeptic. These are often referred to as “alternative medicine.” The claims made by these fields have been debunked thoroughly and repeatedly.
I will name some of the most common medical quackeries. If you see your personal bias listed then take the opportunity to question what you believe and why:
Acupuncture, chiropractics, homeopathy, chelation therapy, energy healing, cranial-sacral therapy, CAM, traditional Chinese medicine, Ayurveda, herbalism, naturopathy, and psychic advice.
The quacker detection kit helps identify the dubious, spurious, and irrelevant. Sagan had 9 guidelines in his baloney detection kit, but he was more advanced than I. The quacker detection kit has only 4 guidelines:
A treatment that does everything actually does nothing.
A non-specific treatment can never take credit for a positive result.
Treatments must have a failure rate.
Medical advice via advertisement is bad advice.
Let us apply the kit to the advertisement I was exposed to in high school, remember the foot bath ad?
Toxins are responsible for aches and fatigue
Vague toxins are being removed to help one “feel better”
Apply it right and it works every time!
It’s an ad! They will take your money but not take responsibility for providing good medical advice
Junk medical interventions often are associated with grandiose claims, juxtapositional inferences, vague symptoms, perfect success (lest the user be blamed for failure), and are designed to be sold.
Many medical conditions occur with fatigue, headache, dysphoria, and anxiety. The same nonspecific symptoms can also be primary unto themselves. In many cases these vague symptoms come and go spontaneously. In other words, many common ailments get better with rest and time. Adding a nonsense intervention induces a false causation bias.
In my psychiatric practice, if I recommend an intervention such as a medicine, and the patient asks about “alternatives,” then I am going to discuss other interventions that have evidence of effectiveness. I do not suddenly recommend ineffective nonsense.
But what about…
Testimonials are not a reliable means of identifying facts. Testimonials are rife with specious reasoning and embellishment. Just about everyone will project success and disguise loss. Normal human brains constantly betray rationality in favor of ego. Even the most well trained thinkers often think and behave irrationally!
A testimonial might pique curiosity but it need not be compelling. It is okay to believe the sincerity of another’s experience while simultaneously remaining skeptical about their explanation.
Let’s use the foot bath as an example. Suppose I had tried the foot bath and I witnessed the water change color. Being a credulous young man I would have believed the explanation the advertisement supplied: unwanted toxins seeped out from my feet and caused the water to darken. I undoubtedly would have shared such a novel experience, and my testimony might have been persuasive. “Hey I haven’t had a headache in a while!” Hopefully someone would have encouraged me to run the foot bath with various substrates and observe the ionization reaction — an experiment designed to disprove the toxin hypothesis.
Our brains automatically assign truth emotions to our own explanations! Even in the face of being factually incorrect our brains will signal to us that we were secretly correct! It is painful to be wrong so we lie to ourselves constantly. Always remember, being confidently correct and unwittingly incorrect both feel like truth emotions. However, we do not have these emotions when it is someone else’s wrong belief that we already disagree with. It is easy to tear down an idea we do not believe is true.
Unfortunately, demonstrating to another person they are wrong often serves to reinforce their wrong belief. This is related to the phenomena known as cognitive dissonance. Prove to a person they are wrong and witness them rationalize how they were actually correct. I have been wrong many times and so have you. I have experienced cognitive dissonance many times and so have you (let us try to catch ourselves next time).
Remember those common medical quackeries? My mention of those as incorrect did not persuade anyone out of their own bias despite my urging. Some readers felt offended. The homeopaths, for example, will continue to be homeopaths despite this essay because they find the emotional comfort associated with belief in the magic of homeopathy more compelling than the irrational absurdity of homeopathy. Homeopathy performs as well as placebo, which means it is inert. (You demonstrate this with experiment at home by yourself as long as you have a way to disguise which is the homeopathic remedy and which is the placebo).
The placebo effect is a measurable phenomenon. It is present in wonderful ways, such as a kind word, a listening ear, and a gentle touch. The placebo effect is strengthened by the cost, size, and invasiveness of the intervention.
Generally speaking: If an intervention is more effective than placebo then the intervention is effective. If the intervention is equally effective as placebo then the intervention is ineffective, and therefore not science based. No time should be spent on theorizing how ineffective medicine works because it does not “work.”
There is a lot to say about the placebo effect, but for now it is important to reiterate that all alternative medicine subsists upon the placebo effect and nothing more.
Stop picking on alternative medicine, how do we evaluate serious medical claims?
Serious medical claims go through rigorous and expensive testing. Medical interventions generally require a specific disease target, a goal improvement, and quantifiable outcomes. Medical interventions must lead to improvement that is both statistically significant (95% likely due to the intervention) and clinically significant (the improvement is noticeable).
Serious medical interventions must do something meaningful, however perfection is unrealistic. Interventions perform successfully at a rate, in other words they will not work for everyone all the time.
If the serious medical intervention fails it does not mean the patient failed! In a previous post I discussed the concept of the number needed to treat. The NNT refers to a failure rate. All real interventions have failure rates.
Physical pains of all type are responsive to psychological intervention. Serious medical interventions must perform beyond the placebo effect. For example, if a real surgery performs no better than sham surgery, then hesitate to go for the operation — it is ineffective.
Information is freely accessible in this modern Information Age. Success and failure rates of interventions can be accessed by anyone with a web search. Discriminating in favor of Google’s top 3 search results is a reasonable heuristic in 2024 when this post was written. In other words, web searches are good for accessing medical research topics.
Interventions can be compared to either a placebo or other interventions that have already been identified as effective. In your googling, if you come across a “meta-analysis,” then you may have hit the mother lode. A meta-analysis aggregates existing studies by the hundreds into a single study. Newer interventions might not yet be included in a meta-analysis, but often are compared to a known effective intervention.
I do not have a foolproof method for protecting oneself from misinformation. We are all vulnerable to misinformation. Accepting that sometimes we will believe misinformation may open the mind to change when presented with contrary evidence. It is healthy to admit when we make a mistake and when we are wrong: true for patients and doubly true for doctors. Web searches sometimes contain misinformation, so referencing what we read against the knowledge of a physician is a straightforward and open-minded approach to evaluating medical claims.
Set the bar high
Physician recommendations should have scientific evidence. Patients bestow consent to doctors, who in return provide life-changing interventions. “Life-changing” can mean for better or for worse! Science is the guiding light that illuminates the helpful interventions. Useless and harmful advice need be identified and subsequently abandoned.
A doctor ought to provide recommendations that are personal to you based on best available evidence. For example, if your buddy recommends back surgery for back pain, then you can assume that your buddy associates back surgery with relief. However, if your doctor recommends back surgery for back pain then the doctor ought to explain how surgery could be helpful, the risks of operating, and the chances of success.
If the doctor is recommending something because he or she has a pet theory then he or she really ought to say so. Doctor-patient rapport is an ingredient of quality care, however it might be confusing when the doctor blurs the line between rapport building and formal recommendations.
The relationship between an intervention and a health outcome must be better than correlation. Science reveals these effective interventions that are causative rather than correlated. Alternative medicine does no better than the placebo effect.
Nevertheless, the placebo effect is captured when a doctor expresses virtue: kindness, compassion, patience, poise, grace, honesty, humility, and confidence.
Patients of the American healthcare system, I advocate that you trust, but verify.