By Mal Vickers

I’m writing this post because I’m just a tad annoyed. The issue is the over use of the phrase “but it worked for me” (BIW4M). Is anyone else getting tired of that one, too?

Inevitably, I find the phrase crops up in discussion I’ve had with those in favour of a particular alternative medicine such as homeopathy, crystal healing, acupuncture, iridology or chiropractic.  BIW4M is used as the clinching argument.

It’s not driving me nuts, it’s just annoying.  (I’m using ‘BIW4M’ because typing it in full over and over is just going to get me more annoyed.)

If the idea is for skeptics and believers in alternative medicine to come to some kind of understanding, does the phrase help? If I try to argue against BIW4M, it’s seen as a personal attack on the integrity of the person that says it. At least in a blog post I can discuss BIW4M without being  seen to be making such personal accusations, although you’re free to add your slings and arrows in comments below this post.

The Control (as opposed to KAOS)

One of the most powerful and basic ideas in science is that of the use of a control. For example, if I wish to test if water is needed to keep a potted tomato plant alive in a hot house, I could do the following; keep two potted plants in the hot house, one of which I water and one I don’t. The one not watered is the control. I think the likely outcome of this experiment would be clear to everyone; one plant wilts and dies, while the other remains healthy. However, my point is that control experiments are logical and they are used to make valid comparisons.

The complications come if we try to do the same type of control testing with people.

–          Obviously it’s unethical to test a person to death.

–          People are smart enough to know whether they are getting real treatment or not.

–          If otherwise left alone, our bodies (usually) have the ability to heal themselves.

If you argue BIW4M on the assumption that it’s better than a scientific test, I’m entitled to ask…

Do you have an identical twin who leads an identical life (the control), and didn’t get the alternative medicine treatment you advocate? Without that, how do I know you wouldn’t have become better anyway? And how do you know that at least some of your improved health isn’t due to the placebo effect?

These are tough questions. They certainly define me as a skeptic in any discussion with an alt-med proponent. I ask those tough questions only because I think the use of a control is reasonable – essential, in fact. However, asking for such a level of rigour is seen as an insulting attack on the integrity of the person claiming BIW4M. I agree, the level of rigour I’m asking for is exceedingly high. Instead of having to ask defensive questions, I’d rather discuss something more interesting, like ways to test alt-med.

After I’ve asked my questions about the lack of controls, an alt-med proponent might walk away muttering “it’s impossible to convince a skeptic”. However, I think I’m like many skeptics. It is possible to convince me. I’m not an automatic denialist. I do change my mind on questions from time to time. If there is credible evidence that something works, I’ll change my mind.

Here’s how it’s done with alt-med

The best way we have of testing medicines is the “double blind, randomised, placebo control trial” (DBRPCT).  It’s basically a way of taking human influence and opinion out of the equation. I strongly recommend anyone who’s unfamiliar with this method to become acquainted. You can read about it here.

DBRPCT trials are the fairest way to establish the effectiveness of any treatment. It doesn’t matter if it’s a service like chiropractic, a herb or a magnetic device; all can be subjected to a DBRPCT. This kind of trial is so useful that it’s been used in other areas of human enquiry such as education, criminology and agriculture.

The nice thing about a DBRPCT is that it can measure overall health outcomes. It doesn’t matter if you find yourself in a discussion with someone who says their favourite treatment has somehow not been discovered by or isn’t explainable by science.  If the claim is simply “You’ll have better health if you use treatment X”, such a positive outcome will be evident in the results of a DBRPCT.

I’ve also heard alt-med proponents say their favourite treatment can’t be tested, because there is no placebo substitute. However, people have come up with some very clever placebo substitutes. It just requires a little imagination. For example:

–          Acupuncture needles that have a sleeve and only press on the skin, rather than penetrate the skin, are an indistinguishable placebo to the patient.

–          Sham chiropractic can be performed in areas of the body NOT traditionally recommended for the treatment of a particular condition.

–          Machines that supposedly treat by producing magnetic fields can be made to produce the same sounds and lights but have the magnets internally switched off.

–          It’s also possible to test keyhole surgical procedures against a placebo. The patient goes through the same process in every way – except the surgeon just makes some cuts in the skin and then stitches them up. Yes, this has been done.

Rigorously conducted controlled trials are a relatively recent scientific development. James Lind is generally credited with taking the first significant step in the right direction by doing the first medical trial in 1747. Lind’s finding was that citrus fruits prevented scurvy, (although Lind’s results were largely ignored and it wasn’t till 1795 that the British Admiralty set up the regular issue of lemon or lime juice to the whole fleet). Hundreds of sailors died unnecessarily in the intervening years.

Controlled trials have slowly been refined as a scientific method as pitfalls have been found. The first controlled trial to include “randomising” was published in 1948. The use of all possible measures to make the testing of a medical claim fair and objective is a relatively recent development. These include blinding, randomising and placebo control.

I’ll also make a slightly subtle point that I hope won’t be misinterpreted. “Double blind, placebo controlled trials” aren’t necessarily always used in science – they’re expensive to conduct and you need lots of people to help find statistically significant results. Science is more open than that. It’s possible to publish a controlled trial with few subjects and single blinding. It just won’t be as highly regarded as evidence when compared with more rigorously conducted trials.

When addressing health risks like passive smoking, it’s more ethical to do a different kind of study called a cohort study.

This whole area of the design and of the advantages and disadvantages of different types of scientific trials is fascinating and complicated; it’s too much for this little blog post.

When someone says, “BIW4M”, well… perhaps it did. Cause and effect are not necessarily linked; unrelated things can happen coincidentally. Anecdotes can be an indicator that proper scientific testing needs to be done, if it hasn’t already been done. However, it’s hard to take the BIW4M claim seriously, when fair testing has been done and the results suggest that the treatment is no better than placebo.

“What’s the big deal?”, I hear you say.

The BIW4M phrase is basically an anecdote. And whilst everyone, including me, loves a good story, many people have died believing they could be cured by a particular treatment based on only anecdotal evidence. Testimonials are anecdotes, and the testimonial is by far the most common marketing tool of the alt-med industry.

“Snake oil” medicine has been with us a long time and it’s still a problem. I think I’m entitled to get annoyed by it. Some people get rich, while other people die too soon, distracted by false hope – that annoys me.

The “What’s the Harm?” website is well worth a visit if you believe alt-med is harmless. It documents the stories of those it didn’t work for. Many cases of harm and death are listed, basically because people believed anecdotes.

I’m not suggesting Science has all the answers and I’m not suggesting modern medical practice is perfect; but at least modern medical practice attempts to include the results of objective testing.

It’s my health and your health that’s under discussion. Isn’t it important to get right?  Are anecdotes good enough?