Newsflash! Advertisers Lie!

Yes, I know it is almost impossible to fathom such depravity, but some pharmaceutical advertisers make unsubstantiated claims in medical journals! (/SARCASM OFF) Such is the conclusion of Ben Goldacre of the Guardian, writing on the Bad Science Blog. Goldacre is writing about a recent research article in the Netherlands Journal of Medicine “Are claims of advertisements in medical journals supported by RCTs?“. The article comes to the conclusion that only 40% of the adverts quoted a high-quality RCT (Randomised Control Trial) that actually supported the claims made in the advert (some trials were high quality but did not support the claims). Scarily, only 17% of the claims were supported by a relevant, good quality RCT that was not sponsored by the pharmaceutical company.

The study had 250 medical students go through 158 RCTs from 94 advertisements using a modified version of the Chalmers’ score. Why medical students? Goldacre suggests amusingly that they are cheap! But the real reason is that the students had just completed the section of their study that deals with appraising evidence-based medicine. They were therefore about as able to judge the trials as an average GP, perhaps more so. They were given an objective scoring system to follow and they had no previous exposure to prescribing the drugs being advertised. This would have given an accurate, unbiased assessment of the worth of these trials without resorting to the use of statisticians or academics. This gives a good “real world” picture of the value of these trials, should a GP or hospital doctor have asked for them.

It comes as no real surprise that so few of the advertisements had decent clinical data to back them up. This has been a common finding in many studies including this large swiss study of 2068 adverts. Goldacre cites this excellent meta-analysis of 24 studies on this subject from the open access journal Plos One. Even as far back as 1992, the highly regarded Annals of Internal Medicine published this study whose conclusions included:

“In 44% of cases, reviewers felt that the advertisement would lead to improper prescribing if a physician had no other information about the drug other than that contained in the advertisement.”

The moral of the story being that when the drug rep or an advert makes a claim, always insist on reading the cited study. I have done this from the time of my graduation, nearly thirty years ago, and my impression is much the same as these studies. About half of the claims made by reps are verifiable. To be fair on the drug reps, most of them do not know how to judge a study and are just going by what they have been told. And they are almost always perfectly willing to find you a copy of the paper.

From an ethical point of view, I cannot see how any doctor could change his prescribing habits without at least assessing the claims made by pharmaceutical companies, formally. I know many of my colleagues take the recommendations of the specialists they use, relying on their judgement as to the worth of the product. I think this is not an unreasonable thing to do, as most specialists are more able to assess these claims than the average GP. However, I still make it a policy to assess papers myself, as even my specialist colleagues may not be immune to the lure of the shiny new pill on the market.

So what makes a paper a bad one? Essentially, bias in selection, lack of controls and low numbers. The first two are quite easy to spot once you know a little about the subject, the last is the commonest reason for considering a trial dubious. I can’t tell you how many times I have seen adverts claiming a great “P” value of 0.001 (That’s Probability, not Methamphetamine, BTW – anything less than 0.05 suggests that this is not a random chance). When you ask how many were in the study, you keep getting answers like 30 or 50. This means that it is dangerously likely that the result may simply be a type 1 error (the small sample has accidentally been taken from an abnormal part of the population – giving a spuriously significant result). This is a particular problem in medical trials. This is not to say that small trials are not worth doing, but that the result should always be treated with caution. Multiple small trials with the same result are much more reassuring, as is a large trial with a similar result.

Clearly adverts effect the way doctors prescribe drugs, otherwise drug companies would not run them. This is worrying considering that half of the claims made are either unverified or blatantly false. Prescribing medicine should be done on a purely evidence-driven basis. Prescriptions based on what pharmaceutical companies want you to believe, leave doctors open to charges that there is no more scientific basis to conventional medicine than to something like homeopathy. Worse, it could seriously disservice a patient, all in the name of the pharmaceutical company making another buck.

Doctors owe it to their patients to check out pharmaceutical company claims in full. It is stupid to accept the word of a person who has a vested financial interest in his product and it is nonsense to get our medical knowledge from the glossy brochures of drug reps.

Hat Tip:  David Whyte

Related posts: