What doctors dont know about the drugs they prescribe Ben Goldacre

hi uh so um this chap here he thinks he

can tell you the future his name is

Nostradamus although here the son have

made him look a little bit like Sean

Connery and like most of you I suspect I

don’t really believe that people can

sense the future I don’t believe in

precognition and every now and then you

hear that somebody has been able to

predict something that happened in the

future and that’s probably because it

was a fluke and we only hear about the

flukes and about the freaks we don’t

hear about all the times that people got

stuff wrong now he expects that to

happen with silly stories about

precognition but the problem is we have

exactly the same problem in academia and

in medicine and in this environment it

costs lives so firstly thinking just

about precognition as it turns out just

last year a researcher called Daryl BEM

conducted a piece of research where he

found evidence of precognitive powers in

undergraduate students and this was

published in a peer-reviewed academic

journal and most of the people who read

this just said okay well fair enough but

I think that’s a fluke that’s a freak

because I know that if I did a study

where I found no evidence that

undergraduate students had precognitive

powers it probably wouldn’t get

published in a journal and in fact we

know that that’s true

because several different groups of

research scientists tried to replicate

the findings of this precognition study

and when they submitted it to the exact

same journal the journal said no we’re

not interested in publishing replication

we’re not interested in your negative

data so this is already evidence of how

in the academic literature we will see a

biased sample of the true picture of all

of the scientific studies that have been

conducted but it doesn’t just happen in

the dry academic field of psychology it

also happens in for example cancer

research so in March 2012 just one month

ago some researchers reported in the

journal Nature how they had tried to

replicate 53 different basic science

studies looking at potential treatment

targets in cancer and out of those 53

studies they were only able to success

we replicate 647 out of those 53 were

unlockable and they say in their

discussion that this is very likely

because freaks get published people will

do lots and lots and lots of different

studies and the occasions when it works

they will publish in the ones where it

doesn’t work they won’t and their first

recommendation of how to fix this

problem because it is a problem because

it sends us all down blind alleys their

first recommendation of how to fix this

problem is to make it easier to publish

negative results in science and to

change the incentives so that scientists

are encouraged to post more of their

negative results in public but it

doesn’t just happen in the very dry

world of preclinical basic science

cancer research it also happens in the

very real flesh and blood of academic

medicine so in 1980 some researchers did

a study on a drug called Lorcan ID and

this was an anti-arrhythmic drug a drug

that suppresses abnormal heart rhythms

and the idea was after people have had a

heart attack they’re quite likely to

have abnormal heart rhythms so if we

give them a drug that suppresses

abnormal heart rhythms this will

increase the chances of them surviving

early on in its development they did a

very small trial just under a hundred

patients 50 patients got Lorcan ID and

of those patients ten died another 50

patients got a dummy placebo sugar pill

with no active ingredient and only one

of them died so they rightly regarded

this drug as a failure and it’s

commercial development was stopped and

because it’s commercial development was

stopped this trial was never published

unfortunately over the course of the

next 5-10 years other companies had the

same idea about drugs that would prevent

arrhythmias in people who’ve had heart

attacks these drugs were brought to

market they were prescribed very widely

because heart attacks are a very common

thing and it took so long for us to find

out that these drugs also caused an

increased rate of death but before we

detected that safety signal over a

hundred thousand people died

unnecessarily in America from the

prescription of anti-arrhythmic drugs

now actually in 1993

the researchers who did that 1980 study

that early study published a mayor culpa

an apology to the scientific community

in which they said when we carried out

our study in 1980 we thought that the

increased death rate that occurred in

the law an ID group was an effective

chance the development of local ID was

abandoned for commercial reasons and

this study was never published it’s now

a good example of publication bias

that’s the technical term for the

phenomenon where unflattering data gets

gets lost gets unpublished it is left

missing in action and they say the

results described here might have

provided an early warning of trouble

ahead now these are stories from basic

science these are stories from twenty

thirty years ago the academic publishing

environment is very different now there

are academic journals like trials the

open access journal which will publish

any trial conducted in humans regardless

of whether it has a positive or a

negative result but this problem of

negative results that go missing in

action is still very prevalent in fact

it’s so prevalent that it cuts to the

core of evidence-based medicine so this

is a drug called reboxetine and this is

a drunk that I myself have prescribed

it’s an antidepressant and I’m a very

nerdy doctor so I read all of the

studies that I could on this drug I read

the one study that was published that

showed that reboxetine was better than

placebo and I read the other three

studies that were published that showed

that reboxetine was just as good as any

other antidepressant and because this

patient hadn’t done well on those other

antidepressants I thought were robots

teams just as good it’s one to try but

it turned out that I was misled in fact

seven trials were conducted comparing

the Box 13 against a dummy placebo sugar

pill one of them was positive and that

was published but six of them were

negative and they were left unpublished

three trials were published comparing

reboxetine against other antidepressants

in which reboxetine was just as good and

they were published but three times as

many patients worth of data was

collected which showed that reboxetine

was worse than those other treatments

and those trials were not published

I felt misled now you might say well

that’s an extreme and unusual example

and I wouldn’t want to

guilty of the same kind of

cherry-picking and selectively

referencing that I’m accusing other

people of but it turns out that this

phenomenon of publication bias has

actually been very very well studied so

here is one example of how you approach

it the classic model is you get a bunch

of studies where you know that they’ve

been conducted and completed and then

you go and see if they’ve been published

anywhere in the academic literature so

this took all of the trials that have

ever been conducted on antidepressants

that were approved over a 15-year period

by the FDA they took all of the trials

which was submitted to the FDA as part

of the approval package so that’s not

all of the trials that were ever

conducted on these drugs because we can

never know if we have those but it is

the ones that were conducted in order to

get the marketing authorization and then

they went to see if these trials had

been published in the peer-reviewed

academic literature and this is what

they found it was pretty much a 50/50

split half of these trials were positive

half of them were negative in reality

but when they went to look for these

trials in the peer-reviewed academic

literature what they found was a very

different picture only three of the

negative trials were published but all

but one of the positive trials were

published now if we just flick back and

forth between those two you can see what

a staggering difference there was

between reality and what doctors

patients commissioners of health

services and academics were able to see

in the peer-reviewed academic literature

we were misled and this is a systematic

flaw in the core of medicine in fact

there have been so many studies

conducted on publication bias now over a

hundred that they’ve been collected in a

systematic review published in 2010 that

took every single study on publication

that bias but they could find

publication bias affects every field of

medicine about half of all trials on

average go missing in action and we know

that positive findings are around twice

as likely to be published as negative

findings this is a cancer at the core of

evidence-based medicine if I flipped a

coin a hundred times but then withheld

the results from you from half of those

tosses I can make it look as if I had a

coin that always came up heads but that

wouldn’t mean that

a two-headed coin that would mean that I

was a chancer and you were an idiot for

letting me get away with it but this is

exactly what we blindly tolerate in the

whole of evidence-based medicine and to

me this is research misconduct if I

conducted one study and I withheld half

of the data points from that one study

you would rightly accuse me essentially

of research fraud and yet for some

reason if somebody conducts ten studies

but only publishes the five that give

the result that they want we don’t

consider that to be research misconduct

and when that responsibility is diffused

between a whole network of researchers

academics industry sponsors journal

editors for some reason we find it more

acceptable but the effect on patients is

damning and this is happening right now

today this is the drug called Tamiflu

tell if Louise a drug which governments

around the world have spent billions and

billions of dollars on stockpiling and

we’ve stockpiled Tamiflu in panic in the

belief that it will reduce the rate of

complications of influenza complications

is a medical euphemism for pneumonia and

death

now when the Cochrane systematic

reviewers were trying to collect

together all of the data from all of the

trials that had ever been conducted on

whether Tamiflu actually did this or not

they found that several of those trials

were unpublished the results were

unavailable to them and when they

started obtaining the write-ups of those

trials through various different means

through Freedom of Information Act

requests through harassing various

different organizations what they found

was inconsistent and when they tried to

get hold of the clinical study reports

the 10,000 page long documents that have

the that the best possible rendition of

the information they were told they

weren’t allowed to have them and if you

want to read the full correspondence and

the excuses and the explanations given

by the drug company you can see that

written up in this week’s edition of

PLoS Medicine and the most staggering

thing of all of this

to me is that not only is this a problem

not only do we recognize that this is a

problem but we’ve had to suffer fake

fixes we’ve had people pretend that this

is a problem that’s been fixed first of

all we had trials registers and

everybody said oh it’s okay we’ll get

everyone to register their trials

whether they’ll post the protocol

they’ll say what they’re going to do

before they do it

and then afterwards we’ll be able to

check and see if all the trials which

have been conducted and completed have

been published but people didn’t bother

to use those registers and so then the

International Committee of medical

journal editors came along and they said

oh well we will hold the line we weren’t

publish any journals we weren’t

published any trials unless they’ve been

registered before they began but they

didn’t hold the line in 2008 a study was

conducted which showed that half of all

of trials published by journals edited

by members of the ICM je weren’t

properly registered and a quarter of

them weren’t registered at all and then

finally the FDA Amendment Act was passed

a couple of years ago say that everybody

who conducts a trial must post the

results of that trial within one year

and in the BMJ in the first edition of

January 2012 you can see a study which

looks to see if people kept to that

ruling and it turns out that only one in

five have done so this is a disaster we

cannot know the true effects of the

medicines that we prescribe if we do not

have access to all of the information

and this is not a difficult problem to

fix we need to force people to publish

all trials conducted in humans including

the older trance because the FDA

Amendment Act only asks that you publish

the trials conducted after 2008 and I

don’t know what world it is in which

were only practicing medicine on the

basis of trials that completed in the

past two years

we need to publish all trials in humans

including the older trials for all drugs

in current use and you need to tell

everyone you know that this is a problem

and that it has not been fixed thank you

very much

you