One evening in 2009, a patient walked into a pharmacy with a prescription, demanding a drug called Tamiflu. “Everyone I know who has the flu is taking it,” she added. Tamiflu, also known by its generic name, Oseltamivir, was the then-answer to the H1N1 flu pandemic. The drug, a neuraminidase inhibitor, was recommended by the World Health Organization (WHO) for the treatment and prevention of influenza – based on a Cochrane Review.
So what is a Cochrane review?
Usually, as in the case of Tamiflu, many studies are done before a drug hits the market, and for various reasons; no two studies ever give identical results. Sometimes the results may even be conflicting.
A Cochrane review uses specialised methods to summarise the evidences from these studies and attempt to come up with an answer as to whether or not a treatment is effective, and if it is, how effective it is. The method used is called a systematic review, which involves rigorous processes such as identifying all published studies of a particular area of interest, critically appraising their methodologies, selecting quality studies, discussing reasons for discrepancies across the studies and summarising the results.
While many other groups also perform systematic reviews, the Cochrane Collaboration is the leader in the field, known for its reliable and high quality reviews. For a review to be called a “Cochrane review”, it has to be the work of one of the 53 independent Collaborative Review Groups (CRG) registered and maintained by the Cochrane Collaboration. Each of these groups sees to a certain healthcare subject area of its own.
Malaysian Cochrane Network Coordinating Group (from left: Prof Dr Jacqueline Ho; Prof Hans Van Rostenberghe, USM; Dr Nor Asiah, Cochrane Unit, IMR, MOH; Dr Moy Foong Ming, Julius Centre UM; and Prof Adinegara Lufti, Melaka Manipal Medical College).
Cochrane in Malaysia
The Cochrane Collaboration consists of a global network of 37,000 dedicated and volunteer experts from 130 countries, including over 200 experts in Malaysia, who work together to help healthcare practitioners, policymakers, patients and carers make well-informed decisions. Named in honour of Archie Cochrane, an advocate of the use of evidence from randomised controlled trials to guide healthcare decisions, Cochrane is one of the most thorough reviewers of health science data.
Before any study is included in a review, it would have gone through a meticulous evaluation for its quality, e.g. methodological rigour, scientific validity, risk of bias and etc. When the best studies have been selected – usually only a handful – their results may be re-analysed and a balanced summary derived.
Each report also comes with a plain language summary for the layperson. If you were to access the reviews for the layperson, you can find a clear explanation of the review question, some background, what studies have been included in the review along with the durations during which they were published, key results, and a conclusion and information on the quality of the evidence. You can also read the full abstract for more succinct information or the full report if you are up for a statistical punch. Although the reviews are in English, some have been translated to several other languages through efforts of local Cochrane volunteers. Translation into Bahasa Malaysia is underway – as many as 33 review summaries have been translated.
In September 2013, the Malaysian Cochrane Network was officially formed. It is made up of five network sites: Universiti Malaya, Penang Medical College (PMC), Universiti Sains Malaysia (USM), the Research and Technical Support Programme of the Health Ministry, and Melaka Manipal Medical College.
According to Prof Dr Jacqueline Ho, the Malaysian Cochrane Network convener and PMC’s deputy dean of Academic Affairs, the network concentrates on helping people understand the reviews – for example, by translating the summaries into Bahasa Malaysia. The network also focuses on disseminating featured, highly accessed or relevant review information to the public through the press or health agencies, one of which is the use of influenza vaccine for preventing middle ear infections in infants and children. The review, headed by a Malaysian author from USM, found that the influenza vaccine, besides preventing influenza, brings small beneficial effects on middle ear infections.
The consequences of biased studies
In the midst of the H1N1 pandemic, the Australian and UK governments specifically requested the Cochrane group to update its review on the Tamiflu drug, which was first published in 1999 and updated twice, in 2006 and 2008. During the course of the update, the reviewers received a comment from a Japanese paediatrician who pointed out that the conclusions from the 2006 review were questionable as it was mostly based on a single paper – an industry-funded summary of 10 trials – of which only two had been published in scientific literature. This launched the reviewers into a series of investigations which subsequently triggered a course of events that would change the fate of evidence-based medicine forever.
There are perhaps hundreds or thousands of studies done in each medical area with conflicting results, which can make it difficult to form a conclusion. The role of Cochrane is to make sense of all the data and translate them into a useful review that can help doctors and the public. Although Cochrane simplifies what could be, to a doctor, a mind-boggling search for a definitive answer, it is still a difficult task as there are many potential flaws – one of them being publication bias, or an under-reporting of negative or unflattering results.
Prof Dr Jacqueline Ho with Prof Amir Khir, Dean and President of Penang Medical College, at the launch of the Malaysian Cochrane Network in KL.
The effects of publication bias can be devastating. During the infamous 1980 trial on a heart arrhythmia drug called lorcainide, 100 patients were tested – half of whom got the drug while another half got a placebo. Ten in the lorcainide group died compared to one in the placebo group. The trial was stopped and the drug was abandoned, but the results were never published. Over the next decade, drugs similar to lorcainide were developed and marketed to treat arrhythmia. An estimated 100,000 people prescribed these drugs died throughout the 1980s – deaths that could have been prevented had the results from the lorcainide trial been published.
There can be many reasons for this “file-drawer effect”. Publishing medical literature is competitive in nature; positive or astounding results are usually seen as more interesting and more likely to be accepted and published, whereas negative results or findings that show no difference end up getting stashed in drawers. Even if they were published, they usually take a longer time to be released.
Another strong reason for the bias is financial interests, especially in an industry-sponsored trial. Such was the case with the Tamiflu trial; the negative results had been deliberately withheld. It wasn’t until the full data were released to Cochrane that a comprehensive review could be executed. In 2014, after obtaining all the data, it was found that Tamiflu had little or no impact on complications of influenza, but might in fact expose the patient to unwanted side effects.
Countering the biases
Systematic reviews are only useful if the reviewers have access to all the data – the good, the bad and the inconclusive. Cochrane reviewers often try to get unpublished clinical study reports from pharmaceutical companies, but most of the time they only have published studies to compare. If there is significant publication bias, it could further bias the reviews. However, they have developed statistical methods that are useful in helping detect whether or not publication bias is present – and what its effects on the results have been.
There are probably many more cases like this, but since Tamiflu was a poster child, efforts have been put in by several parties – including national health agencies – to increase trial transparency and reduce publication bias. A campaign called AllTrials (www.alltrials.net) was initiated in 2013 to petition for all past and present clinical trials to be registered, and their results reported. The US National Institutes of Health has an online registry (ClinicalTrials.gov) that keeps track of research involving human subjects.
With hope of tracking trials whose results are not published, laws and policies worldwide are increasingly making it compulsory for independent researchers and industries to participate in such registries when they begin their trials, and to update and submit their results when their studies end. With increasing pressure to make trials transparent, ClinicalTrials.gov currently lists 188,947 studies (as of April 23, 2015) from 190 countries – a manifold increase from 5,635 studies back in 2000.
And with publication bias slowly moving out of the way, the rigorous and painstaking reassessment of studies in systematic reviews can draw a more reliable conclusion.
The public can access all the plain language review summaries at http:// summaries.cochrane.org for free. The Malaysian Cochrane Network is looking for volunteers to translate the review summaries to Bahasa Malaysia. Anyone with sufficient language knowledge can do this, but it is particularly useful for students in medical, pharmacy, nursing or other allied health fields. Interested volunteers can contact Ms Nila (nila@ pmc.edu.my).
Ch’ng Chin Chin is a researcher, travel enthusiast and aspiring storyteller.