Updated: 3 Dec. 2013
This is the fourth of an ongoing series about idiopathic pulmonary fibrosis (IPF) and the refusal of Canadian provinces to fund the only known effective drug to treat it, pirfenidone (Esbriet).This blogs title comes from a song that George Harrison wrote for the Beatles in 1968.
As noted in the prior blog, I'll try to explain why Canada and the UK made diametrically opposed decisions based on the same data.
To recap, the Canadian Drug Expert Committee (CDEC), under the auspices of CADTH, decided that Canada should not fund Esbriet (18 April, 2013).
But in a report also published in April, 2013 the UK's National Institute for Health and Care Excellence (NICE) recommended public funding of Esbriet.
NICE assessed that Esbriet provided a modest but significant improvement in IPF and "recommended pirfenidone because the benefit to patients justifies the cost."
NICE also recommended pirfenidone be discontinued if there is evidence of disease progression (a decline in per cent predicted FVC of 10% or more within any 12 months). So if Esbriet was shown not to work for an individual, it would not be funded for that person.
The NICE Report is 64 pages and discusses the evidence for and against Esbriet in detail using the principles of evidence-based medicine. NICE also gives supplementary information. See Further Reading below.
In contrast, CDEC presents a 5-page summary of why they said no.
Judge for yourselves which review is more credible. Perhaps unfair, but NICE is more transparent. With CDEC are we just supposed to accept that they know best, accept the evidence and conclusions in the summary, and not worry about explicit details of their in-depth thinking?
This blog will focus on two key issues:
1. Efficacy - contradictory studies
2. Cost - contradictory interpretations
Like each one of us, I interpret findings through the filter of my education, background, and biases. See original reports to assess for yourself.
I've chosen selective data to minimize info overload and eyes glazing over. Although findings are simplified, I’ve tried to retain their essence. 'Panel' refers to either CDEC or NICE's expert committees.
EXECUTIVE VERSION
For those who prefer the bottom line as the first line, my thesis is that Canada’s experts put cost first and chose to see the cup as half empty, whereas UK’s experts put patients first and saw the cup as half full.
Who decided and how?
The UK NICE process is provided with specific details of who, what, how, and when.
NICE assessed multiple sources of information and opinion, including clinical specialists, systematic review of scientific studies on pirfenidone, governments, professional medical and nursing associations, representatives designated by patient groups, and the manufacturer of pirfenidone.
Canada's CDEC used its Common Drug Review process. Information sources included a systematic review of the clinical trials on pirfenidone, a critique of the manufacturers pharmacoeconomic evaluation, information submitted by patient groups on issues important to them, and the opinions of CDEC members.
Criteria used to decide
Both groups based decisions on three main criteria:
- Efficacy (inc. FVC, quality of life and mortality rates)
- Cost and cost effectiveness
- Safety and tolerability
1. EFFICACY (Does Esbriet work?)
In medicine, efficacy means clinical effectiveness. Does a drug produce a measurable effect that is meaningful and beneficial to patients?
To measure the effectiveness of Esbriet, the panels looked at the same two RCTs* of adults with mild-to-moderate IPF in 72-week, double-blind, multicentre trials with similar protocols:
CAPACITY-2 (PIPF-004 in UK)
CAPACITY-1 (PIPF-006 in UK)
*RCT The randomized clinical trial (RCT) is the gold standard of medical research. Similar individuals are randomly assigned to a treatment group (who take the drug) and a control group (who take a placebo). Double blind means that patients and researchers do not know who is in which group. Then, if there is a difference (patients on the drug improve more than those on placebo), it is more likely to be real.The primary outcome in both studies was the change in the percentage of predicted FVC* from baseline to week 72.
*FVC. Forced vital capacity (FVC) is the volume of air that can forcibly be blown out after a deep breath in and is universally acknowledged as a reliable measure of lung function. A decline in FVC indicates disease progression. A10% decline within 6-12 months is associated with a significant increase in mortality.In one study (CAPACITY-2) there was a statistically significant* improvement in the rate of decline in per cent predicted FVC with pirfenidone compared with the placebo (4.4% difference), but not the other study (0.6% difference).
* Statistically significant. A 'statistically significant difference' means the results are unlikely to be due to chance and are more likely to be real.Neither Canadian nor UK assessors could explain why the results of the trials were contradictory.
A pooled analysis (possible because the protocols and methodology in both studies were similar), showed a statistically significant improvement in the rate of decline in lung function (per cent predicted FVC) with pirfenidone compared with placebo of 2.5%.
In other words, if data from both studies were analysed as if in one large study, the results are statistically significant (unlikely to be due to chance) and favour Esbriet, as does the one clinical trial.
Canada v UK #1 (Efficacy and FVC)
Canada's experts noted that, if the 2 studies were contradictory, pooled results must be regarded as ‘exploratory’, i.e., not as valid.
In contrast, UK experts noted the near-identical design of both studies and agreed that analysing pooled data, although exploratory, was reasonable.
They concluded that the trials provided evidence that was adequate for assessing the clinical effectiveness of Esbriet. In one study and using pooled data, the drug showed a modest but significant improvement over a short duration (72 weeks).
Canada v UK #2 (Efficacy and Mortality)
Both studies showed no statistically significant difference in IPF-related deaths.
However, a pooled analysis of IPF-related mortality suggested that pirfenidone was associated with a statistically significantly higher probability of survival compared with placebo.
Canada's experts noted that, in both studies, Esbriet showed no difference in prolonging life. Results of pooled data were invalid. Hence, there was insufficient evidence to determine if Esbriet prolonged life.
In contrast, UK experts noted that it was unlikely that clinical trials for IPF treatments can ever have enough statistical power* to detect a difference in mortality. Because of the relatively small number of IPF patients in the studies, there aren't enough patients dying over the short duration of the two clinical trials (only 72 weeks) to detect a difference.
UK experts recognized this limitation as one that would probably always exist in clinical trials of drugs to treat IPF. Although a possible survival benefit was uncertain, the UK decided this was the only evidence available for decision-making.
Canada's experts chose to dismiss the evidence, apparently ignoring the issue of statistical power.
* Statistical power: Ability of a study to detect a real difference, if one exists. Power is affected by how big the difference is and sample size. If a difference is big, it's easier to detect. And large sample sizes make a real difference easier to detect.Canada v UK #3 (Efficacy and Quality of Life)
Here the two panels agreed. The studies showed insufficient evidence that pirfenidone provides clinical benefit for quality of life. Quality of life was measured with a 50-item questionnaire that measures distress due to respiratory symptoms, mobility and physical activity, and the psychosocial impact of the disease.
Both panels found that in the studies Esbriet did not affect related health indicators such as dyspnea (shortness of breath), respiratory-related hospitalisation, need for supplementary oxygen, etc.
2. COST AND COST-EFFECTIVENESS
In both Canadian and UK assessments, the manufacturer submitted a confidential estimate of costs, which were analysed by the panels’ experts.
How medical and public health experts decide cost effectiveness is complex and one of life’s great mysteries. Voodoo economics is the term that comes to mind. The key terms are QALY, ICER / ICUR. (Stick with me. It won’t be too painful.)
QALY
QALY is a well recognized method that indicates how many extra years of life - of a reasonable quality - a person might gain from treatment. QALYs offer many opportunities for researchers to influence results, including what constitutes a reasonable quality of life. NICE explained its QALY calculation.
ICER / ICUR
ICER: The incremental cost-effectiveness ratio is the ratio between the difference in costs and the difference in benefits of two interventions per QALY.
ICUR: Incremental cost-utility ratio is similar to ICER and is calculated as follows (where A & B are interventions, treatments, drugs, etc.):
Cost of A – Cost of B
---------------------------------------------------------------
No. of QALYs produced by A –those produced by B
Canada vs UK #4
CDEC says the manufacturer reported that pirfenidone compared with placebo is associated with an ICUR of $143,617 per QALY. They found faults with the manufacturer’s cost analysis and thought true costs would be higher.
In contrast, the UK calculated its ICER by comparing Ebriet NOT to placebo but to ‘best supportive care’. The manufacturer's probabilistic ICER was £24,000 ($40,349 CDN) per QALY gained, which was less than the UK’s Evidence Review Group's assessment but ‘an acceptable cost-effectiveness estimate on which to begin to explore further considerations’.
So….assuming ICER and ICUR are the same, Canada’s cost effectiveness model concludes Esbriet is more than 3 times the cost of the UK’s. Huh?
Did Canada’s ICUR compare Esbriet to placebo but not consider actual treatment costs that IPF patients would incur if not on the drug?
Did Esbriet's manufacturer give different cost estimates to Canada vs the UK?
PERSPECTIVE on COSTS
Cost-effectiveness thresholds per QALY vary from country to country:
- UK: £20,000 - £30,000 (NICE)
- USA: $50,000
- Australia: $52,400
- Canada: $20,000 - $100,000
BOTTOM LINE
CANADA
Canada's experts judged the data on Esbriet’s efficacy and costs to be inconsistent, untrustworthy, and insufficient, or as Brit football fans might say about an opposing club, ‘They’re shite, total shite!’
Yet CDEC allowed that, although individual patients may benefit from pirfenidone, they were unable to determine the clinical criteria that would accurately identify these patients in clinical practice.
Can’t identify who may benefit? Then no one gets it.
CDEC took the approach that any evidence that supported Esbriet was iffy. They decided to interpret all study results as the cup half empty. One can only wonder why.
UK
NICE’s approach was the cup is half full. UK experts accepted the positive RCT results of the one study and the pooled study results of both. They acknowledged that, because of small numbers of patients with IPF on Esbriet, it was unlikely that studies could ever show a different in mortality rates. They found Esbriet to be cost effective compared to best supportive care and their threshold for QALY costs.
Can’t identify who may benefit? Then stop funding Esbreit for a patient who shows a significant decline in lung function over 12 months.
So what does all this mean?
To me, it's similar to Canada's blood experts in the 1980s deciding to use stocks of FVIII concentrate for hemophiliacs because they were probably all infected, which turned out not to be true. And what a pompous, cold-hearted decision to make with the lives of others.
It’s also analogous to Canada’s blood experts deciding that surrogate tests implemented in the USA for non-A, non-B hepatitis (now hepatitis C) were not scientific enough for them. Inadequate sensitivity and specificity to use the lingo of diagnostic tests.
In retrospect these expert judgements were huge miscalculations and cost the lives of many Canadians and suffering to thousands of others.
Yes, I know you cannot compare Canada's so-called 'tainted blood tragedy' to its decision not to fund IPF's only known treatment, Esbriet. But some aspects are similar.
Canada’s blood experts put cost first then. And I suspect that CDEC has put costs first in the case of Esbriet to treat IPF. Its committee members would never admit it, even to themselves. But influence is subtle. Today, cost saving in Canada's health system is the pervasive culture. CDEC's decision no doubt pleases provincial governments.
In contrast UK experts decided to put patients first. The Esbriet studies with all their limitations - especially because of the short 72 week duration and not enough patient numbers and time to show if Esbriet prolongs life - still show the drug provides a modest and real improvement.
Some might call that balancing cost with patient-centred care, a principle that Canada’s medical community gives lip service to, but seldom implements.
FOR FUN
'While My Guitar Gently Weeps' epitomizes how I feel about Canada’s decision not to fund Esbriet. The CDEC decision revives memories of earlier cost-based decisions, lacks compassion, and I suspect lets an inflated sense of scientific superiority blind our experts to the big picture.
For interest, Harrison song was ranked #7 on Rolling Stone's list of the 100 Greatest Guitar Songs of All Time, and #10 on their list of The Beatles 100 Greatest Songs.
- While my guitar gently weeps (George Harrison with Eric Clapton on guitar and other celebs)
I look at the world and I notice it's turningFURTHER READING
While my guitar gently weeps.
With every mistake we must surely be learning
Still my guitar gently weeps.
NICE Resources
As always, comments are most welcome.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.