Economic Evaluation of Medical Devices
- Michael Drummond, Michael DrummondCentre for Health Economics, University of York
- Rosanna TarriconeRosanna TarriconeSDA Bocconi School of Management, Bocconi University
- and Aleksandra TorbicaAleksandra TorbicaSDA Bocconi School of Management, Bocconi University
There are a number of challenges in the economic evaluation of medical devices (MDs). They are typically less regulated than pharmaceuticals, and the clinical evidence requirements for market authorization are generally lower. There are also specific characteristics of MDs, such as the device–user interaction (learning curve), the incremental nature of innovation, the dynamic nature of pricing, and the broader organizational impact. Therefore, a number of initiatives need to be taken in order to facilitate the economic evaluation of MDs. First, the regulatory processes for MDs need to be strengthened and more closely aligned to the needs of economic evaluation. Second, the methods of economic evaluation need to be enhanced by improving the analysis of the available clinical data, establishing high-quality clinical registries, and better recognizing MDs’ specific characteristics. Third, the market entry and diffusion of MDs need to be better managed by understanding the key influences on MD diffusion and linking diffusion with cost-effectiveness evidence through the use of performance-based risk-sharing arrangements.
Medical devices (MDs) represent an important group of health technologies. In 2011 it was estimated that there were more than 200,000 MDs on the European market (Fraser, Krucoff, Brindis, Komajda, & Smith, 2011). However, MDs represent a very heterogeneous family of technologies consisting of both diagnostic and therapeutic devices and ranging from “low-risk” devices, such as scalpels, to “high-risk” devices, such as cardiac implantable electrical devices. While some devices require only very simplified assessment, others need to be assessed through a full evaluation of safety, efficacy, effectiveness, and cost-effectiveness.
Many of the early applications of economic evaluation in healthcare were related to the use of medical procedures involving devices, such as kidney dialysis (Klarman, Francis, & Rosenthal, 1968), or of expensive diagnostic devices, such as computed tomography (CT) scanners (Office of Technology Assessment, 1978). However, more recent literature in economic evaluation has focused largely on pharmaceuticals, reflecting the growth of formal requirements for new drugs to demonstrate cost-effectiveness in order to be reimbursed in many jurisdictions (Drummond, 2013). For example, in 2016 the National Institute of Health and Care Excellence (NICE) in the United Kingdom, one of the most active health technology assessment organizations worldwide, published 50 technology appraisals of pharmaceuticals, but only 3 appraisals of medical devices and 6 of diagnostic technologies.
This relative lack of economic evaluations of MDs, as compared with pharmaceuticals, has led some authors to question whether devices have any special characteristics that inhibit their assessment and whether, over time, the method guidelines for economic evaluation have become more suited to the assessment of pharmaceuticals (Tarricone, Torbica, & Drummond, 2017a). The challenges in conducting economic evaluation of medical devices are related to two major issues. The first lies in the policy domain and regards current regulatory requirements, which are typically less stringent than those for pharmaceuticals. As a consequence, evidence produced by manufacturers applying for marketing approval is often not sufficient for subsequent economic evaluation analysis. The second issue is purely methodological and reflects the ongoing debate on whether devices have distinctive features that require different or modified methods for economic evaluation to be applied properly. This article discusses the challenges in conducting economic evaluations of MDs from both policy and methodological viewpoints and provides some suggestions on how they might be addressed.
Regulatory Requirements for MDs
One critical issue relates to the fact the regulatory requirements for devices are typically less stringent than those for pharmaceuticals. In order to obtain approval to market, pharmaceuticals normally require two adequately controlled clinical trials. In contrast, the level of clinical evidence required for the marketing of MDs varies by the risk classification of the device. The risk classifications differ slightly by jurisdiction but generally divide devices into three or four groups (Tarricone, Ferré, Torbica, & Drummond, 2014).
Low-risk devices, such as crutches and scalpels, are generally exempt from premarket approval, although they need to be registered and are subject to labeling regulations. In the European Union this means obtaining a CE (Conformité Européenne) mark. Devices of medium risk, such as catheters and infusion pumps, do require premarket approval, and often the manufacturer is required to submit a dossier of relevant clinical and nonclinical data. The data required usually depend on the level of perceived risk. For example, in some jurisdictions (e.g., the European Union), orthopedic implants are classified as “medium risk” but manufacturers would be expected to provide some clinical data and also establish a registry so that the performance of the device could be monitored after it enters the market.
Finally, high-risk devices such as coronary stents, cardiac pacemakers, and implantable defibrillators always require clinical data, although the nature of that data may differ by jurisdiction. In the United States, the Food and Drug Administration often requires a randomized controlled trial (RCT), similar to its requirement for pharmaceuticals, whereas in the EU it may be possible to obtain market approval by conducting an observational study (e.g., a clinical registry). This was the case for transcatheter aortic valve implantation (TAVI), where an RCT was conducted in the United States but an observational study was established in Europe (Tarricone & Drummond, 2011).
Another feature of the regulation of MDs that differs from pharmaceuticals is that manufacturers can claim that their device is “substantially equivalent” to another marketed device for which clinical data are available. An example of this approach is the 510(k) process in the United States (Sorenson & Drummond, 2014). This can greatly assist some manufacturers (known as “fast followers”) but has the effect of not allowing “data exclusivity” to the manufacturer of the original device, as is the case for pharmaceuticals. This can have the effect of reducing the incentives for manufacturers to invest in clinical research, if that research is not mandatory, since the data generated can also be used by their competitors.
A major impact of the current processes for regulating devices is that there is a relative lack of clinical data at the time of product launch, although there remains the possibility of further clinical data collection while the device is in regular clinical use. This limits the possibilities for conducting economic evaluations early in the life cycle of the technology, especially at the time of product launch. Therefore, it has been argued that a different strategy may have to be followed for conducting economic evaluations of MDs, focusing more on assessments in the postmarketing phase (Tarricone et al., 2014). However, conducting economic evaluations in real-life settings poses methodological challenges in the analysis of observational data, which are subject to numerous biases. Furthermore, evidence synthesis becomes even more demanding when data need to be synthesized across studies with different designs (Schnell-Inderst et al., 2017).
Special Characteristics of MDs
It has been suggested that a number of special characteristics of MDs pose additional challenges for their clinical and economic evaluation (Drummond, Griffin, & Tarricone, 2009). These include the fact that (1) there is often an interaction between the device and the user (e.g., a surgeon), meaning that the performance of the device may change over time (the so-called “learning curve”), as a given user gains more experience, (2) MDs are characterized by incremental innovation, where small improvements are made, either through modifications to an existing device or the introduction of new products, (3) the introduction of MDs can have a substantial organizational impact by requiring training or investments in infrastructure, and (4) the pricing of MDs is more dynamic than that for other health technologies because of the extensive use of competitive tendering in the procurement process. In particular, the learning curve may have substantial effects on both effectiveness and costs. A more experienced surgeon may obtain better clinical results than one performing a procedure for the first time. In turn, better clinical outcomes might reduce the total length of hospital stay or the need for subsequent procedures, which could also have an impact on costs. Also, a more skilled surgeon may perform the initial procedure more quickly, which could reduce the amount of expensive operating theatre time required.
There is some evidence to support these assertions. For example, the learning curve for MDs has been observed in several clinical trials. In the CLASICC Trial of laparoscopic-assisted colorectal surgery, the proportion of patients requiring “revisions” (i.e., repeat procedures) fell from 38% in the first year of the trial to 16% in year six (Guillou et al., 2005). Similarly, in a clinical trial of spinal cord stimulation versus percutaneous myocardial laser revascularization for patients with angina, improvements in the outcome from use of the new procedure caused the estimated incremental cost-effectiveness ratio (ICER) to fall from £230,000 per quality-adjusted life-year (QALY) in 2000/1 to £18,000 per QALY in 2002/3 (Dyer et al., 2008). Learning curve effects have also been observed in the use of procedures in regular clinical practice. Using individual patient level data obtained from a hospital discharges database in Germany over the period 2006–2013, Varabyova, Blankert, and Schreyögg (2017) found that, over time, both the in-hospital mortality and length of stay became lower for patients undergoing endovascular aneurysm repair (EVAR), implying that the impact of the learning curve on cost-effectiveness can be more substantial than that on effectiveness alone. Conversely, there was no similar learning effect for a related procedure, fenestrated endovascular repair (fEVAR), possibly because much of the “learning” from performing EVAR procedures was transferable to fEVAR.
The issue of incremental innovation was raised in a clinical and economic evaluation of stapled hemorrhoidectomy conducted for the National Institute for Health and Care Excellence (NICE) in the United Kingdom (Burch et al., 2008). The researchers found that, although clinical trials comparing staple guns versus conventional surgery did exist, most of the trials were conducted using older versions of the gun, which had undergone several modifications. However, it could be argued that as long as the modifications represented improvements to the device’s performance and the learning curve in using the new version of the gun was not substantial, it would be conservative to use the older clinical evidence in the economic evaluation. Nevertheless, in some cases, using clinical data from an older version of a technology could have a substantial impact on the results of the economic evaluation.
Organizational impacts have been observed but not examined quantitatively. For example, Tarricone and Drummond (2011) discussed the fact that adoption of TAVI in a given hospital would normally involve investments in infrastructure and training. Because of the complexity and high risk of patients, TAVI would ideally need to be performed in a hybrid setting with equipment and facilities including both operating rooms and catheterization laboratories (cathlabs). Alternatively, it could be delivered either in operating rooms with adequate visualization equipment or in sterile cathlabs. This remains a challenge today, since “pure” hybrid operating rooms or appropriately equipped operating rooms and cathlabs are the norm only in the most advanced hospitals and would need to be developed further if TAVI were to be more widely diffused. Logistics and multidisciplinary team approaches also represent an organizational issue for many hospitals and must be assessed against investment costs in training and equipment. Moreover, a minimum number of cases needs to be planned in order to secure the return on the investment and, more importantly, the sufficient level of performance.
The impact of price changes on the cost-effectiveness of MDs was illustrated in the technology appraisals conducted on drug-eluting stents (DES) by NICE in the United Kingdom. In the initial appraisal, conducted in 2004, NICE found that DES were cost-effective (according to its threshold of £20,000 per QALY gained) for patients with a high risk of restenosis. However, when NICE reappraised DES in 2008, the price of bare metal stents (the alternative technology) had fallen substantially, pushing the estimated incremental cost-effectiveness ratio (ICER) above NICE’s threshold (Drummond et al., 2009).
Taken together, these special characteristics can influence the planning and analysis of cost-effectiveness studies. For example, the potential existence of a learning curve suggests that the cost-effectiveness of MDs should be studied over a period of time, as the results when the device is first introduced may be misleading. The potential existence of incremental innovation is a challenge for the clinical component of the assessment, as (in principle) separate clinical studies may need to be conducted for each version of the device. In addition, if the changes in the benefits to patients from using the new version of a device are small, these may not be reflected in changes in the generic instruments used for estimating the QALYs gained. Therefore, analysts wanting to estimate the value of these changes may need to estimate individuals’ willingness to pay or conduct a discrete choice experiment (Wilkinson & Drummond, 2015).
The potential for organizational impacts of devices suggests that the perspective for costing needs to be broad, covering a wide range of hospital departments and possibly also primary care, if use of the device changes the pattern of care. In addition, implementation costs, such as training and the provision of additional support services, may also need to be considered. Finally, the possibility of dynamic pricing suggests that a sensitivity analysis should be performed on both the acquisition cost of the new device and the one that it is replacing.
A more fundamental point relating to the special characteristics of MDs is that, as compared with drugs, the real-world cost-effectiveness may differ substantially from that observed within the context of controlled clinical trials. This suggests that the conduct of economic evaluations might be better focused on the postmarketing rather than the premarketing phase, as in the case of pharmaceuticals.
On the other hand, it has been argued that, to the extent that these differences exist between MDs and other technologies (in particular pharmaceuticals), the differences are only a matter of degree (Taylor & Iglesias, 2009). For example, over time physicians may learn how to prescribe a drug more precisely in order to obtain the optimum balance between efficacy and adverse events, and new modes of administration of a given drug (e.g., oral instead of intravenous infusion) might be viewed as incremental innovations or may lead to shifts in care from an inpatient to an outpatient setting with the associated changes in infrastructure. Furthermore, tendering or price competition applies to pharmaceuticals in some jurisdictions (e.g., US managed care), and there are tendering processes for vaccines in many jurisdictions. Nevertheless, the special characteristics mentioned above are likely to have a greater impact in the economic evaluation of devices than other health technologies, so they need to be considered.
Another important characteristic is that many MDs, such as CT scanners, are diagnostic. The economic evaluation of these technologies presents three specific challenges. First, the major benefit arising from diagnostic technologies is their ability to improve treatment choices and hence final health outcomes. Therefore, an accurate diagnostic test is only as good as the treatment that follows it. This means that the economic evaluation of a diagnostic technology usually encompasses the evaluation of the associated treatment technologies, with all the inherent challenges of conducting the required clinical evaluations.
Second, although the major economic benefit of diagnostic technologies results from the improvement in treatment outcomes, there may be value in just knowing the accurate diagnosis, both to the patient and the physician, even if this does not affect the immediate treatment choices. In some cases there may be value from the reassurance offered by a diagnostic test. Although these aspects of value could, in principle, be detected by the standard measures of outcome used in economic evaluations, such as the QALY, on occasions these measures may lack the adequate sensitivity. Therefore, it might be necessary to use alternative measures of value, such as willingness to pay (Lin, Cangelosi, Lee, & Neumann, 2013).
Third, there are indivisibilities in the use of large diagnostic MDs such as scanners. This means that not only is it necessary to show individual indications for use of the scanner, but also it is necessary that the totality of the workload required to occupy the scanner results in a cost-effective use of resources. If there were only a few potentially cost-effective indications for the scanner, this would not justify the large capital expenditure.
Initiatives to Improve the Economic Evaluation of MDs
Improving the Clinical Evidence Base for MDs
As mentioned above, the paucity of clinical data, particularly prior to market entry of MDs, can limit the possibilities for economic evaluation and health technology assessment (HTA). Drummond, Tarricone, and Torbica (2016) discussed several options for resolving this problem. First, the requirements for conducting clinical studies in the premarketing phase could be strengthened by, for example, insisting on RCTs for all high-risk devices and some medium-risk devices (e.g., orthopedic implants). However, this would impose greater costs on manufacturers and delay access to new treatments (Shuren & Califf, 2016). These considerations need to be balanced by the fact that potentially unsafe or ineffective devices could be allowed on the market before adequate evidence is available (Food and Drug Administration, 2016).
Another initiative would be to tighten the conditions under which a manufacturer can claim “substantial equivalence” to a pre-existing device already on the market. This would go some way toward restoring the incentives for research that follow from maintaining data exclusivity. However, this could result in a waste of resources in conducting clinical studies that are not necessary. Also, since some of the “fast-followers” are small and medium-size enterprises, which might find the cost of conducting clinical studies a challenge, this policy would have the impact of reducing price competition, thereby denying the healthcare system some efficiency gains.
The third possibility would be to strengthen the arrangements for postmarket research. One argument for this policy would be that, due to some of the special characteristics of MDs mentioned above, requiring more RCTs in the premarket phase might not be very helpful. For example, if there is likely to be a substantial learning curve for the device in question, it would be preferable to base the economic evaluation on clinical effectiveness in regular use rather than under the experimental conditions of an RCT. Also, it there are likely to be several modifications of the product in the early years of its use, the clinical data gathered in the trial may quickly become outdated.
There are several disadvantages of delaying economic evaluation to the postmarket phase. It is often more difficult, and costly, to remove a treatment or technology from the market once it has entered the healthcare system. Also, once the technology is routinely available, it is more difficult, if not impossible, to conduct a RCT, since this implies that 50% of patients participating in the trial would not receive the new treatment. However, ways might be found to overcome this by, for example, randomizing one group of patients to delayed treatment rather than no treatment.
These difficulties mean that the clinical studies conducted post-launch are likely to be observational studies, which often suffer from selection bias and are therefore more likely to produce a biased assessment of relative treatment effect (Grieve et al., 2016). Of course statistical approaches are available to minimize the impact of selection bias, but it cannot be eliminated. However, in some situations only observational studies may be possible.
These considerations suggest that any strategy for conducting economic evaluation in the postmarketing phase needs to be part of a broader strategy for the managed entry of MDs, involving decisions about (1) which MDs should be prioritized for further research, (2) the design of the clinical and economic evaluations, and (3) the arrangements for pricing and reimbursement of the device, both while under further study and after the results of the subsequent clinical and economic evaluation become available.
Improving Economic Evaluation Methods
Two recent systematic reviews shed some light on the current status of the economic evaluation of devices. Ciani, Wilcher, van Giessan, and Taylor (2017) analyzed a sample of HTA reports in the cardiovascular field in order to assess whether there were any key differences in the methods employed in studies of MDs as compared with studies of pharmaceuticals. They found that there were several differences in: (1) the types of clinical studies forming the basis of the HTAs (with a greater reliance on observational studies and less reliance on RCTs in the case of MDs); (ii) how the health problem and use of the technology were considered; (iii) the description and technical characteristics of the technology; and (iv) the consideration of the organizational aspects of use of the technology. Most of these differences arose because of the relative complexities in the use of devices, in terms of the number of interacting components, such as the number of groups and organizational levels targeted by the intervention, the number and variability of the outcomes, and the degree of flexibility or tailoring of the intervention.
In another review, Tarricone, Callea, Ogorevc, and Prevolnik Rupel (2017b) reviewed all the published economic evaluations and HTA reports for two cardiac devices: TAVI (an example of an emerging technology) and implantable cardioverter defibrillators (an example of a mature technology). The authors assessed each study using the CHEERS (Consolidated Health Economic Evaluation Reporting Standards) checklist (Husereau et al., 2013), supplemented by some additional categories reflecting the special characteristics of MDs. In general, the methods used in the studies were fairly representative of the broader economic evaluation literature. However, the special characteristics of MDs were not very well addressed. A learning curve was considered in only 16% of studies on TAVI. Incremental innovation was more frequently mentioned in the studies of ICDs, but its impact was considered in only 34% of the cases. Dynamic pricing was the most recognized feature but was empirically tested in less than half of studies of TAVI and only 32% of studies of ICDs. Finally, organizational impact was considered in only one study of ICDs but in almost all studies of TAVI, but none of the studies of estimated the size of any impact.
Improving the Analysis of the Available Clinical Data
If the methods of economic evaluation of MDs are to be improved, progress needs to be made in compensating for the paucity of well-controlled clinical studies of devices and in addressing the special characteristics of MDs. There is much to do in both these areas, but several examples of good practice can be identified. First, in a recent study of total hip replacement, Schnell-Inderst et al. (2017) used a method of (clinical) evidence synthesis that allows for the meta-analysis of RCT and observational data using bias adjustment based on formal elicitation. They performed an elicitation exercise using methodological and clinical experts to determine the strength of beliefs about the magnitude of internal and external bias affecting estimates of treatment effect. They incorporated the bias-adjusted treatment effects into a generalized evidence synthesis, using both frequentist and Bayesian statistical models, calculating relative risks as summary effect estimates with 95% confidence/credibility intervals to capture uncertainty. They found that the pooled effect size strongly depended on the inclusion of observational data as well the use of bias-adjusted estimates. The use of the bias-adjusted estimates shifted the pooled effect toward a lower treatment effect.
There still remains the challenge of producing unbiased estimates of treatment effect when only observational studies are available. In the economic evaluation more generally, propensity scoring, a form of matching, and multivariate regression are frequently used. The methods consider either known confounders only or include unknown confounders through the use of instrumental variables in the multivariate regression (Prentice et al., 2014). Examples of the use of observational data to assess the cost-effectiveness of MDs include the studies by Armeni et al. (2016) on MitraClip for patients with moderate/severe mitral regurgitation and by Rognoni, Ciani, Sommariva, and Tarricone (2016) on transarterial radioembolization for patients with hepatocellular carcinoma. Both of these studies used the propensity scoring method.
A detailed discussion of the methodological considerations in using propensity scoring or multivariate regression to adjust for potential biases in observational studies is beyond the scope of this article. However, it is worth noting that all these methods require data on the clinical characteristics of patients for incorporation in the statistical model. These data may not always be available from administrative databases or even clinical registries. In addition, there may be unobserved factors that lead to bias in observational studies. In principle, the use of an instrumental variable can be useful, but the challenge is often to find a suitable instrument.
As was mentioned earlier, the most common clinical studies of MDs are clinical registries, conducted in either the premarket or postmarket phase. However, the vast majority of these registries study only a single device or have very little information on factors that might be used as covariates in a multivariate analysis. There is an urgent need to improve the quality of registries for MDs by making sure that there is treatment variation (e.g., including more than one device), classifying the risk categories of patients, and, where possible, capturing data on resource use and the quality of life of patients (Tarricone, Torbica, & Drummond, 2017c).
Estimating the Learning Curve
The learning curve for the use of some devices has been estimated, either within the context of a clinical trial or by using an observational data set, such as an administrative database. In the context of an economic evaluation of a new MD, an analyst can either use an estimate of the learning obtained for a similar device or attempt to estimate the learning curve as part of the economic evaluation. An example of the first approach is the study by Varabyova et al. (2017), cited earlier. The main issue in this case is whether previous estimates are good predictors of the likely learning curve for the new device. To this end it may be possible to group MDs in various categories according to whether they are likely to have a shallow or a steep learning curve.
The learning curve has almost never been estimated as part of an economic evaluation. In the review by Tarricone et al. (2017b), the learning curve was addressed in 16% of the studies on TAVI by varying the rates of complications and procedure success in a sensitivity analysis. Although there has been an extensive review of the impact of learning curves on outcomes (Ramsay, Grant, Wallace, Garthwaite, & Monk, 2001), there have been very few assessments on the impact of learning curves on resource use or costs; see van Gestel, De Graeve, Vrints, Rodrigus, and Bosmans (2013), Bell et al. (2015), and Margier, Tchouda, Banihachemi, Bosson, and Plaweski (2015) for recent examples.
Considering Incremental Innovation
Two issues are raised by incremental innovation. First, if the MD under study changes, this change may be substantial enough to render the previous clinical data out of date. This issue arises when product modifications are submitted to the regulatory bodies for approval. One analysis in the United States showed that for 77 original market authorization applications for cardiac implantable electronic devices since 1979, the FDA has approved 5,829 “supplements” reflecting product modifications. In the vast majority of cases the FDA deemed that new clinical data were not necessary, although 37% of the supplements related to changes in the device’s design (Rome, Kramer, & Kesselheim, 2014).
At the very minimum, analysts should note any product modifications that have occurred to the version of the device for which clinical data are available and comment on whether these are likely to impact on device performance. It may be concluded that many of the modifications are quite minor and likely only to impact of convenience of use, such as miniaturization of the device. Others, such as an increase in battery life, may reduce costs if they reduce the need for replacing an implantable device. The impact of other modifications, such a change in the design of the device, could affect performance positively or negatively. Therefore data collection should be put in place to estimate any change, particularly if the design change is likely to mean that users face a new learning curve.
The other issue raised by incremental innovation is that the benefits to patients of a small change in the design of an MD may not be detectable by generic preference-based quality-of-life measures, like the EQ-5D, that are used to estimate the QALY gained by a new therapy. Therefore, some analysts have used bespoke “utility” measures, willingness-to-pay estimates, or discrete choice experiments in order to assess the various attributes of MDs. For example, Chancellor et al. (2008) used bespoke utility scenarios to assess the benefits of inhaled insulin (using a device) as compared with injected insulin. They found a marginal increase in utility owing to the greater convenience offered by the inhaler.
In a systematic review of willingness-to-pay studies and discrete choice experiments of devices, Wilkinson and Drummond (2015) found 30 relevant empirical studies, covering technologies such as hearing aids, insulin-delivery systems, cochlear implants, and endoscopy. They found that, in addition to clinical effectiveness, the studies assessed the value of attributes such as convenience/ease of use, length and/or frequency of treatment, visibility/size, and method of treatment delivery.
These studies are useful in exploring which attributes of treatment are valued by patients and the trade-offs they make among the various attributes, but their use in resource allocation decisions is uncertain. Some healthcare decision-makers may not feel it appropriate to pay extra for benefits like increased convenience unless this leads to an increase in treatment adherence and hence outcome. Also, many decision-makers value the standardization in methods offered by the generic preference-based measures, even if they fail to reflect some of the benefits of therapies.
Assessing Organizational Impact
The potential for organizational impact is often mentioned in economic evaluations of MDs but is almost never explored quantitatively. Therefore, it is important that economic evaluations of MDs consider a wide range of costs—specifically, the costs of the infrastructure necessary to accommodate the device (e.g., catheter laboratories of hybrid operating rooms, the cost of training in the use of the new device, and the creation of multidisciplinary teams). In addition, any impact of the new device on procedure costs should be quantified by measuring any increase in the volume of procedures or the conversion of procedures from an inpatient to an outpatient setting.
Accounting for Dynamic Pricing
Economic evaluations of MDs should anticipate future price changes by conducting a sensitivity analysis of device acquisition cost or by identifying the threshold price at which the device is no longer cost-effective. For example, in the NICE re-appraisal of drug-eluting stents mentioned above (NICE, 2008), it was determined that DES would be cost-effective unless the price was more than £300 higher than the price of bare metal stents.
Managing the Diffusion of MDs
Understanding the factors that lead to the adoption of MDs is especially important to healthcare decision-makers because the regulatory barriers to market access are considerably lower for MDs than for pharmaceuticals. A number of studies have shown that the utilization of MDs varies substantially, both within and between countries (Valzania, Torbica, Tarricone, Leyva, & Boriani, 2015). In a recent study, Torbica, Banks, Valzania, Boriani, and Fattore (2017) used a panel data regression model to investigate the variation in implant rates for cardiac implantable electrical devices across 57 regions in 5 EU countries. By analyzing a total of 1,330,098 hospitalizations extracted from discharge databases in Austria, England, Germany, Italy, and Slovenia from 2008 to 2012, they found implant rates were positively associated with higher levels of tertiary education among the labor force and the percentage of population over 74 years old. Regional per capita GDP and the number of implanting centers appeared to have no significant effect. There were also significant country effects in some of the analyses, suggesting that institutional factors play a role.
In another recent study, Hatz, Schreyögg, Torbice, Oriani, and Blankart (2017) investigated the characteristics of organizations (e.g., hospitals) and individuals (e.g., clinicians) who are more inclined to adopt and utilize cardiovascular devices based on a comprehensive analysis of environmental, organizational, individual, and technological factors. Seven random intercept hurdle models were estimated using data obtained from 1,249 surveys completed by members of the European Society of Cardiology. They found that better (device) manufacturer support increased the adoption probability of new cardiac devices (defined by CE mark approval dates) and that budget pressure increased the adoption probability of older devices. They argue that the role of manufacturer support should be investigated in more detail in order to assess whether it functions as a substitute for medical evidence on new devices and to gain insights into its relationship with clinical and cost-effectiveness.
This goes to the heart of the matter. Namely, if wide variation is observed in the utilization of MDs across countries and regions, is the level of utilization in each jurisdiction “appropriate” (defined in terms of clinical and cost-effectiveness), or are devices being overused in some jurisdictions and inappropriately restricted in others? Ideally, one would investigate the link between the use of the technologies and the patient outcomes obtained, but this is not possible using routinely available data. Alternatively, it may be possible to investigate whether physicians’ practices are in line with accepted clinical guidelines, although with a few notable exceptions (e.g., NICE’s clinical guidelines in the United Kingdom) these do not consider cost-effectiveness.
Another approach would be to use economic evaluation to manage the diffusion of MDs by conducting studies not only at market entry but also in the postmarket phase. In recent years there has been a growth in managed entry agreements that are collectively known as performance-based risk-sharing arrangements (PBRSAs). These schemes are known by different names and take slightly different forms in different jurisdictions, such as “coverage with evidence development” in the United States, “field evaluations” in Canada, and “only with research” in the United Kingdom. The basic approach is that new technologies are reimbursed by the healthcare system on the condition that further research is conducted into the clinical and/or cost-effectiveness of the technology. This approach is particularly well suited to MDs, since they are less well studied in clinical trials in the premarket phase and, because of the potential for a learning curve and incremental innovation, it is likely that clinical and cost-effectiveness in regular clinical practice will differ substantially from that observed in an experimental setting such as a controlled clinical trial.
Garrison et al. (2013) reviewed PBRSAs existing in a number of countries. Under their working definition, a PBRSA exhibits the following key characteristics:
There is a program of data collection agreed upon between the manufacturer and the payer.
The data collection is typically initiated during the time period following regulatory approval.
The price, reimbursement, and/or revenue for the product are linked to the outcome of the program of data collection.
The data collection is intended to address uncertainty (e.g., uncertainty about the efficacy or effectiveness in the treatment population tested in the premarket phase, the efficacy or effectiveness in a broader population or in the longterm, adverse events or adherence issues, whether healthcare providers’ actual management of patients affects outcomes, the size and value of cost offsets, the number and attributes of the patients treated in practice).
The arrangements provide a different distribution of risk between the payer and manufacturer than under the normal reimbursement arrangements.
Examples of PBRSA for procedures involving devices initiated in the United States include those for positron emission tomography for cancer diagnosis, percutaneous transluminal angioplasty and stenting for secondary prevention of stroke, and spinal cord stimulation for failed back surgery syndrome (Garrison et al., 2013). In another example of a PBRSA in Ontario, the generalizability of existing RCTs of DES conducted in the United States was questioned. Therefore, a pragmatic registry of all patients receiving DES was established in order to conduct a “field evaluation.” Coverage was provided for the stents provided under the registry. It was found that DES were more effective only in patients at high risk of stenosis (e.g., those with diabetes or particularly long or narrow lesions). Since this represented about 30% of the whole patient population, it was argued that this policy saved between $35 million and $58 million, as compared with the potential uncontrolled adoption of DES (Goeree et al., 2009; Levin, Goeree, Sikich, & Jorgensen, 2007).
The key questions in using economic evaluation to manage the diffusion of MDs concern the timing of studies (e.g., before or after providing reimbursement) and the nature of the research (e.g., the parameters estimated and the methods used). Making a device available early can have benefits, as it allows patient access to potentially beneficial therapies. However, decisions about the use of the device when the evidence base is relatively immature can lead to clinical and financial risk. Also, once the device is generally available, the possibilities for further research to reduce uncertainty may be limited, either because manufacturers have fewer incentives to invest in further research or because patients and clinicians are unwilling to participate in studies, especially RCTs.
Health economists have explored the use of value-of-information analysis to determine the balance between the benefits of early access to a technology and the value of additional evidence to resolve uncertainty (Claxton et al., 2012). In a recent paper, Rothery et al. (2017) set out a framework for considering the issues relating to the uncertainty in the diffusion and use of MDs, considering the learning curve, incremental innovation, investment and irrecoverable costs, and dynamic pricing. They use this framework to illustrate the circumstances under which it might be preferable to restrict access to the device until further research establishes its value or to allow access as long as further research is conducted.
Rothery et al. (2017) also discuss the incentives for further research and how the value and cost of such research might be shared between the manufacturer and the healthcare sector. Drummond et al. (2016) argue that incentives to manufacturers to conduct research need particular attention, given the lack of data mentioned earlier. In a situation where a PBRSA has been established for the first device to market, this might provide an opportunity for different manufacturers to share the costs of the research, as opposed to most of the costs being borne by the first to market. For example, “fast follower” devices claiming substantial equivalence could be included in the scheme, thereby sharing the costs of the research and generating evidence on the comparative effectiveness of the different devices.
This article outlines the key challenges in conducting economic evaluations of MDs and how they might be tackled. However, many methodological and policy challenges remain, and further research is required. First, approaches need to be developed to integrate regulatory and health technology assessment for MDs, so that future data needs can be anticipated when clinical studies are being planned. It is likely that the balance of data collection for the economic evaluation of MDs will be in the postmarketing phase. Therefore, effort needs to be made in the analysis of “real-world” costs and effects of devices in regular clinical use. Linked to this, attention needs to given to the design of clinical registries, so that these will also be useful for conducting economic evaluations as well as establishing the efficacy and safety of MDs.
Second, more study is required to explore the impact of the various distinctive characteristics of medical devices. In particular, it would also be useful to determine whether it is possible to categorize devices in terms of their likely learning curve, according to the types of technology (diagnostic or therapeutic; implantable or nonimplantable), the medical specialty, or the device risk class. In addition, there should be more quantitative assessments of the organizational impact of MDs.
Finally, further study is required of the various influences on the diffusion of devices, especially the motivational factors facing patients, clinicians, and the institutions in which they work. In turn, this could help improve the design and conduct of PBRSAs, which are likely to be central to the economic evaluation of MDs in the future.
The specific characteristics of MDs: The following paper discusses the arguments for and against considering the specific characteristics of medical devices when conducting economic evaluations.
- Drummond, M. F., Griffin, A., & Tarricone, R. (2009). Economic evaluation for devices and drugs. Same or different? Value in Health, 12(4), 402–404.
The need to strengthen the regulation of MDs: This paper discusses this issue in the context of the United States and Europe.
- Sorenson, C., & Drummond, M. F. (2014). Improving medical device regulation: the US and Europe in perspective. The Milbank Quarterly, 92(1), 114–150.
Challenges in the assessment of medical devices: This journal supplement describes the results of the European Union MedtecHTA Project, a recent and comprehensive research effort in the economic evaluation of MDs.
- Tarricone, R., Torbica, A., Drummond, M. F., & Schreyögg, J. (2017). Assessment of medical devices: Challenges and solutions. Health Economics, 26(Suppl. 1), 1–152.
- Armeni, P., Boscolo, P. R., Tarricone, R., Capodanno, D., Maggioni, A. P., Grasso, C., et al. (2016). Real-world cost effectiveness of MitraClip combined with medical therapy versus medical therapy alone in patients with moderate of severe mitral regurgitation. International Journal of Cardiology, 209, 153–160.
- Bell, R., Pandanaboyana, S., Hanif, F., Shah, N., Hidalgo, E., Lodge, J. P. A., et al. (2015). A cost effective analysis of a laparoscopic versus an open left lateral sectionectomy in a liver transplant unit. HPB (Oxford), 17(4), 332–336.
- Burch, J., Epstein, D., Baba-Akbari, A., Weatherly, H., Fox, D., Golder, S., et al. (2008). Stapled haemorrhoidectomy (haemorrhoidopexy) for the treatment of haemorrhoids: A systematic review and economic evaluation. Health Technology Assessment, 12, 1–214.
- Chancellor, J., Aballea, S., Lawrence, A., Sheldon, R., Cure, S., Plun-Favreau, J., et al. (2008). Preferences of patients with diabetes for inhaled versus injectable insulin regimens. PhamacoEconomics, 26, 217–234.
- Ciani, O., Wilcher, B., van Giessan, A., & Taylor, R. S. (2017). Linking the regulatory and reimbursement processes for medical devices: The need for integrated assessments. Health Economics, 26(Suppl. 1), 13–29.
- Claxton, K., Palmer, S., Longworth, L., Boyke, L., Griffin, S., McKenna, C., et al. (2012). Informing a decision framework for when NICE should recommend the use of technologies only in the context of an appropriately designed programme of evidence development. Health Technology Assessment, 16, 1–103.
- Drummond, M. F. (2013). Twenty years of using economic evaluations for drug reimbursement decisions. What has been achieved? Journal of Health Politics, Policy and Law, 38(6), 1081–1102.
- Drummond, M. F., Griffin, A., & Tarricone, R. (2009). Economic evaluation for devices and drugs—Same or different? Value in Health, 12, 402–404.
- Drummond, M. F., Tarricone, R., & Torbica, A. (2016). Incentivizing research into the effectiveness of medical devices. European Journal of Health Economics, 17(9) 1055–1058.
- Dyer, M. T., Goldsmith, K. A., Khan, S. N., Sharples, L. D., Freeman, C., Hardy, I., et al. (2008). Clinical and cost-effectiveness analysis of an open label, single-centre, randomised trial of spinal cord stimulation (SCS) versus percutaneous myocardial laser revascularisation (PMR) in patients with refractory angina pectoris: The SPiRiT trial. Trials, 9, 1–11.
- Food and Drug Administration. (2016). Unsafe and ineffective devices approved in the EU that were not approved in the US. http://www.elsevierbi.com/~/media/Supporting%20Documents/The%20Gray%20Sheet/38/20/FDA_EU_Devices_Report.pdf
- Fraser, A. G., Krucoff, M. C., Brindis, R. G., Komajda, M., & Smith, S. C., Jr. (2011). Commentary: International collaboration needed on device clinical trials. British Medical Journal, 342, d2952.
- Garrison, L., Towse, A., Briggs, A., de Pouvourville, G., Greuger, J., Mohr, P. E., et al. (2013). Performance-based risk-sharing arrangements—Good practices for design, implementation, and evaluation: Report of the ISPOR Good Practices for Performance-Based Risk-Sharing Arrangements Task Force. Value in Health, 16, 703–719.
- Goeree, R., Bowen, J. M., Blackhouse, G., Lazzam, C., Cohen, E., Chiu, M., et al. (2009). Economic evaluation of drug-eluting stents compared to bare metal stents using a large prospective study in Ontario. International Journal of Technology Assessment in Health Care, 25, 196–207.
- Grieve, R., Abrams, K., Claxton, K., Goldacre, B., James, N., Nicholl, J., et al. (2016, September 27). Cancer Drugs Fund requires further reform. British Medical Journal, 354, i5090.
- Guillou, P. J., Quirke, P., Thorpe, Walker, J., Jayne, D. G., Smith, A. M., et al. (2005). Short-term endpoints of conventional versus laparoscopic-assisted surgery in patients with colorectal cancer (MRC CLASICC trial); Multicentre, randomised controlled trial. Lancet, 365, 1718–1726.
- Hatz, M. H. M., Schreyögg, J., Torbice, A., Oriani, G., & Blankart, C. R. B. (2017). Adoption decisions for medical devices in the field of cardiology: Results from a European survey. Health Economics, 26(Suppl. 1), 124–144.
- Husereau, D., Drummond, M. F., Petrou, S., Carswell, C., Moher, D., Greenberg, D., et al. (2013). Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and elaboration: A report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value in Health, 16(2), 231–250.
- Klarman, H. E., Francis, J., & Rosenthal, G. (1968). Cost-effectiveness analysis applied to the treatment of chronic renal disease. Medical Care, 6, 48–54.
- Levin, L., Goeree, R., Sikich, N., & Jorgensen, B. (2007). Establishing a comprehensive continuum from an evidentiary base to policy development for health technologies: The Ontario experience. International Journal of Technology Assessment in Health Care, 23, 299–309.
- Lin, P.-J., Cangelosi, M. J., Lee, D. W., & Neumann, P. J. (2013). Willingness to pay for diagnostic technologies: A review of the contingent valuation literature. Value in Health, 16, 797–805.
- Margier, J., Tchouda, S. D., Banihachemi, J. J., Bosson, J. L., & Plaweski, S. (2015). Computer-assisted navigation in ACL reconstruction is attractive but not yet cost efficient. Knee Surgery Sports Traumatology Arthroscopy, 23(4), 1026–1034.
- National Institute of Health and Care Excellence (NICE). (2008). Drug-eluting stents for the treatment of coronary artery disease. Technology Appraisal 152. London, UK: NICE.
- Office of Technology Assessment. (1978). Policy implications of the computed tomography (CT) scanner. http://ota.fas.org/
- Prentice, J. C., Conlin, P. R., Gellad, W. F., Edelman, D., Lee T. A., & Pizer, S. D. (2014). Capitalizing on prescribing pattern variation to compare medications for type 2 diabetes. Value in Health, 17, 854–862.
- Ramsay, C. R., Grant, A. M., Wallace, S. A., Garthwaite, P. H., & Monk, A. F. (2001). Statistical assessment of the learning curves of health technologies. Health Technology Assessment, 5(12), 1–104.
- Rognoni, C., Ciani, O., Sommariva, S., & Tarricone, R. (2016). Real-world data for the evaluation of transarterial radioembolization versus sorafenib in hepatocellular carcinoma: A cost-effectiveness analysis. Value in Health, 20(3), 336–344.
- Rome, B. N., Kramer, D. B., & Kesselheim, A. S. (2014). FDA approval of cardiac implantable electronic devices via original and supplement premarket approval pathways. Journal of the American Medical Association, 311, 385–391.
- Rothery, C., Claxton, K., Palmer, S., Epstein, D., Tarricone, R., & Sculpher, M. (2017). Characterising uncertainty in the assessment of medical devices and determining future research needs. Health Economics, 26(Suppl. 1), 109–123.
- Schnell-Inderst, P., Iglesias, C., Arvandi, M., Ciani, O., Matteucci Gothe, R., Peters, J., et al. (2017). Synthesis of RCT and observational data: A bias-adjusted evidence synthesis of total hip replacement. Health Economics, 26(Suppl. 1), 46–69.
- Shuren, J., & Califf, R. M. (2016). Need for a national evaluation system for health technology. Journal of the American Medical Association,
- Sorenson, C., & Drummond, M. F. (2014). Improving medical device regulations: The United States and Europe in perspective. Milbank Quaterly, 92, 114–150.
- Tarricone, R., & Drummond, M. F. (2011). Challenges in the clinical and economic evaluation of medical devices: The case of transcatheter aortic valve implantation. Journal of Medical Marketing, 11(3), 221–229.
- Tarricone, R., Torbica, A., & Drummond, M. F. (2017a). Challenges in the assessment of medical devices: The MedtecHTA project. Health Economics, 26(Suppl. 1), 5–12.
- Tarricone, R., Callea, G., Ogorevc, M., & Prevolnik Rupel, V. (2017b). Improving the methods for the economic evaluation of medical devices. Health Economics, 26(Suppl. 1), 70–92.
- Tarricone, R., Ferrè, F., Torbica, A., & Drummond, M. F. (2014). Generating appropriate clinical data for value assessment of medical devices: what role does regulation pay? Expert Review of Pharmacoeconomics & Outcomes Research, 14(5), 707–718.
- Tarricone, R., Torbica, A., & Drummond, M. F. (2017c). Key recommendations from the MedtecHTA project. Health Economics, 26(Suppl. 1), 145–152.
- Taylor, R. S., & Iglesias, C. P. (2009). Assessing the clinical and cost-effectiveness of medical devices—Are they that different? Value in Health, 12(4), 404–406.
- Torbica, A., Banks, H., Valzania, C., Boriani, G., & Fattore, G. (2017). Investigating regional variation of cardiac implantable electrical device implant rates in European healthcare systems: What drives differences? Health Economics, 26(Suppl. 1), 30–45.
- Valzania, C., Torbica, A., Tarricone, R., Leyva, F., & Boriani, G. (2015). Implant rates of cardiac implantable electrical devices in Europe: A systematic literature review. Health Policy, 120(1), 1-15
- van Gestel, R., De Graeve, D., Vrints, C., Rodrigus, I., & Bosmans, B. (2013). Hospitalization costs of TAVI in one Belgian university hospital. Acta Cardiologica, 68, 263–270.
- Varabyova, Y., Blankart, C. R., & Schreyögg, J. (2017). The role of learning in health technology assessments: An empirical assessment of endovascular aneurysm repairs in German hospitals. Health Economics, 26(Suppl. 1), 93–108.
- Wilkinson, G., & Drummond, M.F. (2015). Alternative approaches for assessing the socioeconomic benefits of medical devices: A systematic review. Expert Review of Medical Devices, 12(5), 629–648.