Qualitative Methods in Health Economics
- Joanna CoastJoanna CoastDepartment of Population Health Sciences, Bristol Medical School, University of Bristol
- and Manuela De AllegriManuela De AllegriInstitute of Public Health, Heidelberg University
Qualitative methods are being used increasingly by health economists, but most health economists are not trained in these methods and may need to develop expertise in this area. This article discusses important issues of ontology, epistemology, and research design, before addressing the key issues of sampling, data collection, and data analysis in qualitative research. Understanding differences in the purpose of sampling between qualitative and quantitative methods is important for health economists, and the key notion of purposeful sampling is described. The section on data collection covers in-depth and semistructured interviews, focus-group discussions, and observation. Methods for data analysis are then discussed, with a particular focus on the use of inductive methods that are appropriate for economic purposes. Presentation and publication are briefly considered, before three areas that have seen substantial use of qualitative methods are explored: attribute development for discrete choice experiment, priority-setting research, and health financing initiatives.
The use of qualitative methods has become increasingly common in health economics, with a particular growth in relation to research conducted in lower- and middle-income countries and around issues associated with priority setting. The methods have also been extensively used in mixed-methods approaches where the aim is to generate new quantitative data around values and preferences, as in discrete choice experimentation, or the understanding of these values and preferences, as in Q methodology (Baker, Thompson, & Mannion, 2006). Qualitative methods are not exclusive to the application of economics to health, however; they also influence institutional economics and, particularly, feminist economics (Berik, 1997; Starr, 2014).
This article begins with a very brief exploration of the nature of economics, and issues of ontology and epistemology in the application of qualitative methods in (health) economics, before discussing methods for sampling, data collection, analysis, and presentation of findings.
The Nature of Economics, and Issues of Ontology and Epistemology
Some economists/health economists may question whether qualitative methods can ever really be part of “economics,” particularly if they tend to see economics as a collection of quantitative or “econometric” approaches. A useful way of thinking about the nature of economics, however, is to follow Alan Williams in defining economics as a discipline: “a systematic body of knowledge with its own distinctive concepts and modes of thinking” (Williams, 1997b, p. 206). This mode of thinking forms a conceptual focus on issues like resources, scarcity, rationality, incentives, values, and so on. It can be contrasted, for example, as Williams did, with the disciplinary mode of thinking associated with sociology, “which employs characteristic concepts such as role, social class, stigmatization etc.” (Williams, 1997a, p. 40). In both cases, the concepts draw upon, and are used within, relevant theory, and in neither are the modes of thinking bound to particular methods; within sociology, for example, both quantitative and qualitative methods are extensively used to study phenomena of interest. Within economics and health economics, the application of qualitative methods has lagged behind its use in other disciplines, but the methods have the potential to assist in understanding complex economic phenomena (Coast & Jackson, 2017a).
Unlike those in many of the other social science disciplines, economists do not tend to spend much time considering issues of ontology and epistemology. Questions of the nature of being and theories of knowledge are largely taken for granted by economists, with realist, positivist, and deductive approaches seen as standard (Coast, 1999). There is a single reality that can be accessed through well-designed research. This view contrasts with the relativist, constructivist approaches that are common in much qualitative work, where there is a belief in multiple realities that are constructed and interpreted through the eyes of the researcher, whose focus is usually on generating theory through induction (Lincoln, 1992). It may seem that these views cannot meet, but there are options for the (health) economist wanting to conduct qualitative research that can both inform and be accepted by economists. While some economists do work within a constructivist approach (see, e.g., Husbands, Jowett, Barton, & Coast, 2017b), perhaps more commonly, economists have drawn on more critical approaches to realism and situated their work within these paradigms (Coast, Kinghorn, & Owen-Smith, 2017; Vosper, Flynn, & Coast, 2017). Approaches like critical realism or notions of “subtle” realism can be helpful in allowing a realist ontology but a more interpretive epistemology, in which it is accepted that that reality can only be accessed imperfectly through prisms associated with different perceptions and viewpoints (Hammersley, 1992; Lawson, 1997, 2003; Maxwell, 2012). It is notable that the linear approach to research associated with deducing theory and then testing associated hypotheses that is common in quantitative health economics is replaced in qualitative health economics with a much more iterative approach to research, where questions are explored with initial data collection and early analysis, and the early analysis then drives further sampling and data-collection decisions, in a process that continues until theory is fully developed or understanding is complete. Interested readers are referred to fuller expositions of methodology in economics (Blaug, 1992) and specifically in relation to qualitative methods in health economics (Coast & Jackson, 2017a).
Qualitative Research Design
Qualitative methods are associated with many traditions, including anthropology, sociology, education, health, and psychology, and they come in different forms, often linked to the different traditions. It is not feasible to give a full taxonomy here, but some of the approaches that economists are most likely to come across include ethnography, narrative research, phenomenology, and grounded theory (Creswell, 2013; Coast, Kinghorn, & Owen-Smith, 2017). Ethnography is often used for exploring new cultures, and it is associated with both anthropology and sociology; research involves immersion in the field of study (Hammersley & Atkinson, 1983). Ethnography draws extensively on observational methods as well as interviews. Narrative research is used to focus on individuals and their stories, and it is associated with the humanities and storytelling (Creswell, 2013; Riessman, 2011). Narrative research focuses on individual’s life stories and allowing them to be told; interviews are the primary method of data collection, although narrative approaches can also be used in documentary analysis. Phenomenology is associated with psychology and philosophy. It focuses on understanding the essence or lived experience of a particular situation. It has often been applied within health research and nursing to understand how individuals themselves experience particular conditions and to find the essence and “hidden meanings” of that lived experience (Creswell, 2013; Grbich, 2013; Hansen, 2006). Grounded theory developed within sociology and focuses on generating new theory, particularly around processes and interactions (Creswell, 2013). Phenomena are explored in depth to generate new theoretical insights, and grounded theory provides a particular methodological approach to sampling, data collection, and data analysis, with the aim of generating a fully “grounded” theory of a particular topic (Charmaz, 2014; Glaser & Strauss, 1968; Strauss & Corbin, 1990). Qualitative research within health economics may draw on any of these traditions, with some being more applicable to some settings and research questions than others. It may also draw on analytic approaches that are less theoretically focused, such as framework (Gale, Heath, Cameron, Rashid, & Redwood, 2013; Ritchie & Lewis, 2003) or thematic analysis (Braun & CLarke, 2006), because these more clearly enable the analyst to also bring in questions of economic theory.
The design of qualitative research in health economics thus depends not just on qualitative theories but also, importantly, needs to be located within relevant economic theory. For research where the primary aim of the qualitative research is to inform later quantitative research, the relevant economic theory will usually be easy to identify; for example, qualitative research to identify attributes for capability-based outcome measures for economic evaluation (Al-Janabi, Flynn, & Coast, 2012; Canaway, Al-Janabi, Kinghorn, Bailey, & Coast, 2017a; Sutton & Coast, 2014) was explicitly located within Sen’s capability theory (Sen, 1993), whereas much attribute development for discrete choice experimentation is located within notions of utility maximization (Coast et al., 2012), and Baker’s work generating Q sets in diabetes was based within economic concepts of rationality (Baker, 2006). Qualitative research that is intended to inform the interpretation of existing quantitative data is also likely to be clearly related to existing aspects of economic theory, as for example with research intended to explain how health-state valuations are generated (Robinson, 2017; Robinson, Dolan, & Williams, 1997; Spencer, 2003). Research that is intended to inform understanding and generate theoretical development on health systems, in relation to issues such as the use of financial incentives, priority setting in healthcare, aid effectiveness, and markets in health, has often drawn on principal-agent theory as a starting point (Coast, 2001a; De Allegri, Sanon, & Sauerborn, 2006; Martínez-Álvarez, 2017).
Finally, research needs to be designed for rigor, which means different things for those working in a qualitative paradigm than in the quantitative research that economists are more familiar with (Coast, Kinghorn, & Owen-Smith, 2017), covering issues such as triangulation (different approaches resulting in similar findings), reflexivity (awareness of the researcher’s role), rich thick description, prolonged engagement with the field, and multiple coding of data sets (Creswell, 2013). Indeed, it is a characteristic of qualitative research that the research design emerges as the research progresses, with decisions about further sampling and data collection being made in response to analysis of earlier data (De Allegri, 2017). Research design may also need to tackle issues such as combining methods and making decisions about translation when research is conducted in multiple countries or in countries where the research team is from a different linguistic setting. Translation may relate both to the translation of research tools for use in the research and to decisions about translation of data for analysis or analysis in the original language. In relation to the former, a particular challenge is the iterative and emerging nature of qualitative research, where information collected from early phases of fieldwork is used to modify research tools in later phases to improve the focus on the research question, but where rapid translations of field materials (e.g., transcripts) are needed to allow for an equally rapid adaptation of the tools. A further challenge for researchers working outside their own linguistic setting is that they may face instances when certain concepts are not translatable in the local language, and they may struggle to adapt an idea generated in a given linguistic expression to another sociocultural context.
Sampling is of key importance in qualitative research, but its aims are not the same as in quantitative health economics research, where the usual purpose of the sampling strategy is to obtain representativeness and generalizability. Sampling in qualitative research aims to generate new theoretical insights or greater understanding of a particular issue (Owen-Smith & Coast, 2017b). Rather than generalizability, the focus is on the extent to which insights are transferable to other settings. The large quantities of textual data acquired in qualitative research mean that it is generally not possible to have very large sample sizes—an issue that economists can see as problematic because it does not fit with their established view of the world. Sampling in qualitative research instead is purposeful (also sometimes termed purposive; Patton, 2015) with the aim often being to obtain samples of those who can bring particular insight to an issue—termed information-rich samples (Patton, 2015)—or to ensure that those with different characteristics and insights are included. There are different forms of purposeful sampling but all involve the deliberate selection of individuals based upon some specific criterion/criteria.
Theoretical sampling is associated with grounded theory: Sampling choices are determined by the emerging theory, and the approach is particularly associated with the inductive development of theory and is often linked with iterative constant comparative analytical approaches (Strauss & Corbin, 1990). Theoretical sampling has been used in economic research on priority setting (Coast, 2001a). Sampling can also be used to obtain a diverse sample, as when sampling for maximum variation, ensuring that a wide a variety of perspectives is captured across defined relevant categories, or by using quota sampling to capture those with characteristics seen as being particularly important in the context of the study, as for example with the use of distance from a health facility in the work of De Allegri and colleagues on demand for health insurance (De Allegri, Sanon, Bridges, & Sauerborn, 2006; De Allegri, Sanon, & Sauerborn, 2006). Such methods can also be particularly valuable for economic studies focusing on preference elicitation, to ensure that all relevant concepts are captured (Abiiro, Leppert, Mbera, Robyn, & De Allegri, 2014; Coast et al., 2012; Coast & Horrocks, 2007). Also of importance for economic research with citizens are sampling approaches that enable membership of the population to be a relevant characteristic for sampling, as in purposeful random sampling, where the opportunity to participate in the research is randomly allocated and thus citizen input is obtained in an unbiased manner, although those who choose to participate may have particular characteristics—this approach has been combined with maximum variation sampling in generation of capability-based measures for adults (Al-Janabi et al., 2012) as well as in studies exploring values related to end-of-life care with members of the public (Kinghorn, Canaway, Bailey, & Coast, 2017). Finally, particularly for hard-to-reach groups, sampling may require the use of key informants to identify particular types of individuals or of snowballing, where participants are asked to recommend other participants. The use of key informants can be particularly helpful when the aim is identify those with specialized expertise, as in Husbands’s work on modeling processes within health economics (Husbands, Jowett, Barton, & Coast, 2017a; Husbands et al., 2017b).
Sampling techniques can be combined and used flexibly as the research progresses in an iterative manner, and theoretical understanding emerges. Examples of this in health economics include the research by Owen-Smith and colleagues on priority setting at the micro-level, which combined key informant and snowball sampling, purposeful quota sampling, and maximum variation sampling (Owen-Smith & Coast, 2017b), work by Kalolo and colleagues, which combined key informant and snowball sampling with maximum variation sampling to explore issues related to the implementation of an insurance scheme in Tanzania (Kalolo et al., 2017), and research by Ridde et al. that combined diverse sampling techniques within the framework of a multiple-case-study design to understand the implementation of a performance-based financing intervention in Burkina Faso (Ridde et al., 2014).
Sample size or adequacy is also an issue of concern to economists used to quantitative approaches. With qualitative research, the adequacy of the sample depends not just on the number of informants, but also on the amount and quality (depth) of the data obtained (Morse, 1994). Judgments are made about the adequacy of the sample with regard to the original aims and objectives of the research. One way in which this is sometimes done is with regard to the notion of theoretical saturation. This is an approach associated with the development of grounded theory and essentially focuses on whether sufficient data have been obtained to have fully explicated the developing theory (Charmaz, 2014; Glaser & Strauss, 1968; Strauss & Corbin, 1990). More generally, iterative approaches to data collection and analysis are likely to ensure that the researcher is fully aware of the adequacy of the sample in terms of meeting the research aims (Owen-Smith & Coast, 2017b). Issues with the adequacy of sampling may arise particularly where the aim is to sample hard-to-reach groups, when sampling is done from large and diverse groups, or where gatekeepers (particularly within health services) have control over who can and cannot be approached (Coast, Kinghorn, & Owen-Smith, 2017).
Data collection in qualitative research uses a number of methods, many of which have several variants. Individual interviews are probably the qualitative technique with which economists will be most familiar and that they are most likely to use. Such interviews involve the researcher in talking with the informant, usually with audio recording of the interview, and allow the researcher to see the world from the perspective of their informants. Unlike structured interviews, where the aim is to ask identical questions to avoid bias, qualitative interviews are generally semistructured or in-depth (Bryman, 2004), with a series of prompts contained in a topic schedule, to encourage the informant to speak openly about the issue under discussion, but with the informant directing the pace and ordering of the interview (Owen-Smith & Coast, 2017a). Semistructured interviews may be more tightly defined and focused on a specific issue and can be particularly appropriate for issues like developing appropriate language for measures or preference-elicitation tasks (Coast et al., 2008), and understanding completion of preference-elicitation tasks (Geneau, Massae, Courtright, & Lewallen, 2008). In-depth interviews are less directed and may just focus on encouraging the informant to discuss the topic area of interest, prompted by the researcher as he/she pursues particular lines of enquiry. In-depth interviewing can be particularly valuable when the researcher is operating in a very new arena, such as in the work of developing capability measures for the first time (Grewal et al., 2006).
There may be other types of interviews that economists wish to conduct, and that might combine completion of a task alongside the semistructured or in-depth interviews. Examples include completion of questionnaires or preference-elicitation tasks, including conducting cognitive interviews using techniques such as “think-aloud” (Al-Janabi, Keeley, Mitchell, & Coast, 2013; Bailey et al., 2016; Bailey, Kinghorn, Orlando, & Coast, 2017; Spencer, 2003), as well as the use of more novel tasks that facilitate information collection during the interviews, such as comment on a vignette, or the use of pictorial methods, such as hierarchical mapping (Canaway, Al-Janabi, Kinghorn, Bailey, & Coast, 2017b). Such tasks allow the collection of specific information associated with the completion of the task, but they also can be helpful in enabling the researcher to develop rapport with the informant.
The researcher is the main “tool” in conducting interview-based qualitative research; therefore, it is important that the researcher has developed interviewing skills through appropriate methodological training. The interviewer will often use a topic guide (interview schedule), which can act as an aide memoire during the interview and contains a list of prompts to be covered. The topic guide is the basis for the interview and so it is important that sufficient time is given to ensuring that all the relevant issues are covered; as interviews and analysis progress, the topic guide may be adjusted to enable further exploration of emerging themes. Issues that the researcher needs to consider when conducting interviews include how to contact potential informants, where the interview will take place—preferably somewhere quiet to facilitate adequate audio recording (King & Horrocks, 2010)—and the development of rapport between the informant and the interviewer (Gold, 1997; Hammersley & Atkinson, 1983). The latter requires careful thinking about personal presentation, clothing, providing appropriate and encouraging responses, and so on, to improve the likelihood that the informant will talk honestly and openly to the interviewer (Owen-Smith & Coast, 2017a). These issues may be particularly important when working in different sociocultural settings, where different rules may apply in terms of acceptable and polite behavior.
Focus groups are group interviews. Whereas individual interviews are particularly useful when the emphasis is on individual preferences or experiences—and in some settings would be the only culturally appropriate methods for eliciting these issues within the private sphere—focus groups are particularly useful for eliciting group beliefs or practices. They can be an efficient method of gaining information from a larger group of individuals and are particularly valuable where there is an interest in shared values and/or interaction (Kitzinger, 2006). They can also be helpful where reflection is desirable among participants, which may be facilitated as participants hear the views of others (Dolan, Cookson, & Ferguson, 1999). The aim of the group interview is to encourage discussion between participants that may allow new themes that had not been envisaged by the economist to arise.
Focus groups generally involve a facilitator who runs the group, a dedicated note-taker who can aid transcription, and an additional individual who can act as an observer, pulling out issues that the facilitator might have missed, but who also deals with issues such as taking consent from late arrivals or comforting participants who have become upset by discussion during the focus group. Thought needs to be given to the venue (including the setup of the room), the timing of the group, and the composition of the group, particularly the heterogeneity of participants (Finch & Lewis, 2003; King & Horrocks, 2010). As with individual interviews, topic guides are generally used to form the basis for discussion, and skilled facilitation is needed to ensure that the focus group stays on track and that all participants have the opportunity to contribute. Focus groups, particularly, lend themselves to the inclusion of elicitation tasks alongside the discussion, and examples where economists have included these tasks in focus groups can be seen in the literature (Cookson & Dolan, 1999; Dolan et al., 1999; Kinghorn et al., 2017). Compared with individual interviews, focus groups may seem to enable economists to efficiently sample from a number of informants, but the resultant data will have much less depth than those from individual interviews. Hence, focus groups should be used in cases where the research purpose is enhanced by the interaction that the group setting facilitates. It should be noted that some informants may find it difficult to contribute (Kitzinger, 1994, 1995) and some individuals may have a tendency to conform to the views of more dominant members of the group (Gibbs, 1997; Kitzinger, 1995; MacDougall & Baum, 1997), which means that economists seeking to obtain a “short cut” to larger numbers through the focus group approach should be wary.
While it is often very valuable to obtain information using individual and group interviews, particularly in obtaining information about preferences, there are also situations where individual views alone may not provide a fully rounded view and where direct insights into the world being studied would be valuable. In particular, the study of health systems or processes may also benefit from observation of what actually happens in practice. Observation may be conducted on either a nonparticipant or a participant basis (Bryman, 2004; Hammersley & Atkinson, 1983). In the former, the researcher is not involved in the process and merely observes, whereas in the latter, the researcher acts as a full participant in the research. Empirical examples of the two approaches applied to the same topic can be found in research on the use of economic evaluation in healthcare decision-making at the local level, where McDonald acted as a participant observer taking on a role in the local healthcare system that enabled her to acquire detailed knowledge of that system from a particular perspective (McDonald, 2002), while Eddama took on a nonparticipant observational role, observing priority-setting meetings and mechanisms alongside the conduct of in-depth interviews, where acting as a participant would not have been viable given the required experience and seniority of those involved (Eddama & Coast, 2009).
The researcher will need to decide what to observe, and access may have to be negotiated through an observer; it will also require obtaining informed consent from participants, as with interviews or focus groups. When observing, it is important to take comprehensive notes, which form the main data for analysis. Notes may include information about conversations and interactions but also contextual information. Observation may enable the researcher to see how processes and institutions work, without having to rely on the accounts of others, but the researcher has less control over the process and may have to conduct a large number of observations before the particular phenomenon of interest is observed (Owen-Smith & Coast, 2017a).
Audio-recorded data should be transcribed using a clear standard (Atkinson & Heritage, 1999; Poland, 1995). The economist also needs to be aware that the detail required in the transcription may differ depending upon the type of analysis that they wish to undertake (Davidson, 2009), for example, the detail required to undertake conversation analysis is much greater than that required for thematic approaches to analysis.
Other forms of data collection, not considered here in detail, are focused on the collection and analysis of relevant documents, often in conjunction with other methods, and the secondary analysis of existing qualitative data (Al-Janabi & Owen-Smith, 2017).
The process of analyzing qualitative data is likely to be unfamiliar to most economists, who may have a sense that there is some sort of mystical process whereby “themes arise from the data” (Coast & Jackson, 2017b). It is also not immediately obvious what forms of analysis might be most appropriate to different forms of qualitative work within health economics, with researchers drawing from a number of approaches depending on their particular aims and objectives. Perhaps most commonly used (Coast & Jackson, 2017b) are approaches drawing on constant comparison (Glaser & Strauss, 1968) and those drawing on thematic approaches (Braun & CLarke, 2006) allied with framework analysis (Gale et al., 2013; Ritchie, Spencer, & O’Connor, 2003). Coast and Jackson have argued that, while this is the case, economists’ analytical choices have very often been adjusted somewhat to meet the needs of economics in ways that might be considered unconventional in other disciplines (Coast & Jackson, 2017b). Indeed, economists who study particular issues using qualitative methods bring with them their disciplinary “mode of thinking” in terms of how they “see” the data. The prevailing analytical choices in health economists’ qualitative analyses suggest that they are informed by economic theory, but not bound by it. Coast and Jackson therefore suggested that two aspects of analysis do, and should, characterize the analysis of qualitative data for health economics: the application of an “economic lens” drawing on the underlying concerns of economics, such as scarcity, allocation, resources, distribution, equity, efficiency, incentives, markets, rationality, productivity, and so on, and the generation of outputs in the form of a model comprising a set of relationships that explain an economic phenomenon (Coast & Jackson, 2017b).
In practice, there are many commonalities across analytic approaches (Creswell, 2013; Miles & Huberman, 1994; Rapley, 2011), and Rapley identified the following central methods that are common to most qualitative data analyses:
Always start by engaging in some kind of close, detailed reading of a sample/section/bit of your archive of data.•
Always read and systematically label your archive of data.•
Always reflect on why you’ve done what you’ve done.•
Always review and refine your labels and labeling practices.•
Always focus on what you feel are the key labels and the relationship between them.•
Always make notes of your thinking behind why you’ve done what you’ve done.•
Always return to the field with the knowledge you have already gained in mind and let this knowledge modify, guide, or shape the data you want to collect next (Rapley, 2011, pp. 278–279).
While economists will be faced with a number of analytic choices, it is important to be close to (“immersed in”) the data and to develop theoretical sensitivity (Strauss & Corbin, 1990). Synthesis and interpretation of the data are key. Coast and Jackson (2017b) outlined a process for analysis of qualitative data in health economics that starts with making initial analytic design choices (around the combinations and ordering of an analysis, as well as dealing with any language issues) and choices about the management of data (e.g., through use of pen and paper, word-processing software, or dedicated qualitative analysis software). Analysis then proceeds through immersion in the data, which can be achieved by transcribing the data and/or listening to and correcting transcriptions of the data and/or repeated re-reading of transcripts. Once fully immersed, the researcher is in a position to start coding or categorizing the data. This generally starts with very open or expansive coding (Merriam & Tisdell, 2016; Strauss & Corbin, 1990), often through making notes directly onto transcripts, followed by establishing a coding structure (preferably hierarchical, and often developed through the use of visualization techniques) and coding the data into that structure. At this point, the researcher begins to apply the economics lens, being sensitive to relevant concepts arising in the data. The researcher then begins the review and refinement of their analysis, through the writing of detailed analytic accounts that bring the data back together in new ways. These often start by being very descriptive and become more interpretive over time (Al-Janabi et al., 2012; Ritchie & Lewis, 2003), as understanding is increased and theoretical insights start to develop; again, the application of the economics lens is important at this point. The rigor of writing these accounts forces analysts to confront the level of their understanding and the extent to which their emerging theory is supported by the data. Key relationships are then determined through a combination of further writing, the use of visualization techniques, and the exposure of ideas to external critique. The final stage of the analysis is further reflection and the generation of the final model, through reducing the data to key categories and linkages, and explicit application of the economic lens, drawing not just on the analytic accounts but also on other relevant evidence in the literature, which enables the researcher to think about how typical the findings are and how they relate to existing economic evidence (Coast & Jackson, 2017b).
This approach may be particularly helpful for health economists analyzing qualitative data, but there are very many choices available within the literature (Creswell, 2013; Grbich, 2013; Patton, 2015). An alternative, with which some economists may be more comfortable, is to use deductive approaches to data analysis, such as thematic or framework analysis, where the deductive approaches are either explicitly grounded in economic theory or more implicitly grounded in such theory through basing coding on interview schedules that are themselves grounded in these theories. On the other hand, there are some who advocate that health economists should draw much more clearly and explicitly on existing qualitative theoretical analytical approaches (Smith, 2017; Smith, Mitton, & Peacock, 2009). Analytical approaches may also need to differ for very specific uses of qualitative methods in health economics, as, for example, in the development of Q sets for Q methodology, where the aim is to find and list statements of opinion within the transcripts (Baker, McHugh, & Mason, 2017).
Presentation of qualitative research can be challenging for economists, as editors and reviewers may not be used to the presentational norms for qualitative research (Coast, McDonald, & Baker, 2004; Robinson, 2017), which tend to focus on the use of quotes to illustrate the findings and to provide evidence for their veracity. Nevertheless, qualitative papers are published in many health economics journals, although there are some, such as the Journal of Health Economics, that do not typically choose to publish qualitative research (Gagliardi & Dobrow, 2011). It is also notable that qualitative research in healthcare is often highly cited and influential (Greenhalgh, 2016). In terms of presenting findings, a norm within the health field is to adhere to the COREQ checklist of 32 items (Tong, Sainsbury, & Craig, 2007), and use of this checklist is also good advice for economists (and indeed is required by some journals), because it ensures that relevant information is not forgotten in the reporting of findings (Coast, 2017a).
Qualitative research invariably involves the conduct of research with human subjects and as such is almost always subject to approval by appropriate ethical review boards, who are likely to require an understanding of the aims, methods, and practicalities of the research as well as review of topic schedules (interview guides), information sheets, and written consent forms (Owen-Smith & Coast, 2017a). Ethical issues that qualitative researchers may have to consider include the confidentiality of informants, the potential for causing distress in discussions of sensitive or upsetting subjects, the potential for disclosure of issues (such as abuse), and the safety of both informants and researchers (Owen-Smith & Coast, 2017a). With regard to the latter, there may need to be lone-researcher protocols that maximize the physical safety of the respondent, as well as a concern for the researchers’ emotional health, with the use, for example, of debriefing of researchers after the conduct of research on distressing subjects. The debriefing may also need to be extended to other members of the research team, such as transcribers (Bailey et al., 2017; Owen-Smith & Coast, 2017a).
Exemplar Topics That Have Seen Growth in Use of Qualitative Research
There are a small number of areas where qualitative methods are starting to be influential in health economics, and this section provides three exemplars that illustrate how the use of qualitative methods is starting to influence the broader discipline.
Attribute Development for Preference Elicitation
The use of qualitative methods in the development of attributes for preference-elicitation studies, and indeed in the generation of outcomes measures more generally, has been rapidly increasing over recent years. Key leaders in the field of discrete choice experiments recommended the use of qualitative methods more than 15 years ago as the basis for the development of attributes that are then valued quantitatively (Louviere, Hensher, & Swait, 2000), and there has been increasing development of qualitative methods for use in this context (Coast et al., 2012; Coast & Horrocks, 2007). A recent systematic review of the use of qualitative methods in this area showed that more than half of discrete choice experiment studies published since 2001 used qualitative methods, and that where these methods were not used, it was often seen as a clear limitation of the study (Vass, Rigby, & Payne, 2017). Qualitative methods have also been used as the basis for generating outcome measures for economic evaluation in three key areas: a health-related quality of life measure for children, (Stevens, 2009, 2010, 2017; Stevens & Palfreyman, 2012), a care-related quality of life measure for unpaid careers (Al-Janabi, Coast, & Flynn, 2008), and the ICECAP suite of capability measures, currently comprising the ICECAP-O (Coast et al., 2008; Grewal et al., 2006), the ICECAP-A (Al-Janabi et al., 2012), and two measures for use at the end of life, the ICECAP Supportive Care Measure (Sutton & Coast, 2014) and the ICECAP Close Person Measure (Canaway et al., 2017a). Indeed, the choice to focus on “capability” in the ICECAP measures was driven by the findings from the first stage of qualitative research (Grewal et al., 2006). Stevens advocated that the use of qualitative methods in these contexts was more likely to ensure sensitivity to change, given that measures are designed from the start to focus on the factors that the relevant group perceive as important (Stevens, 2015). Where qualitative methods are used for this purpose, their use is driven as much by the economic requirements related to the preference elicitation as the requirements of the qualitative methodology. Detailed guidance on the use of qualitative methods in discrete choice experiments is available (Coast et al., 2012).
Priority Setting and Use of Economic Evaluation
The area of priority-setting research in health economics, including study of the use of economic evaluation in priority setting, has also seen considerable increase in use of qualitative methods over the past 20 years. Contributions in this area have focused on issues at the micro-level (Coast, 2001b; Owen-Smith, 2008; Owen-Smith, Coast, & Donovan, 2009, 2010; Owen-Smith, Donovan, & Coast, 2015), meso-level (Eddama & Coast, 2009; McDonald, 2002; Mitton, Patten, Donaldson, & Waldner, 2005; Mitton, Patten, Waldner, & Donaldson, 2003; Patten, Mitton, & Donaldson, 2006; Smith, Mitton, Dowling, et al., 2016; Smith, Mitton, Hiltz, et al., 2016), and macro-level (Bryan, Williams, & McIver, 2007; Eckard, 2016; Eckard, Janzon, & Levin, 2014; Eckard & Nedlund, 2017; Lafi, Robinson, & Williams, 2012; Williams & Bryan, 2007) of priority setting. Research has covered exploration of how priority setting happens in practice in different settings, as well as the use of different types of economic intervention, particularly in relation to the technique of program budgeting and marginal analysis. There have been calls for such research by health economists to be more theoretically aware, in terms of qualitative approaches, so as to increase the benefits of the research (Smith, 2017; Smith et al., 2009), but it is also clear that research in this area also draws on economic theory in understanding and interpreting the qualitative research findings, with particular use made of principal-agent theory (Coast, 2001a), theories of utility maximization (Coast, 2001b), and theories of efficiency generation in the context of marginal decision-making (Mitton & Patten, 2004). Research in this area has undoubtedly contributed to a better understanding of how decisions are made in practice; for example, through understanding that local prioritization often focuses on investment in specific resources, rather than intervention choice, as assumed in health economics (Eddama & Coast, 2009), enabling health economists to make stronger contributions to future priority-setting research.
Qualitative Methods in the Framework of Health Financing Interventions
In recent years, health financing interventions have largely dominated the landscape of health-policy reforms in lower- and middle-income countries (LMICs). Specifically, interventions have included the introduction and/or scale-up of health insurance schemes, user-fee removal or reduction policies, targeted exemption mechanisms, voucher schemes, and performance-based financing. A commonality across all health financing interventions is that they rely on multiple actors (e.g., policymakers, healthcare providers, communities), entail multiple components (e.g., setup of new institutions, marketing campaigns), and aim at generating change at the health-system level. These traits characterize them as complex interventions (Campbell et al., 2000; Craig et al., 2008). Therefore, the scientific community increasingly recognizes the impossibility of capturing their effects and monitoring their implementation using exclusively quantitative methods rooted in the economic tradition of evaluation.
It is against this background that a growing number of economists working in health financing in LMICs have turned to qualitative methods. A variety of qualitative designs have been employed, each fulfilling a specific research objective. Torbica et al. relied on focus-group discussions to generate information necessary to develop attributes for a best−worst scaling experiment aimed at identifying which factors guide decision-making on user-fee removal and reduction policies in sub-Saharan Africa (Torbica, De Allegri, Belemsaga, Medina-Lara, & Ridde, 2014) . Similarly, Borghi et al. used focus groups to develop attributes for a contingent valuation study in Nepal (Borghi, Shrestha, Shrestha, & Jan, 2007) and Abiiro et al. employed a mixture of focus groups and key informant interviews to develop attributes for a discrete-choice experiment in Malawi (Abiiro et al., 2014). Focus-group discussions and in-depth interviews were used across settings to explore elements associated with the decision to participate in voluntary insurance schemes (De Allegri, Sanon, & Sauerborn, 2006) as well as the distributional effects of such schemes (Aji, Yamamoto, & Sauerborn, 2014; Macha et al., 2012). Qualitative methods have also increasingly been used within the framework of mixed-methods impact evaluations of health financing interventions. In these cases, quantitative and qualitative designs are merged into a single study unit to provide the most accurate description of an intervention and its effects. These designs prove to be particularly complex because the inclusion of qualitative methods rooted in a constructivist tradition challenges the positivist approach at the core of the experimental and quasi-experimental evaluation tradition. According to Greene, these studies overtly require researchers to juggle multiple mental models at once, and by doing so, they call into question adherence to a single epistemological model (Greene, 2007). Examples of the mixed-methods approach to impact evaluations are the studies described by Brenner et al. (2014), McMahon et al. (2016), and Nimpagaritse et al. (2016), all related to the evaluation of performance-based financing interventions. Finally, a growing number of health economists have relied on qualitative methods to conduct implementation and policy analyses relevant to the introduction of health financing reforms, such as the work by Suphanchaimat et al. in Thailand (Suphanchaimat, Kantamaturapoj, Pudpong, Putthasri, & Mills, 2016) and the work by Onoka et al. in Nigeria (Onoka, Hanson, & Mills, 2016). Interestingly, qualitative studies conducted by economists with a focus on policy, such as the two above-mentioned studies, often adopt a case-study approach, merging multiple sources of qualitative data and relying on multiple qualitative analytical models.
An interesting question is why qualitative research into financing reforms seems to have been prevalent among economists working in LMICs, but has been much less prevalent in investigations of health financing interventions in higher-income countries. Insights into policy changes, such as the Affordable Care Act in the United States or the removal of prescription charges in Scotland and Wales, could potentially be afforded to economists undertaking qualitative and mixed-methods research on these topics.
Qualitative methods are not traditionally used in health economics or economics more widely, but they have been increasingly employed over the last 20 years and there are now a number of health economists with experience and expertise in these methods. Their use across the discipline is variable, with some areas making extensive use of the methods and gaining considerable insight as a result, but little use of qualitative research in other areas. In part, this is because disciplinary challenges still exist (Coast, 2017b; Coast et al., 2004), although they are not the focus here, and some economists remain suspicious of the methods (Basole & Ramnarain, 2016). Nevertheless, there is an increasing interest among health economists in utilizing the full range of social science methods available to them, to better understand issues surrounding resource allocation in the health economy (Coast, Al-Janabi, Jackson, Kinghorn, & Owen-Smith, 2017).
- Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
- Coast, J. (Ed.). (2017b). Qualitative methods for health economics. London: Rowman & Littlefield International.
- Creswell, J. W. (2013). Qualitative inquiry & research design. Choosing among five approaches. Thousand Oaks, CA: SAGE.
- Denzin, N. K., & Lincoln, Y. S. (2017). The SAGE handbook of qualitative research (5th ed.). Thousand Oaks, CA: SAGE.
- Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. Medical Research Methodology, 13, 117.
- Glaser, B. G., & Strauss, A. L. (1968). The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine.
- Ritchie, J., & Lewis, J. (2003). Qualitative research practice: A guide for social science students and researchers. London: SAGE.
- Abiiro, G. A., Leppert, G., Mbera, G. B., Robyn, P. J., & De Allegri, M. (2014). Developing attributes and attribute-levels for a discrete choice experiment on micro health insurance in rural Malawi. Health Services Research, 14, 235.
- Aji, B., Yamamoto, S. S., & Sauerborn, R. (2014). The economic impact of the insured patients with severe chronic and acute illnesses: A qualitative approach. Global Health Action, 7(1), 22526.
- Al-Janabi, H., Coast, J., & Flynn, T. N. (2008). What do people value when they provide unpaid care to an older person? A meta-ethnography with interview follow-up. Social Science and Medicine, 67, 111–121.
- Al-Janabi, H., Flynn, T. N., & Coast, J. (2012). Development of a self-report measure of capability wellbeing for adults: The ICECAP-A. Quality of Life Research, 21, 167–176.
- Al-Janabi, H., Keeley, T., Mitchell, P., & Coast, J. (2013). Can capabilities be self-reported? A think aloud study. Social Science and Medicine, 87, 116–122.
- Al-Janabi, H., & Owen-Smith, A. (2017). Understanding secondary data analysis. In J. Coast (Ed.), Qualitative methods for health research (pp. 119–138). London: Rowman & Littlefield International.
- Atkinson, J. M., & Heritage, J. (1999). Jefferson’s transcript notation. In A. Jaworski & N. Coupland (Eds.), The discourse reader (pp. 158–166). London: Routledge.
- Bailey, C., Kinghorn, P., Orlando, R., Armour, K., Perry, R., Jones, L., & Coast, J. (2016). “The ICECAP-SCM tells you more about what I’m going through”: A think-aloud study measuring quality of life among patients receiving supportive and palliative care. Palliative Medicine, 30(7), 642−652.
- Bailey, C., Kinghorn, P., Orlando, R., & Coast, J. (2017). Using “think-aloud” and interview data to explore patient and proxy completion of health and capability measures at the end of life. In J. Coast (Ed.), Qualitative methods for health economics (pp. 231–244). London: Rowman & Littlefield International.
- Baker, R., McHugh, N., & Mason, H. (2017). Constructing statement sets for use in Q methodology studies. In J. Coast (Ed.), Qualitative methods for health economics (pp. 163–174). London: Rowman & Littlefield International.
- Baker, R., Thompson, C., & Mannion, R. (2006). Q methodology in health economics. Journal of Health Services Research and Policy, 11(1), 38–45.
- Baker, R. M. (2006). Economic rationality and health and lifestyle choices for people with diabetes. Social Science and Medicine, 63, 2341–2353.
- Basole, A., & Ramnarain, S. (2016). Qualitative and ethnographic methods in economics. In F. S. Lee & B. Cronin (Eds.), Handbook of research methods and applications in heterodox economics (pp. 135–164). Cheltenham, UK: Edward Elgar.
- Berik, G. (1997). The need for crossing the method boundaries in economic research. Feminist Economics, 3(2), 121–125.
- Blaug, M. (1992). The methodology of economics. Or how economists explain. Cambridge, UK: Cambridge University Press.
- Borghi, J., Shrestha, D. L., Shrestha, D., & Jan, S. (2007). Using focus groups to develop contingent valuation scenarios—A case study of women’s groups in rural Nepal. Social Science & Medicine, 64(3), 531–542.
- Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
- Brenner, S., Muula, A. S., Robyn, P. J., Bärnighausen, T., Sarker, M., Mathanga, D. P., … De Allegri, M. (2014). Design of an impact evaluation using a mixed methods model—An explanatory assessment of the effects of results-based financing mechanisms on maternal healthcare services in Malawi. BMC Health Services Research, 14(1), 180.
- Bryan, S., Williams, I., & McIver, S. (2007). Seeing the NICE side of cost-effectiveness analysis: A qualitative investigation of the use of CEA in NICE technology appraisals. Health Economics, 16(2), 179–193.
- Bryman, A. (2004). Social research methods. Oxford: Oxford University Press.
- Campbell, M., Fitzpatrick, R., Haines, A., Kinmouth, A. L., Sandercock, P., Spiegelhalter, D., & Tyrer, P. (2000). Framework for design and evaluation of complex interventions to improve health. BMJ, 321, 694–696.
- Canaway, A., Al-Janabi, H., Kinghorn, P., Bailey, C., & Coast, J. (2017a). Development of a measure (ICECAP-Close Person Measure) through qualitative methods to capture the benefits of end-of-life care to those close to the dying for economic evaluation. Palliative Medicine, 31(1), 53–62.
- Canaway, A., Al-Janabi, H., Kinghorn, P., Bailey, C., & Coast, J. (2017b). Incorporating novel qualitative methods within health economics: The use of pictorial tools. In J. Coast (Ed.), Qualitative methods for health economics (pp. 205–216). London: Rowman & Littlefield International.
- Charmaz, K. (2014). Constructing grounded theory (2nd ed.). London: SAGE..
- Coast, J. (1999). The appropriate uses of qualitative methods in health economics. Health Economics, 1999(8), 345–353.
- Coast, J. (2001a). Citizens, their agents and health care rationing: An exploratory study using qualitative methods. Health Economics, 10, 159–174.
- Coast, J. (2001b). Who wants to know if their care is rationed? Views of citizens and service informants. Health Expectations, 4, 243–252.
- Coast, J. (2017a). Presenting and publishing qualitative research in health economics. In J. Coast (Ed.), Qualitative methods for health economics (pp. 139–151). London: Rowman & Littlefield International.
- Coast, J. (Ed.). (2017b). Qualitative methods for health economics. London: Rowman & Littlefield International.
- Coast, J., Al-Janabi, H., Jackson, L. J., Kinghorn, P., & Owen-Smith, A. (2017). Afterword: Walking the disciplinary tightrope. In J. Coast (Ed.), Qualitative methods for health economics (pp. 335–338). London: Rowman & Littlefield International.
- Coast, J., Al-Janabi, H., Sutton, E., Horrocks, S., Vosper, J., Swancutt, D., & Flynn, T. N. (2012). Using qualitative methods for attribute development for discrete choice experiments. Health Economics, 21(6), 730–741.
- Coast, J., Flynn, T. N., Natarajan, L., Sproston, K., Lewis, J., Louviere, J. J., & Peters, T. J. (2008). Valuing the ICECAP capability index for older people. Social Science and Medicine, 67, 874–882.
- Coast, J., & Horrocks, S. (2007). Developing attributes and levels for discrete choice experiments using qualitative methods. Journal of Health Services Research and Policy, 12, 25–30.
- Coast, J., & Jackson, L. (2017a). Theoretical and methodological positions and the choice to use qualitative methods. In J. Coast (Ed.), Qualitative methods for health economics (pp. 3–18). London: Rowman & Littlefield International.
- Coast, J., & Jackson, L. (2017b). Understanding primary data analysis. In J. Coast (Ed.), Qualitative methods for health economics (pp. 92–118). London: Rowman & Littlefield International.
- Coast, J., Kinghorn, P., & Owen-Smith, A. (2017). Designing qualitative research studies in health economics. In J. Coast (Ed.), Qualitative methods for health economics (pp. 19–41). London: Rowman & Littlefield International.
- Coast, J., McDonald, R., & Baker, R. (2004). Issues arising from the use of qualitative methods in health economics. Journal of Health Services Research and Policy, 9, 171–176.
- Cookson, R., & Dolan, P. (1999). Public views on health care rationing: A group discussion study. Health Policy, 49, 63–74.
- Craig, P., Dieppe, P., Macintyre, S., Mitchie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ, 337, a1655.
- Creswell, J. W. (2013). Qualitative inquiry & research design. Choosing among five approaches. Thousand Oaks, CA: SAGE.
- Davidson, C. (2009). Transcription imperatives for qualitative research. International Journal of Qualitative Methods, 8(2), 35–52.
- De Allegri, M. (2017). Using qualitative methods in impact evaluation: The case of the results-based financing for maternal and neonatal care impact evaluation in Malawi. In J. Coast (Ed.), Qualitative methods for health economics (pp. 277–292). London: Rowman & Littlefield International.
- De Allegri, M., Sanon, M., Bridges, J., & Sauerborn, R. (2006). Understanding consumers’ preferences and decision to enroll in community-based health insurance in rural West Africa. Health Policy, 76, 58–71.
- De Allegri, M., Sanon, M., & Sauerborn, R. (2006). “To enroll or not to enroll?” A qualitative investigation of demand for health insurance in rural West Africa. Social Science & Medicine, 62, 1520–1527.
- Dolan, P., Cookson, R., & Ferguson, B. (1999). Effect of discussion and deliberation on the public’s views of priority setting in health care: Focus group study. British Medical Journal, 318, 916–919.
- Eckard, N. (2016). Reaching agreement in uncertain circumstances: The practice of evidence-based policy in the case of the Swedish National Guidelines for heart diseases. Evidence & Policy, 4, 687–707.
- Eckard, N., Janzon, M., & Levin, L.-A. (2014). Use of cost-effectiveness data in priority setting decisions: Experiences from the National Guidelines for heart diseases in Sweden. International Journal of Health Policy and Management, 3(6), 323–332.
- Eckard, N., & Nedlund, A.-C. (2017). Using cost-effectiveness evidence in making priority gradings: The case of the Swedish National Guidelines for heart diseases. In J. Coast (Ed.), Qualitative methods for health economics (pp. 307–318). London: Rowman & Littlefield International.
- Eddama, O., & Coast, J. (2009). Use of economic evaluation in local health care decision-making in England: A qualitative investigation. Health Policy, 89, 261–270.
- Finch, H., & Lewis, J. (2003). Focus groups. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 170–198). London: SAGE.
- Gagliardi, A. R., & Dobrow, M. J. (2011). Paucity of qualitative research in general medical and health services and policy research journals: Analysis of publication rates. BMC Health Services Research, 11(268), 1–7.
- Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. Medical Research Methodology, 13, 117.
- Geneau, R., Massae, P., Courtright, P., & Lewallen, S. (2008). Using qualitative methods to understand the determinants of patients’ willingness to pay for cataract surgery: A study in Tanzania. Social Science and Medicine, 66, 558–568.
- Gibbs, A. (1997). Focus groups. Social Research, 19, 1–4.
- Glaser, B. G., & Strauss, A. L. (1968). The discovery of grounded theory: Strategies for qualitative research. London: Weidenfeld and Nicolson.
- Gold, R. L. (1997). The ethnographic method in sociology. Qualitative Inquiry, 3(4), 388–402.
- Grbich, C. (2013). Qualitative data analysis. An introduction (2nd ed.). London: SAGE.
- Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco: Wiley.
- Greenhalgh, T. (2016). An open letter to the BMJ editors on qualitative research. BMJ, 352(i563), 1–4.
- Grewal, I., Lewis, J., Flynn, T. N., Brown, J., Bond, J., & Coast, J. (2006). Developing attributes for a generic quality of life measure for older people: references or capabilities? Social Science and Medicine, 62, 1891–1901.
- Hammersley, M. (1992). What’s wrong with ethnography? Methodological explorations. London: Routledge.
- Hammersley, M., & Atkinson, P. (1983). Ethnography: Principles in practice. London: Tavistock.
- Hansen, E. C. (2006). Successful qualitative health research. A practical introduction. Maidenhead, UK: Open University Press.
- Husbands, S., Jowett, S., Barton, P., & Coast, J. (2017a). How qualitative methods can be used to inform model development. Pharmacoeconomics, 35(6), 607–612.
- Husbands, S., Jowett, S., Barton, P., & Coast, J. (2017b). Using non-participant observation and think-aloud to understand and improve modelling processes. In J. Coast (Ed.), Qualitative methods for health economics (pp. 217–227). London: Rowman & Littlefield International.
- Kalolo, A., Gautier, L., Radermacher, R., Stoermer, M., Jahn, A., Meshack, M., & De Allegri, M. (2017). Implementation of the redesigned Community Health Fund in the Dodoma region of Tanzania: A qualitative study of views from rural communities. International Journal of Health Planning and Management.
- King, N., & Horrocks, C. (2010). Interviews in qualitative research. London: SAGE.
- Kinghorn, P., Canaway, A., Bailey, C., & Coast, J. (2017). Use of deliberative methods to facilitate and enhance understanding of the weighting of survey attributes. In J. Coast (Ed.), Qualitative methods for health economics (pp. 257–274). London: Rowman & Littlefield International.
- Kitzinger, J. (1994). The methodology of focus groups: The importance of interaction between research participants. Sociology of Health and Illness, 16(1), 103–121.
- Kitzinger, J. (1995). Introducing focus groups. British Medical Journal, 311, 299–302.
- Kitzinger, J. (2006). Focus groups. In C. Pope & N. Mays (Eds.), Qualitative research in health care (3rd ed., pp. 21–31). Oxford: Blackwell Publishing.
- Lafi, R., Robinson, S., & Williams, I. (2012). Economic evaluation and the Jordan Rational Drug List: An exploratory study of national-level priority setting. Value in Health, 15, 771–776.
- Lawson, T. (1997). Economics and reality. London: Routledge.
- Lawson, T. (2003). Reorienting economics. Abingdon, UK: Routledge.
- Lincoln, Y. S. (1992). Sympathetic connections between qualitative methods and health research. Qualitative Health Research, 2(4), 375–391.
- Louviere, J. J., Hensher, D. A., & Swait, J. D. (2000). Stated choice methods—Analysis and application. Cambridge, UK: Cambridge University Press.
- MacDougall, C., & Baum, F. (1997). The devil’s advocate: A strategy to avoid groupthink and stimulate discussion in focus groups. Qualitative Health Research, 7(4), 532–541.
- Macha, J., Harris, B., Garshong, B., Ataguba, J. E., Akazili, J., Kuwawenaruwa, A., & Borghi, J. (2012). Factors influencing the burden of health care financing and the distribution of health care benefits in Ghana, Tanzania and South Africa. Health Policy and Planning, 27(Suppl. 1), i46–i54.
- Martínez-Álvarez, M. (2017). An assessment of an international declaration on aid effectiveness through qualitative methods at the country level. In J. Coast (Ed.), Qualitative methods for health economics (pp. 293–306). London: Rowman & Littlefield International.
- Maxwell, J. A. (2012). A realist approach for qualitative research. Thousand Oaks, CA: SAGE.
- McDonald, R. (2002). Using health economics in health services. Rationing rationally? Buckingham, UK: Open University Press.
- McMahon, S. A., Brenner, S., Lohmann, J., Makwero, C., Torbica, A., Mathanga, D. P., … De Allegri, M. (2016). Evaluating complex health financing interventions: Using mixed methods to inform further implementation of a novel PBI intervention in rural Malawi. BMC Health Services Research, 16(414), 1–11.
- Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and implementation. San Francisco: Wiley.
- Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). London: SAGE.
- Mitton, C., & Patten, S. (2004). Evidence-based priority-setting: What do the decision-makers think? Journal of Health Services Research and Policy, 9, 146–152.
- Mitton, C., Patten, S., Donaldson, C., & Waldner, H. (2005). Priority-setting in health authorities: Moving beyond the barriers. The Calgary experience. Healthcare Quarterly, 8(3), 49−55.
- Mitton, C., Patten, S., Waldner, H., & Donaldson, C. (2003). Priority setting in health authorities: A novel approach to a historical activity. Social Science and Medicine, 57, 1653–1663.
- Morse, J. M. (1994). Designing funded qualitative research. In N. K. Denzin (Ed.), Handbook of qualitative research (pp. 220–235). Thousand Oaks, CA: SAGE.
- Nimpagaritse, M., Korachais, C., Roberfroid, D., Kolsteren, P., Driss, M., Eddine, Z., … Meessen, B. (2016). Measuring and understanding the effects of a performance based financing scheme applied to nutrition services in Burundi—A mixed method impact evaluation design. International Journal for Equity in Health, 15(93), 1–11.
- Onoka, C. A., Hanson, K., & Mills, A. (2016). Growth of health maintenance organisations in Nigeria and the potential for a role in promoting universal coverage efforts. Social Science & Medicine, 162, 11–20.
- Owen-Smith, A. (2008). Knowing about healthcare rationing (Doctoral thesis, University of Bristol, Bristol, UK).
- Owen-Smith, A., & Coast, J. (2017a). Understanding data collection: Interviews, focus groups and observation. In J. Coast (Ed.), Qualitative methods for health economics (pp. 59–91). London: Rowman & Littlefield International.
- Owen-Smith, A., & Coast, J. (2017b). Understanding sampling and recruitment. In J. Coast (Ed.), Qualitative methods for health economics (pp. 42–58). London: Rowman & Littlefield International.
- Owen-Smith, A., Coast, J., & Donovan, J. (2009). “I can see where they’re coming from, but when you’re on the end of it … you just want to get the money and the drug”: Explaining reactions to explicit healthcare rationing. Social Science and Medicine, 68, 1935–1942.
- Owen-Smith, A., Coast, J., & Donovan, J. (2010). The desirability of being open about health care rationing decisions: Findings from a qualitative study with patients and clinical professionals. Journal of Health Services Research and Policy, 15(1), 14–20.
- Owen-Smith, A., Donovan, J., & Coast, J. (2015). How clinical rationing works in practice: A case study of morbid obesity surgery. Social Science and Medicine, 147, 288–295.
- Patten, S., Mitton, C., & Donaldson, C. (2006). Using participatory action research to build a priority setting process in a Canadian regional health authority. Social Science & Medicine, 63(5), 1121–1134.
- Patton, M. Q. (2015). Qualitative research & evaluation methods (4th ed.). Thousand Oaks, CA: SAGE.
- Poland, B. D. (1995). Transcription quality as an aspect of rigor in qualitative research. Qualitative Inquiry, 1(3), 290–310.
- Rapley, T. (2011). Some pragmatics of data analysis. In D. Silverman (Ed.), Qualitative research (3rd ed., pp. 273–290). Thousand Oaks, CA: SAGE.
- Ridde, V., Turcotte-Tremblay, A.-M., Souares, A., Lohmann, J., Zombre, D., Koulidiati, J. L., … De Allegri, M. (2014). Protocol for the process evaluation of interventions combining performance-based financing with health equity in Burkina Faso. Implementation Science, 9(149), 1–12.
- Riessman, C. K. (2011). What’s different about narrative inquiry? Cases, categories and contexts. In D. Silverman (Ed.), Qualitative research (3rd ed., pp. 310–330). Thousand Oaks, CA: SAGE.
- Ritchie, J., & Lewis, J. (2003). Qualitative research practice: A guide for social science students and researchers. London: SAGE.
- Ritchie, J., Spencer, L., & O’Connor, W. (2003). Carrying out qualitative analysis. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 219–262). London: SAGE.
- Robinson, A., Dolan, P., & Williams, A. (1997). Valuing health status using VAS and TTO: What lies behind the numbers? Social Science and Medicine, 45(8), 1289–1297.
- Robinson, S. (2017). Getting beyond the numbers: Using qualitative methods to explore the validity of health state valuation techniques. In J. Coast (Ed.), Qualitative methods for health economics (pp. 245–256). London: Rowman & Littlefield International.
- Sen, A. (1993). Capability and well-being. In M. C. Nussbaum (Ed.), The quality of life (pp. 30–53). Oxford: Clarendon Press.
- Smith, N. (2017). Contributions of qualitative methods to the study of priority setting and resource allocation in health care. In J. Coast (Ed.), Qualitative methods for health economics (pp. 319–332). London: Rowman & Littlefield International.
- Smith, N., Mitton, C., Dowling, L., Hiltz, M. A., Campbell, M., & Gujar, S. A. (2016). Introducing new priority setting and resource allocation processes in a Canadian healthcare organisation: A case study analysis informed by multiple streams theory. International Journal of Health Policy and Management, 5(1), 23–31.
- Smith, N., Mitton, C., Hiltz, M.-A., Campbell, M., Dowling, L., Magee, J. F., & Gujar, S. A. (2016). A qualitative evaluation of program budgeting and marginal analysis in a Canadian pediatric tertiary care institution. Applied Health Economics and Health Policy, 14(5), 559–568.
- Smith, N., Mitton, C., & Peacock, S. (2009). Qualitative methodologies in health-care priority setting research. Health Economics, 18, 1163–1175.
- Spencer, A. (2003). The TTO method and procedural invariance. Health Economics, 12, 655–668.
- Starr, M. A. (2014). Qualitative and mixed-methods research in economics: Surprising growth, promising future. Journal of Economic Surveys, 28(2), 238–264.
- Stevens, K. (2009). Developing a descriptive system for a new preference-based measure of health-related quality of life for children. Quality of Life Research, 18, 1105–1113.
- Stevens, K. (2010). Working with children to develop dimensions for a preference-based, generic, pediatric health-related quality-of-life measure. Qualitative Health Research, 20, 240–351.
- Stevens, K. (2015). How well do the generic multi-attribute utility instruments incorporate patient and public views into their descriptive systems? The Patient, 9(1), 5–13.
- Stevens, K. (2017). Using qualitative methods to develop a preference-based health-related quality of life measure for use in economic evaluation: The development of the Child Health Utility 9D. In J. Coast (Ed.), Qualitative methods for health economics (pp. 193–204). London: Rowman & Littlefield International.
- Stevens, K., & Palfreyman, S. (2012). The use of qualitative methods in developing the descriptive systems of preference-based measures of health-related quality of life for use in economic evaluation. Value in Health, 15(8), 991–998.
- Strauss, A., & Corbin, J. (1990). Basics of qualitative research. Grounded theory procedures and techniques. London: SAGE.
- Suphanchaimat, R., Kantamaturapoj, K., Pudpong, N., Putthasri, W., & Mills, A. (2016). Health insurance for people with citizenship problems in Thailand: A case study of policy implementation. Health Policy and Planning, 31(2), 229–238.
- Sutton, E., & Coast, J. (2014). Development of a supportive care measure for economic evaluation of end-of-life care, using qualitative methods. Palliative Medicine, 28, 151–157.
- Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357.
- Torbica, A., De Allegri, M., Belemsaga, D., Medina-Lara, A., & Ridde, V. (2014). What criteria guide national entrepreneurs’ policy decisions on user fee removal for maternal health care services? Use of a best−worst scaling choice experiment in West Africa. Journal of Health Services Research & Policy, 19(4), 208–215.
- Vass, C., Rigby, D., & Payne, K. (2017). The role of qualitative research methods in discrete choice experiments: A systematic review and survey of authors. Medical Decision Making, 37, 298−313.
- Vosper, J., Flynn, T., & Coast, J. (2017). Qualitative methods in discrete choice experiments. In J. Coast (Ed.), Qualitative methods for health economics (pp. 175–192). London: Rowman & Littlefield International.
- Williams, A. (1997a). One economist’s view of social medicine. In A. J. Culyer & A. Maynard (Eds.), Being reasonable about the economics of health. Selected essays by Alan Williams (pp. 38–44). Cheltenham, UK: Edward Elgar.
- Williams, A. (1997b). The role of economics in the evaluation of health care technologies. In A. J. Culyer & A. Maynard (Eds.), Being reasonable about the economics of health. Selected essays by Alan Williams (pp. 203–237). Cheltenham, UK: Edward Elgar.
- Williams, I., & Bryan, S. (2007). Cost-effectiveness analysis and formulary decision making in England: Findings from research. Social Science and Medicine, 65(10), 2116–2129.