This article discusses how four types of program evaluations can be used in social service programs: (a) needs assessments, (b) process evaluations, (c) outcome evaluations, and (d) cost-efficiency evaluations. Different approaches to evaluation are also reviewed, including culturally appropriate evaluations. The future of program evaluation within the social work profession is also discussed along with various trends.
Peter Gabor, Richard M. Grinnell Jr., and Yvonne A. Unrau
Mental illnesses are highly prevalent and can have considerable, enduring consequences for individuals, families, communities, and economies. Despite these high prevalence rates, mental illnesses have not received as much public policy commitment or funding as might be expected. One result is that mental illness often goes unrecognized and untreated. The resultant costs are felt not only in healthcare systems, but across many other sectors, including housing, social care, criminal justice, welfare benefits, and employment. This article sets out the basic principles of economic evaluation, with illustrations in this mental health context. It also discusses the main practical challenges when conducting and interpreting evidence from such evaluations. Decisions about whether to spend resources on a treatment or prevention strategy are based on whether it is likely to be effective in avoiding, reducing, or curing symptoms, improving quality of life, or achieving other individual-level outcomes. The economic evaluation question is whether the outcomes achieved are sufficient to justify the cost that is incurred in delivering the intervention. An economic evaluation has five elements: clarification of the question to be addressed; specification of the intervention to be evaluated and with what alternative it is being compared; the outcomes to be measured; the costs to be measured (including the cost of implementing the intervention and any savings that might accrue); and finally, how outcome and cost findings are to be blended to make a recommendation to the decision-maker. Sometimes, if an evaluation finds that one intervention has better outcomes but higher costs, then the evaluation should also how one (the outcomes) might be trade-off for the other (the costs). The article illustrates how economic evaluations have been undertaken and employed to address a range of questions, from the very strategic issue to the more specific clinical question. The purpose of the study can, to some extent, determine the type of evaluation that is needed. Examples of evaluations are given in a number of areas: perinatal maternal mental illness; parenting programs for conduct disorder; anti-bullying programs in schools; early intervention services for psychosis; individual placement and support; collaborative care for physical health problems; and suicide prevention. The challenges of economic evaluation are discussed, specifically in the mental health field.
Cost-benefit analysis (CBA) is a widely used economic appraisal method that aims to support politicians in making decisions about projects and policies. Several researchers have tried to uncover the extent to which CBA actually impacts decision-making by investigating the statistical relation between the results of CBA studies and political decisions. Although these studies show that there is no significant statistical relation between the outcomes of CBA studies and political decisions, there is clear evidence that the institutionalization of CBA affects the planning and decision-making process within the bureaucracy. Civil servants, for instance, use CBAs to government projects in the early phases of the planning process. The literature identifies various barriers that hamper politicians’ use of CBA when forming their opinion. First, politicians often receive results of CBA studies too late in the process. When politicians receive a CBA after they already made up their mind and communicated their viewpoint, the chance is low that the results of the CBA will (substantially) influence their decision. A second important barrier that limits the use of CBA by politicians is that they do not have enough trust in CBA’s impartiality. A third barrier is that politicians contest value judgments implicit in CBA. The literature distinguishes six ideological value judgments that inevitably need to be made when conducting a CBA: (a) Which individuals have standing in a CBA? (b) Which preferences have standing in a CBA? (c) Which procedure is used to value impacts? (d) On which dimensions are standard numbers differentiated? (e) Which weight is assigned to preferences of individuals in the social welfare function? (f) Which approach is adopted to select the social discount rate? The implication of the fact that CBA analysts cannot escape from making value judgments when conducing the study is that CBA is currently a problematic tool for democratic decision-making because, when applied in practice, the analysis is based on a specific set of politically loaded premises that fosters (damages) the interests of politicians (not) endorsing these premises. It is possible to overcome this problem through informing politicians about the extent to which switching value judgements leads to different CBA outcomes. The introduction of so-called normative sensitivity analyses safeguards that politicians with different belief systems are equally equipped to use the results of a CBA to arrive at a well-founded evaluation of a government project.
Massimo Florio and Chiara Pancotti
In economics, infrastructure is a long-term investment aimed at the delivery of essential services to a large number of users, such as those in the field of transport, energy, or telecommunications. A research infrastructure (RI) is a single-sited, distributed, virtual, or mobile facility, designed to deliver scientific services to communities of scientists. In physical sciences (including astronomy and astrophysics, particle and nuclear physics, analytical physics, medical physics), the RI paradigm has found several large-scale applications, such as radio telescopes, neutrino detectors, gravitational wave interferometers, particle colliders and heavy ion beams, high intensity lasers, synchrotron light sources, spallation neutron sources, and hadrontherapy facilities. These RIs require substantial capital and operation expenditures and are ultimately funded by taxpayers. In social cost–benefit analysis (CBA), the impact of an investment project is measured by the intertemporal difference of benefits and costs accruing to different agents. Benefits and costs are quantified and valued through a common metric and using the marginal social opportunity costs of goods (or shadow price) that may differ from the market price, as markets are often incomplete or imperfect. The key strength of CBA is that it produces information about the project’s net contribution to society that is summarized in simple numerical indicators, such as the net present value of a project. For any RIs, consolidated cost accounting should include intertemporal capital and operational expenditure both for the main managing body and for experimental collaborations or other external teams, including in-kind contributions. As far as social intertemporal benefits are concerned, it is convenient to divide them into two broad classes. The first class of benefits accrue to different categories of direct and indirect users of infrastructure services: scientists, students, firms benefiting from technological spillovers, consumers of innovative services and products, and citizens who are involved in outreach activities. The empirical estimation of the use value of an RI depends on the scientific specificities of each project, as different social groups are involved to different degrees. Second, there are benefits for the general public of non-users: these benefits are associated with social preferences for scientific research, even when the use of a discovery is unknown. In analogy with the valuation of environmental and cultural goods, the empirical approach to non-use value aims at eliciting the willingness to pay of citizens for the scientific knowledge that is created by an RI. This can be done by well-designed contingency valuation surveys. While some socio-economic impact studies of RIs in physics have been available since the 1980s, the intangible nature of some benefits and the uncertainty associated with scientific discoveries have limited the diffusion of CBA in this field until recently. Nevertheless, recent studies have explored the application of CBA to RIs in physics. Moreover, the European Commission, the European Strategy Forum on Research Infrastructures, the European Investment Bank, and some national authorities suggest that the study of social benefits and costs of RIs should be part of the process leading to funding decisions.