meta-analysis

Table of Contents

References & Edit History Related Topics

meta-analysis

statistics

Written by Donna F. Stroup, Stephen B. Thacker•All

Fact-checked by The Editors of Encyclopaedia Britannica

meta-analysis, in statistics, approach to synthesizing the results of separate but related studies. In general, meta-analysis involves the systematic identification, evaluation, statistical synthesis, and interpretation of results from multiple studies. It is useful particularly when studies on the same or a similar subject or problem present contradictory findings, thereby challenging interpretation of the collective results. Meta-analysis is especially common in the fields of medicine and epidemiology, where it often is used to combine results from observational studies, to guide policy decisions, and to help determine the effectiveness of medical interventions.

One of the first to use meta-analysis to interpret the findings of multiple clinical studies was British statistician Karl Pearson, who in 1904 used quantitative analysis to increase statistical power in determining the efficacy of a vaccine for enteric fever. The term meta-analysis was later coined by Gene V. Glass, who in 1976 applied it specifically to describe systematic review and quantitative synthesis.

Elements of meta-analysis

Literature search

Meta-analyses typically are undertaken when a conflict in research findings is observed. The particular research problem under investigation can be framed by population, intervention (or exposure), comparison, or outcome. In order to ensure thoroughness, a systematic search for relevant studies is performed. Computerized databases have aided this step, particularly in meta-analyses of randomized controlled trials (RCTs; studies that test the effectiveness of clinical interventions by randomly assigning individuals to treatment or control groups). A comprehensive search includes multiple databases, the reference lists of recent review articles and meta-analyses, and contact with experts to find unpublished results. Owing to the specific knowledge and skills required for complex bibliographic retrieval and verification of information, science librarians typically are called upon to contribute to the search process. Science librarians play an especially important role in gathering health and information research.

Data collection

The next step in meta-analysis is to collect data from the gathered studies. A search for relevant data requires explicit, scientifically valid inclusion and exclusion criteria. Commonly used criteria include time (e.g, time period covered in a review), variables of interest and their operational definitions, study quality, and publication language.

In order to reduce bias during data abstraction, researchers may blind the data abstractor to the identity of the journal or the results. However, blinding is difficult to achieve, is time-consuming, and might not substantially alter results. In an alternative approach, multiple abstractors may assess the data.

Evaluation of evidence

Reporting of publication bias, the tendency to publish findings (or not) based on bias at the investigator or editorial level (e.g., failure to publish results of studies demonstrating negative results), is a major problem for meta-analysis. This bias can be related to the strength or implication of the results, the author’s native language or sex, or the country of publication.

Various methods have been developed to address publication bias. The funnel plot, for example, is a type of scatter plot with estimate of sample size on one axis and effect size estimate on the other. The funnel plot is used to assess publication bias based on the statistical principle that sampling error decreases as sample size increases. Other statistical tests can help assess deviation from symmetry, though they are controversial owing to their tendency for Type I errors (false positives). A more robust approach includes a comprehensive search and estimating contribution from the components of publication bias.

When studies to be combined in a meta-analysis are heterogeneous, the interpretation of any summary effect might be difficult. Statistical methods have been developed to assist with determining the source and nature of heterogeneity. Whether the heterogeneity is important, however, requires judgment beyond statistics.

Quantitative synthesis of evidence

If the studies identified are appropriate for quantitative synthesis, fixed-effects or random-effects models can be used in a meta-analysis, depending on the presence or absence of heterogeneity. The fixed-effects model applies to a situation that assumes each study result estimates a common (but unknown) pooled effect. The random-effects model assumes that each study result estimates its own (unknown) effect, which has a population distribution (having a mean value and some measure of variability). Thus, the random-effects model allows for between- and within-study variability. Nonetheless, even when using a random-effects model, summary estimates from heterogeneous studies must be interpreted with caution.

Bayesian meta-analysis, which allows both the data and the model itself to be random parameters, can also be used. Bayesian methods further allow the inclusion of relevant information external to the meta-analysis and allow for the consideration of the utility of different clinical outcomes. Owing to the latter, in the case of epidemiologic studies, Bayesian methods can facilitate the extension of meta-analysis to decision-making processes. Cumulative meta-analysis is the process of performing a new (or updated) meta-analysis as results become available.

Meta-analysis in epidemiology

In epidemiology, systematic reviews (and quantitative synthesis where appropriate) are conducted as a way of evaluating the evidence of effectiveness for new medical or health interventions. The results of a meta-analysis can be translated into a recommendation supporting the use of an intervention or a finding of insufficient evidence for its implementation. In that way, meta-analysis in epidemiology has contributed to advances in public health science.

In addition, since the completion of the sequencing of the human genome in 2003, numerous studies have assessed the impact of human genome variation on population health and the use of genetic information to improve health and prevent disease. Systematic reviews of genetic studies serve an important role in helping to ensure the quality of reporting of genotype prevalence and gene-disease association. Moreover, particularly within medical fields, meta-analysis has contributed to improvements in the reporting of scientific abstracts and primary studies, has helped identify research gaps, and has shifted attention away from statistical significance to consideration of effect size and confidence interval.

Challenges in meta-analysis

A controversial issue in meta-analysis is the question of whether to include studies that are of doubtful or poor quality. Critics argue that any meta-analysis that summarizes studies of widely differing quality is likely to be uninformative or flawed. Other researchers counter by noting that assessing methodologic quality is often difficult, and researchers often disagree on what constitutes quality. Despite a researcher’s best attempts to provide an objective measure of quality, decisions to include or exclude studies introduce bias into the meta-analysis. Still other researchers note that the quality of a study might not have an effect on the study’s outcome.

In addition, as meta-analysis has become more widely used, new methods have emerged. For example, meta-analysis is often complicated by a lack of information on standard deviation of estimates in reports; the validity of various methods of imputing this information from other sources has been studied. Combining information from different study designs or evidence on multiple parameters is of interest to researchers. Models for this situation are complex but provide opportunities to assess whether data are consistent among studies.

Stephen B. Thacker Donna F. Stroup

evidence-based medicine

Table of Contents

References & Edit History Quick Facts & Related Topics

evidence-based medicine

health care

Also known as: evidence-based health care

Written by Ray Fitzpatrick

Fact-checked by The Editors of Encyclopaedia Britannica

Article History

Also called:: evidence-based health care

Related Topics:: health care

See all related content

evidence-based medicine, approach to patient care in which decisions about the diagnosis and management of the individual patient are made by a clinician, using personal experience and expertise combined with the best, most relevant, and most up-to-date scientific information available.

Evidence-based medicine developed in the 1990s primarily out of a need to assess the reliability of a growing body of current research information and to apply new procedures and products. Although the initial impetus came from academic medicine, the idea appealed especially to funding agencies, given the prospect of the development of services that were particularly appropriate and cost-effective for the population served. As a result, evidence-based medicine received the necessary financial, managerial, and ideological support to sustain its development.

Since its emergence, evidence-based medicine has led to the generation of detailed guidelines and explicit protocols for the delivery of services, developments that in principle have made it easier to monitor and steer the performance of health professionals than was the case in the past. The provision of unbiased information about the effectiveness of interventions has been of special importance for patients who are otherwise dependent on commercial or other potentially biased sources of information and advice.

The five essential steps

The practice of evidence-based medicine emphasizes five essential steps. First, the clinician identifies a clear clinical question that arises out of the management of an individual patient; the question leads to a need for information. In the second step, the best source of evidence available to address the need is identified. Third, the evidence is critically appraised for its validity and applicability to the problem at hand. Fourth, the evidence is combined with clinical experience and the patient’s own preferences and values to determine an intervention. Fifth, the outcomes of the intervention in the patient are evaluated.

Best evidence

Central to evidence-based medicine is the use of the best possible evidence in diagnostic and treatment decisions, where best is defined by a hierarchy of quality-of-study designs providing evidence. The most-reliable evidence is generated by systematic reviews of randomized controlled trials (RCTs), which minimize bias and allow for causal interpretations of new interventions. Properly designed RCTs, in which study subjects are assigned by chance to either the new intervention or the standard treatment, themselves represent the next-most-reliable level of evidence. Below RCTs are well-designed cohort or case-control analytic studies, which allow for observational (but not causal) interpretations. Less-reliable evidence can be obtained from quasi-experimental multiple time series designs, which differ from other quasi-experimental designs in that they include a comparison group that did not receive the intervention. Least reliable of all, and therefore at the bottom of the hierarchy, is evidence in the form of the opinions of respected authorities, regardless of whether those opinions are based on clinical experience, descriptive studies, case reports, or reports of expert committees.

Technical, political, and practical issues

Technical problems

Evidence-based medicine has drawn attention to important issues in medicine, some of which have hindered its acceptance. For example, some important questions in health care may never be resolved by RCTs for practical reasons. That may occur when adverse events are so infrequent that trials would require impossibly large sample sizes or when health outcomes lie so far in the future that maintaining a trial would be impractical. In the field of critical care, RCTs are embedded within ethical concerns. In the past there was debate about whether RCTs should be the gold standard in proving an evidence base for practice. Another important technical problem is the relevance of results from clinical trials and systematic reviews to decisions about individual patients. The research evidence is usually about the average effect of an intervention across all types of patients. The extent to which that average effect is applicable to individual patients, however, may be unclear.

Political critiques

A second challenge facing the development of evidence-based medicine stemmed from political critiques. One powerful analysis argued that evidence-based medicine represents a fundamental and undesirable erosion of professional autonomy of health professionals, especially physicians. Some observers argued that the reduction of clinical decisions to explicit guidelines and protocols results in the practice of “cookbook” medicine with important decisions taken at much higher levels in the overall management of health care organizations. Higher-level priorities raised the possibility that business models of efficiency and cost control, rather than the interests of patients, would drive the field. Such critiques viewed evidence-based medicine as one of a number of managerially led developments that encourage the deprofessionalization of medicine, motivated either by the profit motives of business or by government concerns to control costs of the welfare state. It is also argued that significant investments by major organizations in smaller bodies, particularly those that were involved in the assessment of the evidence base for health care interventions, were the outcome of a conflict for power and resources won by statisticians, accountants, and economists over traditionally powerful groups such as the medical profession. The scale, scope, and reality of such scenarios were somewhat exaggerated. Moreover, the political critique underplayed the extent to which the overall goals of evidence-based medicine were welcomed by practicing clinicians and overlooked the extent of collaboration of clinicians with statistical and other nonclinical disciplines in developing the scientific underpinnings of the field.

Practical challenges

A third issue surrounding evidence-based medicine was whether it was an approach to medicine that was feasible to implement in practice. In the early 2000s studies in countries worldwide, including Australia, Canada, New Zealand, and the United Kingdom, suggested that only a minority of clinicians used evidence-based information resources such as the Cochrane Library, the primary database of systematic reviews. One survey found that only 4 percent of a sample of U.K. general practitioners had ever used the Cochrane Library to help in clinical decisions. Studies identified a range of reasons for the relative lack of uptake. Many clinicians, for example, were unaware of what constituted high-quality forms of evidence and continued to rely on traditional reviews and textbooks. Clinicians were sometimes unaware of how to access systematic reviews. Because of heavy workloads, many simply did not have the time to address evidence-based approaches.

By the second decade of the 21st century, however, evidence-based medicine had entered into a mature phase, where the complexities of what constituted good evidence were accepted and the difficulties of applying evidence to individual practice were acknowledged and addressed. Constant advances in information technology encouraged optimism that increasing numbers of feasible applications of evidence-based medicine would emerge.

Ray Fitzpatrick The Editors of Encyclopaedia Britannica