There are many potential sources of missing data in a systematic review or meta-analysis (see ). For example, a whole study may be missing from the review, an outcome may be missing from a study, summary data may be missing for an outcome, and individual participants may be missing from the summary data. Here we discuss a variety of potential sources of missing data, highlighting where more detailed discussions are available elsewhere in the Handbook.
Whole studies may be missing from a review because they are never published, are published in obscure places, are rarely cited, or are inappropriately indexed in databases. Thus review authors should always be aware of the possibility that they have failed to identify relevant studies. There is a strong possibility that such studies are missing because of their ‘uninteresting’ or ‘unwelcome’ findings (that is, in the presence of publication bias). This problem is discussed at length in Chapter 10. Details of comprehensive search methods are provided in Chapter 6.
Some studies might not report any information on outcomes of interest to the review. For example, there may be no information on quality of life, or on serious adverse effects. It is often difficult to determine whether this is because the outcome was not measured or because the outcome was not reported. Furthermore, failure to report that outcomes were measured may be dependent on the unreported results (selective outcome reporting bias; see Chapter 8, Section 8.13). Similarly, summary data for an outcome, in a form that can be included in a meta-analysis, may be missing. A common example is missing standard deviations for continuous outcomes. This is often a problem when change-from-baseline outcomes are sought. We discuss imputation of missing standard deviations in Section 16.1.3. Other examples of missing summary data are missing sample sizes (particularly those for each intervention group separately), numbers of events, standard errors, follow-up times for calculating rates, and sufficient details of time-to-event outcomes. Inappropriate analyses of studies, for example of cluster-randomized and cross-over trials, can lead to missing summary data. It is sometimes possible to approximate the correct analyses of such studies, for example by imputing correlation coefficients or standard deviations, as discussed in Section 16.3 for cluster-randomized studies and Section 16.4 for cross-over trials. As a general rule, most methodologists believe that missing summary data (e.g. “no usable data”) should not be used as a reason to exclude a study from a systematic review. It is more appropriate to include the study in the review, and to discuss the potential implications of its absence from a meta-analysis.
It is likely that in some, if not all, included studies, there will be individuals missing from the reported results. Analyses of randomized trials that do not include all randomized participants are not intention-to-treat (ITT) analyses. It is sometimes possible to perform ITT analyses, even if the original investigators did not. We provide a detailed discussion of ITT issues in Section .
Missing data can also affect subgroup analyses. If subgroup analyses or meta-regressions are planned (see Chapter 9, Section 9.6), they require details of the study-level characteristics that distinguish studies from one another. If these are not available for all studies, review authors should consider asking the study authors for more information.