Missing data is a design decision, not a cleaning step

Missing data usually arrives on the agenda late, somewhere between database lock and the first draft of the tables, framed as a technical question about which imputation to apply. By then the important decision has already been made by default.

How absent observations are handled is not a step applied to an analysis. It is part of what the analysis estimates. Two teams can run the same model on the same data, differ only in their handling of dropouts, and produce two answers to two genuinely different questions.

Absence has a shape, and the shape matters

Before any method is chosen, it is worth drawing the pattern. Subjects down, visits across, a mark where a value exists.

Most trials produce something close to monotone missingness: a subject is present until a point and absent from then on. That is withdrawal, and it usually carries a reason. A smaller set of gaps are intermittent, where a subject misses one visit and returns. That is a missed appointment, and it usually does not.

The two are not interchangeable. Intermittent gaps are frequently close to ignorable, because the reason for the gap has little to do with the outcome. Monotone dropout rarely is, because people leave trials for reasons connected to how they are doing. Applying one method to both, which is the default in most analysis plans we review, treats the informative case with machinery designed for the harmless one.

Three assumptions, only one of which the data can speak to

The standard taxonomy is worth restating precisely, because it is often stated loosely.

Data are missing completely at random when the probability of being missing is unrelated to anything, observed or not. This is testable, in a limited way, and it is almost never true.

Data are missing at random when the probability of being missing depends only on things you observed. This is the assumption behind mixed models for repeated measures and behind multiple imputation, and it is not testable. Nothing in the data distinguishes it from the third case.

Data are missing not at random when the probability depends on the value that is missing. A patient who stops attending because the treatment is not working is the canonical example, and it is not a rare one.

The practical consequence is that the primary analysis rests on an assumption the dataset cannot confirm. That is acceptable, and it is normal. What is not acceptable is leaving it unstated, because a reviewer who cannot see the assumption cannot judge how much the result depends on it.

Every missing data method is an assumption about the values you did not observe. The method is the easy part. The assumption is the part that needs defending.

Last observation carried forward is not the safe choice

The habit of treating carry-forward as conservative persists, and it deserves to be retired. Carrying the last value forward assumes the subject stopped changing at the moment they left, which is a strong claim and usually a false one.

In a progressive condition where the untreated course is decline, carrying forward a value from before the decline flatters the arm with more dropouts. That is the opposite of conservative. In a trial where response improves over time, the same method penalises it. The direction of the bias depends on the disease and the dropout pattern, which means calling it conservative is not a simplification, it is a guess.

It also understates uncertainty in every case, because an imputed value is treated as though it were measured. The standard error does not know the difference.

The estimand framework made this explicit, which is the point

ICH E9(R1) did not introduce a new statistical method. It asked sponsors to say what they are estimating, including what happens when a patient stops treatment or takes a rescue medication.

That reframing puts missing data where it belongs. A treatment policy strategy says the outcome is what it is regardless of what the patient did afterwards, which means data after discontinuation are wanted, not missing. A hypothetical strategy says you want the outcome as if the intercurrent event had not happened, which makes those observations irrelevant even if collected. The same raw dataset supports both, and the difference between them is not a modelling choice, it is a question about what the trial is for.

Once the estimand is written down, most arguments about imputation resolve themselves, because it becomes clear which values are genuinely unknown and which are simply not part of the question.

Sensitivity analysis is the argument, not the appendix

Because the primary assumption is untestable, the result is only as strong as the demonstration that it does not depend on the assumption holding exactly.

A tipping point analysis is the clearest form of that argument. Rather than presenting one alternative and declaring the result robust, it asks how badly the missing outcomes would have to differ from the assumed ones before the conclusion changes, and then reports that threshold. If the trial only fails when dropouts in the treatment arm would have done dramatically worse than anyone observed, that is a strong result. If a modest departure flips it, everyone should know that before the filing.

The threshold is a number a clinician can evaluate. A table of three alternative imputations, all producing similar estimates, mostly demonstrates that similar assumptions produce similar answers.

Prevention beats every method

None of this is as valuable as not losing the data. Retention is a protocol and operations question, and it is the one lever that improves the analysis without requiring an assumption.

Two things repay the effort more than any statistical technique. Collecting the reason for discontinuation, in enough structure to be analysed rather than as free text, gives the sensitivity analysis something real to condition on. And continuing to collect outcomes after a patient stops treatment, where the estimand calls for it and the patient consents, converts values that would have been imputed into values that were measured.

Both of those are decided when the protocol is written. That is the point at which missing data is a design question, and it is the last point at which the cheap options are still available.