Harvard Team Outlines Rigorous Framework for Evaluating Heart Devices With Real-World Data

Cardiovascular devices — from stents to transcatheter heart valves — are frequently adopted into clinical practice with limited randomized trial evidence. Randomized controlled trials are expensive, sometimes ethically difficult, and often impractical for device evaluations where blinding is nearly impossible. As a result, observational analyses using electronic health records, registries, and claims data have become a critical source of evidence. But their credibility has long been questioned due to confounding, selection bias, and inconsistent analytic standards.

This review from Harvard's Smith Center for Outcomes Research, published in Circulation, directly addresses these concerns. The authors outline a principled framework for planning and executing observational comparative effectiveness research on cardiovascular devices. At the core is the 'target trial framework' — a conceptual tool borrowed from causal inference methodology that asks researchers to explicitly emulate what a well-designed randomized trial would look like before analyzing observational data.

The framework organizes research activities around three guiding questions: defining the specific research question ('What do we want?'), inventorying available background knowledge, analytic methods, and data sources ('What do we have?'), and mapping out the precise analytic steps needed to answer the question ('What do we do?'). This structure forces transparency and discipline in study design, reducing the risk of post-hoc analytic decisions that can introduce bias.

The authors also emphasize the distinction between using observational data to study treatment utilization patterns and safety signals — a well-established role — versus using it to make causal comparative effectiveness claims, which is more demanding and where most methodological failures occur.

For clinicians and health-system decision-makers, the practical implication is significant: when observational studies on devices follow these principles, their conclusions about risks and benefits become meaningfully more trustworthy. This has direct relevance for FDA post-market surveillance, payer coverage decisions, and clinical guideline development. Broader adoption of these methods could raise the evidentiary bar for new cardiovascular devices entering practice.