In the new issue of the Journal of Infectious Diseases, a group of researchers headed by Peter Gilbert take a statistical tour around the results of the RV144 HIV vaccine efficacy trial in Thailand, in order to draw lessons for future research. When the trial results were first announced, there was some controversy over whether the analysis presented – called the modified intent-to-treat (MITT) – was appropriate. The Gilbert paper echoes many commentators at the time by concluding that this is, in fact, the standard and appropriate primary analysis for a vaccine efficacy trial, as it excludes individuals who were subsequently found to be HIV-infected at baseline (who are included in the strict intent-to-treat analysis).
However, using Bayesian statistics, the researchers show that there is considerably more uncertainty about the robustness of the 31% efficacy estimate than has been cited to date in public discourse about the trial. They write: “these considerations lead to our conclusion that the RV144 data provide moderate evidence of low-level positive VE [vaccine efficacy]– with ≥22% chance remaining for no efficacy under a range of prior assumptions− an inference that reflects greater uncertainty than has much of the discussion about this trial.” Because Bayesian statistics can consider the impact of prior beliefs, the paper is able to also note that “the skeptic who assigns prior chance of 90% that the vaccine is ineffective will conclude after seeing the RV144 data that there remains a 70% chance that the vaccine is ineffective.”
Despite the uncertainty about the results, the researchers take pains to emphasize that the evidence of efficacy is sufficiently compelling to justify confirmatory trials. They suggest that the use of Bayesian statistical analyses be considered in these trials, and also argue for additional efforts to improve adherence to vaccine regimens, noting that the per-protocol analyses of RV144 – which is limited to only those participants who received immunizations on the exact schedule (or very close to it) – was underpowered and difficult to interpret because of a dramatic 24% reduction in the analyzed population.
J Infect Dis. (2011) 203 (7): 969-975.
doi: 10.1093/infdis/jiq152
Peter B. Gilbert1,2, James O. Berger4, Donald Stablein5, Stephen Becker3, Max Essex7, Scott M. Hammer9, Jerome H. Kim6 and Victor G. DeGruttola8
1Vaccine Infectious Disease Division, Fred Hutchinson Cancer Research Center
2Department of Biostatistics, University of Washington
3Bill and Melinda Gates Foundation, Seattle, Washington
4Department of Statistical Science, Duke University, Durham, North Carolina
5The EMMES Corporation
6Department of Molecular Virology and Pathogenesis, Division of Retrovirology, Walter Reed Army Institute of Research, US Military HIV Research Program, Rockville, Maryland
7Harvard School of Public Health AIDS Initiative, Department of Immunology and Infectious Diseases
8Department of Biostatistics, Harvard School of Public Health, Boston, Massachusetts
9Division of Infectious Diseases, Columbia University, College of Physicians and Surgeons, New York, New York
Reprints or correspondence: Peter Gilbert, PhD, Vaccine Infectious Disease Division, Fred Hutchinson Cancer Research Center, 1100 Fairview Ave N, PO Box 19024, Seattle, WA 98109.
Abstract
Recently, the RV144 randomized, double-blind, efficacy trial in Thailand reported that a prime-boost human immunodeficiency virus (HIV) vaccine regimen conferred ∼30% protection against HIV acquisition. However, different analyses seemed to give conflicting results, and a heated debate ensued as scientists and the broader public struggled with their interpretation. The lack of accounting for statistical principles helped flame the debate, and we leverage these principles to provide a more scientific interpretation. We first address interpretation of frequentist results, including interpretation of P values, synthesis of results from multiple analyses (ie, intention-to-treat versus per-protocol/fully immunized), and accounting for external efficacy trials. Second, we address how Bayesian statistics, which provide clearly interpretable statements about probabilities that the vaccine efficacy takes certain values, provide more information for weighing the evidence about efficacy than do frequentist statistics alone. Third, we evaluate RV144 for completeness of end point ascertainment and integrity of blinding, necessary tasks for establishing robustly interpretable results.
Comments