HIV-1 cure requires preventing viral rebound after treatment interruption, but quantitative criteria defining the rebound-competent reservoir are lacking. We studied individuals undergoing observational treatment interruption without confounding interventions to identify virologic and immunologic determinants of rebound. In 9 of 13 participants, rebound viruses were genetically identical or similar to proviruses in circulating resting CD4+ T-cells. We found no evidence of recombination among rebound sequences. Instead, resistance to autologous neutralizing antibodies (aNAbs) was a critical determinant of viral rebound. Increased suppression of viral outgrowth by contemporaneous IgG isolated from plasma was correlated with longer time to rebound. Using inhibitory potential (IP), the log reduction in single-round infection at physiologic IgG concentrations, we defined quantitative limits governing rebound-competency with respect to contemporaneous aNAbs. Contemporaneous IgG antibodies inhibited different reservoir variants with a wide range of IP values (0.4 to 8.2 logs), whereas rebound viruses were minimally inhibited (0.5 to 2.8 logs), indicating that inhibition by even up to 2.8 logs (631-fold) cannot prevent rebound. Longitudinal analyses revealed that waning aNAb potency over time on antiretroviral therapy (ART) allows previously neutralized variants to gain rebound potential, consistent with the finding that rebound can come from variants deposited in the reservoir at different pre-ART time points. Thus, rebound competency is a dynamic, immune-governed property defined by quantitative immunologic constraints, including those exerted by aNAbs.