Experiencing stress during sensitive periods of brain development has a major impact on how individuals cope with later stress. Although many become more prone to develop anxiety or depression, some appear resilient. The mechanisms underlying these differences are unknown. Key answers may lie in how genetic and environmental stressors interact to shape the circuits controlling emotions. Here we studied the role of the habenulo-interpeducuncular system (HIPS), a critical node of reward circuits, in early stress-induced anxiety. We found that a subcircuit of this system, characterized by Otx2 expression, is particularly responsive to chronic stress during puberty, which induces HIPS hypersensitivity to later stress and susceptibility to develop anxiety. We further show that Otx2 deletion restricted to the HIPS counteracts these effects of stress. Together, these results demonstrate that Otx2 and stress interact, around puberty, to shape the HIPS stress-response, revealed here as a key modulator of susceptibility/resilience to develop anxiety.