Chimeric antigen receptor (CAR) T cell immunotherapy is promising for treatment of blood cancers; however, clinical benefits remain unpredictable, necessitating development of optimal CAR T cell products. Unfortunately, current preclinical evaluation platforms are inadequate due to their limited physiological relevance to humans. We herein engineered an organotypic immunocompetent chip that recapitulates microarchitectural and pathophysiological characteristics of human leukemia bone marrow stromal and immune niches for CAR T cell therapy modeling. This leukemia chip empowered real-time spatiotemporal monitoring of CAR T cell functionality, including T cell extravasation, recognition of leukemia, immune activation, cytotoxicity, and killing. We next on-chip modelled and mapped different responses post CAR T cell therapy, i.e., remission, resistance, and relapse as observed clinically and identify factors that potentially drive therapeutic failure. Finally, we developed a matrix-based analytical and integrative index to demarcate functional performance of CAR T cells with different CAR designs and generations produced from healthy donors and patients. Together, our chip introduces an enabling ‘(pre-)clinical-trial-on-chip’ tool for CAR T cell development, which may translate to personalized therapies and improved clinical decision-making.