Decision making strategies in the face of conflicting or uncertain sensory input have been successfully described by a drift diffusion to bound model (DDM) in many different species. Here we analyze large behavioral datasets of larval zebrafish engaged in a ‘coherent dot’ optomotor assay and compare DDM performance with trial-to-trial variation in our experimental results. We find a clear discrepancy in the low-level structure of modeled and experimental data, which compels us to implement a critical extension to the classic DDM. To correct for the discrepancy, we propose to add an extra variable that explicitly controls the transition between an engaged and a disengaged state. This addition not only accurately captures the lower-level structure in our data, but also provides a framework to explain general behavioral switches between different tasks and contextual priorities. We use these insights to define two behavioral performance metrics for the zebrafish, labeled ‘focus’ and ‘competence’, which describe two orthogonal aspects of performance in a task. We further show that the extent of ‘focus’ is largely inherited from the parents, while ‘competence’ is influenced mainly by environmental context. This quantitative framework for analyzing decision making can be used to screen genetic perturbations for their impact on these two aspects of performance, and eventually help identify a genetic basis, and a neural mechanism for attention, that extends across organisms.