Background: Agreement (and disagreement) assessments are essential steps in the evaluation of new and existing methods. We aimed to provide a statistical approach to assess systematic disagreement between two measures/methods when both are attended by random error and high variability.
Methods: We applied ordinary least products (OLP) regression and the Bland-Altman method in six simulated pairs of samples. In OLP regression, fixed bias defined if 95% confidence intervals (CIs) of the intercept did not include 0. Proportional bias was defined if 95% CIs of the slope did not include 1. As a comparator, we assessed fixed and proportional bias by the Bland-Altman method.
Results: We found divergence between studied statistical method outcomes only for measures with low variability (coefficient of variation, CV < 25,0%).
Conclusion: OLP regression is a simple and powerful tool for detecting systematic disagreement when the measures are attended by high variability, as well as behavioral variables.