Contemporary machine learning models for computer vision, although abundant, are largely inappropriate for clinical diagnostics. Clinical sophistication must address data consistency, avoid large parametric needs to reduce model complexity, and achieve stable generalizability across new patient data. Here, we achieve these goals in SIENNA a “Lightweight Energy-efficient Adaptive Next generation” artificial intelligence (LEAN AI) platform along with development of new algorithms for DICOM data consistency and approaches for improved integration of clinical data with deep learning architectures. Applied in the context of brain tumor diagnostics, SIENNA is a nimble AI that requires 175K-285K trainable parameters, 122X less in comparison to other state-of-the-art AI ML tumor models, while outperforming these models. SIENNA is generalizable across diverse patient datasets in inductive tests on benchmark and clinical datasets, achieving high average accuracies of 93–96% in three-way multiclass classification of MRI tumor data, across mixed 1.5 and 3.0 Tesla data and machines. We apply no DICOM MRI data preprocessing beyond data consistency while achieving a parameter-efficient generalizable ML pipeline. SIENNA demonstrates that small clinical datasets can be sufficient to design robust clinical ready architectures to facilitate expanded ML applications in multimodal data integration in a wider range of clinical diagnostic tasks.