Transcriptome-wide association studies (TWAS) have shown great promises in extending GWAS loci to a functional understanding of disease mechanisms. In an effort to fully unleash the TWAS and GWAS information, we propose MTWAS, a statistical framework that partitions and aggregates cross-tissue and tissue-specific genetic effects in identifying gene-trait associations. Different from previous methods, we introduce a non-parametric imputation strategy to augment the inaccessible tissues, which allows for barren conditions, such as complex interactions and non-linear expression data structure across tissues. We further classify eQTLs into cross-tissue eQTLs (ct-eQTLs) and tissue-specific eQTLs (ts-eQTLs) via a step-wise procedure based on the extended Bayesian information criterion, which was consistent under high-dimensional settings. We have shown that MTWAS significantly improves the imputation accuracy across all 47 GTEx tissues compared with other single-tissue and multi-tissue methods, such as PrediXcan and UTMOST. MTWAS also identifies more predictable genes that can be replicated with independent studies. Applications to 84 UKBB GWAS studies have provided novel insights into disease etiology. The R package implementing MTWAS is available at https://github.com/szcf-weiya/MTWAS.