Electronic medical records (EMRs) are essential in clinical practice. Although current medical large language models (LLMs) excel in tasks like US Medical Licensing Examination, they struggle with real-world clinical applications due to insufficient large-scale EMR data in their training, hindering their clinical expertise. To address this limitation, we proposed EMR-LLM, an LLM for clinical practice using EMRs. Firstly, we continually pre-trained a general LLM on medical corpora to enhance its domain knowledge. Then, we designed three categories of instruction tasks using EMRs: structure understanding, numerical understanding, and downstream tasks. Finally, we introduced an ability-boosting instruction-tuning method, which mimics human learning, progressing from simple to complex tasks while introducing a data replay strategy to retain learned knowledge. Experimental results demonstrated that EMR-LLM outperformed strong competitors on six EMR tasks, nine medical benchmarks, and three open-domain benchmarks. Moreover, in discharge summary generation, EMR-LLM achieved performance levels close to those of expert clinicians.