The advent of the Transformer has significantly altered the course of research in Natural Language Processing (NLP) within thedomain of deep learning, making Transformer-based studies the mainstream in subsequent NLP research. There has alsobeen considerable advancement in domain-specific NLP research, including the development of specialized language modelsfor medical. These medical-specific language models were trained on medical data and demonstrated high performance. Whilethese studies have treated the medical field as a single domain, in reality, medical is divided into multiple departments, eachrequiring a high level of expertise and treated as a unique domain. Recognizing this, our research focuses on constructinga model specialized for cardiology within the medical sector. Our study encompasses the creation of open-source datasets,training, and model evaluation in this nuanced domain.