Cloth-Changing person Re-identification(CC-ReID) aims at retrieving the target person despite changes in clothing. However, this task faces some challenges: (1) CC-ReID involves considerable within-class variability due to clothing changes, while between-class variations can be subtle. (2) There's a need to explore more discriminative fine-grained features. Existing research has primarily tackled these issues by integrating cross-modality information (\emph{e.g.}, silhouettes, gaits, and 3D shapes) using offline models to extract biologically meaningful features. However, the effectiveness of these models heavily depends on the accuracy of the incorporated cross-modality information. In response to these challenges, we first illustrate that contrastive learning is suitable for CC-ReID and we further propose the Local-Global Interaction Attention (LGIA) framework. LGIA is a unified contrastive learning framework designed to extract discriminative and robust pedestrian representations. It incorporates the Identity-Cloth Center Aware Constraint and Global-Local Mutual Learning. Specifically, we leverage a combination of CNN-based and Transformer-based models to extract local-aware and global-aware fine-grained representations, respectively. To mitigate the impact of clothing changes, we introduce the Identity-Cloth Aware Center Constraint module between these two branches. Within this module, a memory bank is employed to regularize instance features concerning identity-class centers and clothing-class centers, spanning from low-level to high-level features. Additionally, we design Local-Global Mutual Learning to enhance the representations extracted by CNN and Transformer branches by exchanging multi-semantic information. To validate the effectiveness of our method, we conducted evaluations on three publicly available CC-ReID datasets. Extensive experiments demonstrate that our approach significantly outperforms existing state-of-the-art methods for cloth-changing person re-identification. We will release our code for reproduction.