Water conservation (WC) is a critical ecological service function in the Yellow River Basin (YRB). There is currently a lack of detailed exploration of WC development processes and the impact mechanisms of driving factors at spatiotemporal scales in the YRB. By collecting data on DEM, land use, soil, meteorology, reservoirs, and observed discharge, this study established a large-scale WC model using the Soil and Water Assessment Tool (SWAT). The abrupt change test, empirical orthogonal function (EOF), wavelet analysis, hierarchical partitioning analysis (HPA), geodetectors, and aridity index were employed to analyze the multi-spatiotemporal characteristics and driving forces of WC calculated using the water balance method. The results are as follows: (1) The average WC among the YRB was 9.11 mm (74.68 × 108 m3) from 1960 to 2020. Pasture and forests contributed to 48.65% and 22.05% of the average annual WC, respectively. (2) WC exhibited four forms: less/more in the YRB, more in the southeast (northwest), and less in the northwest (southeast). (3) Forests and pastures in land use had higher average WC capacity, while Gansu, Shaanxi, and Qinghai ranked in the top three for average WC among the nine provinces. (4) Precipitation was the major driving force affecting WC variations, with the interaction between precipitation and actual evapotranspiration being the most significant. (5) Drought was a significant cause of negative WC. Protecting and managing crucial WC areas was essential for improving the ecological environment. This research elucidates the driving forces of WC in the YRB, providing scientific support for improving regional WC and promoting sustainable development.