Aqueous batteries (ABs) have attracted increasing attention because of their inherent safety and low cost. Nevertheless, hydrogen evolution reaction (HER) at the anode presents severe challenges for stable and safe operation of ABs. Instead of passivating the anode surface to hinder HER kinetics, a novel design strategy is proposed here to suppress the HER via alternating its thermodynamics pathway. By adding a hydrogen bond acceptor, dimethyl sulfoxide (DMSO), the onset potential of HER can be delayed by as much as 1.0 V (on titanium mesh). Spectral characterization and molecular dynamics simulation confirm that the formation of hydrogen bonds between DMSO and water molecules can reduce the water activity, thereby suppressing the HER. This strategy has proven to be universal in expanding the electrochemical window of aqueous electrolytes. For instance, unconventional V2+↔V3+ redox processes in Na3V2(PO4)3 (-1.2 V versus Ag/AgCl) and highly stable Zn plating/stripping processes can be realized in ABs.