The frequent sand-dust weather in inland areas has severely affected the local outdoor computer vision applications. Different from the previous ideas of the sand-dust image enhancement algorithm, this paper proposes a generative adversarial network to enhance the sand-dust images. We improve the classic pix2pix network by introducing the dual attention mechanism to the U-net architecture and improve the loss function of the generator through Smooth L1 and SSIM to further enhance the color reproduction, detail features, the structural similarity, and the convergence speed of the generator. In addition, we also publish the first artificially synthesized sand-dust image data set online. The experimental results show that the enhancement method proposed in this paper has obvious advantages in both artificially synthesized images and natural real images, compared with the current traditional sand-dust enhancement algorithms and the previous network models.