Cartoonization of face images is a new art forms applicable to various scenes, but there are problems in the research of incomplete extraction of image details after style transfer, simple superposition between domains, and poor generation quality. Since StyleGAN has better results in the generation of artistic images, on this basis, this paper proposes a new EI-StyleGAN to construct a network model applicable to face style migration, in which the inner and outer style restriction modules are introduced to obtain different detailed features in the images of the two domains, respectively. Meanwhile, the latent space coding is obtained by image inversion before inputting into the generated model, which can perform the transformation between images between the source and target domains. The whole process adopts an incremental generation strategy to smoothly transform the generation space of the model to the target domain, and gradually generates high-quality face cartoon style transfer result image. Experiments demonstrate the effectiveness of the method in improving the image quality after style transfer and the smooth conversion between different styles.