The majority of research work on triboelectric energy harvesting is on material science, manufacturing and electric circuit design. There is a lack of in-depth research into structural dynamics which is crucial for power generation in triboelectric energy harvesting. In this paper, a novel triboelectric energy harvester with a compact structure working in sliding mode is developed, which is in the form of a casing and an oscillator inside. Unlike most sliding-mode harvesters using single-unit films, the proposed harvester utilizes grating-patterned films which are much more efficient. A bistable mechanism consisting of two pairs of magnets is employed for broadening the frequency bandwidth. A theoretical model is established for the harvester, which couples the structural dynamics domain and electrical dynamics domain. This paper presents the first study about the nonlinear structural dynamics of a triboelectric energy harvester with grating-patterned films, which is also the first triboelectric energy harvester integrating grating-patterned films with a bistable magnetic system for power performance enhancement. Theoretical studies are carried out from the perspectives of both structural and electrical dynamics. A comparison between the coupled model and uncoupled model reveals that the electrostatic force between the electrodes can be neglected. Great differences in structural response and electrical output are found between a velocity-dependent model and Coulomb’s model for modelling the friction in the harvester. The bistable mechanism can effectively improve the output voltage under low-frequency excitations. Additionally, the output voltage can also be obviously enhanced through increasing the number of the hollowed-out units of the grating-patterned films, which also results in a slight decrease of the optimal load resistance of the harvester. These findings enable innovative designs for triboelectric energy harvesters and provide fabrication guidelines in practical applications.