There is low diagnostic accuracy of transformer winding deformation and slight inter-turn short circuit fault diagnosis. Thus an early fault diagnosis method for transformer windings is proposed. First, ANSYS simulation software is used to establish a finite element model consistent with the relevant parameters of the experimental transformer. It also analyzes the distribution law of the leakage magnetic field of the transformer in the winding faults, and selects the appropriate fault characteristics and the installation position of the optical fiber leakage magnetic field sensor according to these laws. Then, the optimal hyperparameters of residual networks (ResNet) are found by improved coati optimization algorithm (ICOA), and the ICOA-ResNet model is constructed based on these parameters. The fault characteristics are input into the model for fault diagnosis. Finally, the feasibility of the proposed early fault diagnosis model for transformer windings is verified by simulation data and dynamic model experiments. Compared with four other models such as the support vector machine, the proposed model has higher accuracy in early fault diagnosis. This shows that the proposed method is effective in transformer winding deformation and inter-turn short circuit fault diagnosis.