Because of the high penetration of distributed generation (DG), the operation of DG is complex and nonlinear, and the harmonic characteristics of fault current are significant. This makes it difficult for traditional relay protection to accurately distinguish between inrush and fault current. Therefore, this paper first analyzes the expression of harmonics generated by photovoltaics and a doubly-fed wind turbine, deduces the generation mechanism of inrush current and harmonic characteristics, and summarizes the proportion of each frequency component of inrush and short-circuit current. Taking full advantage of variational mode decomposition (VMD) to avoid mode aliasing, the fault current is decomposed into different central frequency modes using the VMD algorithm, and the magnetizing inrush current is identified by comparing the variance contribution of different modes. The simulation results show that the proposed identification method can accurately identify the fault in conditions of different fault types, types of distributed generation and switching angles, and the identification can be completed within one cycle of the fault with high sensitivity.