Unmanned Aerial Vehicle (UAV) inspection has become an important means to ensure the stable operation of a power grid. For intelligent processing of the inspection image, a deep learning-based multi-component inspection of the power grid is proposed. The problem of small sample defect detection is resolved in two stages: target detection and classification. For multi-target detection, a new loss function and prediction box selection based on the smallest convex set is proposed. These allow YOLOv3 to detect various target components accurately. After that, one-class classification is employed for small sample learning to estimate the state of the detected components in high-dimensional space. The test images are captured from the 220 kV power transmission line called the Anhui Xuanzao 4883 line. Experimental results show that the recognition rate is above 96% and the false negative rate is lower than 2% for common defects of a power grid. The method can effectively identify the defects of various components in the power grid. In the future, combined with edge computing to accelerate processing, UAV onboard inspection can be realized. This work is supported by the National Natural Science Foundation of China (No. 61701201).