The current numerical weather prediction (NWP) wind speed correction methods are based on the amplitude characteristic evaluation modeling of the longitudinal error. They ignore the potential impact of the lateral error caused by the lead or delay effect between the NWP wind speed series and historical wind speed series on NWP wind speed correction. Therefore, an NWP wind speed reinforcement correction method based on the recurrent deterministic policy gradient (RDPG) algorithm and considering lateral and longitudinal errors is proposed. First, based on weighted Euclidean distance similarity, the lead/lag timing interval (LLTI) boundary of each NWP wind speed to be corrected is determined to form a strongly correlated timing interval. Subsequently, the statistical characteristics of the LLTI are used to replace the original meteorological characteristics of the NWP. Based on the Shapley additive explanations (SHAP) attribution theory, the influence of different features on the model output is evaluated using the marginal contribution of each feature. Finally, an NWP wind speed correction model based on the RDPG is established. Through an example analysis, the effectiveness and feasibility of the proposed method for modifying the NWP wind speed on different time scales of ultra-short-term and short-term predictions are verified.