Aiming at the fact that the current stability research methods of hydraulic turbine regulating system (HTRS) under frequency control mode (FCM) ignore hydro-turbine nonlinearity, a controller parameter constraint determination algorithm (HBBC) considering hydro-turbine nonlinearity is proposed. This is done by combining Hopf bifurcation theory, and a bisection method and stability criterion, to realize the stability analysis of the nonlinear HTRS of all operating conditions in FCM. First, a high-precision nonlinear HTRS numerical simulation platform containing a nonlinear hydro-turbine model is built in FCM. Then a quantitative calculation process of stability region constraints based on HBBC is introduced in detail. Finally, taking an actual hydropower station as an example, the stability region of nonlinear HTRS of all operating conditions in FCM is calculated based on HBBC. The calculation results of the stability region in special operating conditions are verified. The results show that HBBC can replace the traditional stability region calculation method in HTRS stability analysis, and is superior in accuracy and reliability; HTRS has some "bad operating conditions" with the small stability region.