Microgrid is a unified whole which is composed of distributed power sources, energy storage device and load. Main source control ability of voltage and frequency is crucial to stability of microgrid when it is isolated network running. Due to inherent characteristics of power electronic devices, the stability of mircogrid isolated running is poor. In order to realize regulating function of frequency and voltage of the main control source in AC mircogrid and improve frequency stability of the AC microgrid operation, a mircogrid with solar and storage system as main power supply is built under DigSILENT. Then a VSG controller based on virtual synchronous generation control technology and nonlinear robust control technology is designed to inverter. The Expansion State Observer (ESO) and Terminal Sliding Mode Control (TMSC) are introduced into the power-frequency controller to improve the control performance. Through simulation analysis, VSG controller can effectively regulate the frequency and voltage of microgrid when load or photovaltaic power changes. By comparing with the general controller simulation, the VSG control based on ESO and TMSC can not only realize frequency and voltage regulation of microgrid, but also improve the transient frequency stability more significantly and increase the inertia of AC microgrid. This work is supported by National Natural Science Foundation of China (No. 60971077).