In the non-solid earthed network fault line selection, nowadays the frequency band selection method is based on special frequency band (SFB). However, this method is liable to be influenced by the distribution network operation type and line parameters. To overcome the disadvantages, a new selection method of frequency band based on matrix pencil algorithm is proposed. In ungrounded distribution network, the minimum frequency is 0 Hz, in compensated distribution network, the minimum frequency is 150 Hz. The maximum frequency of frequency band is determined by the way as follows: first the common frequencies and phases of all line transient zero sequence current are extracted by the matrix pencil method. Then the maximum frequency that meets a certain relationship is found and the maximum frequency is the maximum of the frequency band. The simulation on PSCAD and the recording data analysis verify that the method proposed is effective. The method is hopeful to improve the accuracy of the fault line selection.