In this thesis, we studied the problem of DOA estimation in the presence of mutual coupling. The new proposed method for reconstructing the steering vector can estimate the DOAs in the situation of a large number of source signals with unknown mutual coupling. The main advantage of the proposed method is that the capacity of the number of source signals is higher than other methods in the case of high accuracy system model. The proposed method can also be extended to an iterative method that estimate the mutual coupling coefficients and the DOAs respectively. Then the estimation accuracy has improved. Moreover, the proposed method can work well by applying the spatial smoothing technique even if source signals are coherent. Simulation results confirm the advantage of the proposed method.