In this paper, structural damage detection analysis is studied with consideration of incomplete modal data under the noisy conditions. In the process of damage detection, a two-step method in combination with system identification technique is employed. In the first step, we proposed a modification to the conventional modal residual force method, so that possibly damaged elements in a structure can be located more reliably. In the second step, the damage extent of possibly damaged elements is then evaluated using the least-squares method and a global optimization method, respectively. Numerical simulation results show that the proposed method is accurate and robust using incomplete modal data under noisy conditions.