In this study, a filter-design scheme based on Independent Component Analysis (ICA) is proposed for defect inspection in backlight and TFT-LCD (Thin film transistor liquid crystal display) panels. In a backlight panel, the sensed image is uniform. The gray levels of defects and the background are very similar and results in a low-contrast image. In a TFT-LCD array panel, the image involves a homogeneous texture pattern. The irregular defects embedded in the textured surface make the defect detection task difficult. The proposed method is based on an ICA filtering scheme that computes the output response of energy from the convolution of an inspection image with the ICA filter. In ICA, independent sources and the mixing matrix that constructs the observed signals can be estimated form the training subimages by maximizing the independency of the sources. Since any subimages in either the backlight panel or the TFT-LCD panel can be considered as a shifted version of the same pattern, only one source is needed to be estimated. The corresponding demixing vector, which is obtained form the inverse mixing matrix, of the estimated source is used as the filter for defect detection. The ICA model with a prior constraints is applied to determine the filter so that the convolved responses of all training subimages have the variance as small as possible. In this research, Particle Swarm Optimization (PSO) algorithm is used to solve for the constrained ICA model. The designed filter can stress the difference of convolved responses between defects and regular regions in the inspection image. Experimental results have shown that the proposed method can effectively detect defects in low-contrast backlight panel images and textured TFT-LCD panel images.