In this paper an effective classification algorithm based on significant region and Gabor multi-frequency domain was proposed to overcome the identification problems of the texture image caused by the rotating and scaling. On rotation invariant, Gabor wavelets are used to acquire the texture’s features. On scaling resisting, the similarity between two textures was calculated by comparing the significant region. In our proposed algorithm the wavelet transform and clustering method were used to locate the significant region so that four scaling resistance parameters were generated from the significant region. Then, support vector machines (SVMs) are introduced as the classifier. Experimental results reveal that this proposed algorithm outperforms existing design algorithms.