Several studies have been reported on the characteristics of data sets which are directly correlated with the capability of the classifier. Therefore, a study in the cognition is conceived, and we suggest the feature optimization to guarantee class separability. We present that the available resource of feature extraction concepts of neural networks(NN) can be applied to the feature optimization problem. Thus, we propose the NN-SVM to set a sufficient number of features compensating for the lack of information. In the NN-SVM algorithm, we use the NN to transform data sets to extract features for support vector machines (SVM) classification. In this way, any validation set and test set subjected to the same transformation before it is classified by the classifier. The experiments on several existing data sets show that, when the augmented data are utilized, the classification errors estimated are reduced by experimental evidence. This implies that the class labels can be used as extra helpful information to feature extraction.