Dictionary learning methods have achieved a considerable number of success in many problems. Inspired by the dictionary learning algorithm K-SVD, we argue that the reconstruction error is useful in image classification problem. We propose Paired Discriminative K-SVD (PD-KSVD) to learn paired dictionaries by use positive training signals and negative training signals. The experimental results show that the proposed method improves the accuracy in comparison with existing methods. The benefits of PD-KSVD is: 1). These paired dictionaries have more discriminative property and achieve higher accuracy. 2). The discrimination ability increases when the size of the learned dictionary is decreased. We also propose a Positive Reconstruction Error Selection (PRES) Scheme and Negative Reconstruction Error Selection (NRES) Scheme to learn balanced dictionary from unbalanced dictionary. The learned balanced dictionary is beneficial to image classification problem.