Color words always play an essential role in literary works. In Hong Lou Meng, they are of a high frequency, and a great number of color words in it are closely associated with Chinese culture. However, researches that analyze the translation of color words from the cultural perspective have not received much attention. By illustrating the associative meaning of basic color words in Chinese and English, the research finds that different cultural backgrounds have an influence on translation. Thus, translators cannot take the words too literally and should consider the cultural connotation of the words and the context in the process of translating the color words. The findings of the study can help translators when they translate the original text that involves Chinese culture or contains special meanings.