Xue and Hong proposed an anonymous authentication scheme with key agreement based on chaotic maps. After analyzing Xue and Hong’s scheme thoroughly, we find that their scheme cannot ensure user anonymity as claimed. In this paper, we will show this found security flaw which Xue and Hong’s scheme suffers from.