There are many content-based retrieval methods for image databases, however, none of them have coped with both graph and image simultaneously. Moreover, existing graph retrieval methods handle either a silhouette or a graph component rather than the whole graph. Hence, it is the goal of this paper to propose a graph/image legend retrieval method. The proposed method consists of two phases: legend extraction and legend retrieval. In the extraction stage the legend images are first converted from RGB into YIQ color spaces to get Y values as gray-level images. The foreground is then separated from background for the legend image so as to determine the characteristics of the legend for later extraction and retrieval. Since each legend may not occupy the whole legend image, the region enclosing the legend only should be detected first to increase retrieval accuracy. In the retrieval stage, features are extracted for each legend to proceed legend matching. The features used in our method include aspect ratio, number of legend components and spatial histograms of pixel number, border length and gray level. Since the processed legends can be properly divided into two categories: graph and image, type-based matching is adopted to evaluate the similarity between the query legend and each database legend using different similarity measures according to the type of the query legend which can be automatically determined. In this way, the correct legend can be retrieved. The effectiveness and practicability of the proposed method have been demonstrated by various experiments.