The fruit and vegetable wholesale market has a nature of public utility, and there is an inducement which market managers can adjust the surplus. Both factors cause the profit and loss information on the annual report cannot show the true operating state appropriately. If in terms of using the traditional amount or value as analyzing the efficiency of output, it will not reflect the true efficiency of fruit and vegetable wholesale market. A better way is to analyze each market's performance according to its labor-use efficiency. This study in the literature is the first time to explore this inefficiency in the agricultural product market, as we apply stochastic frontier analysis to measure and compare the labor-use efficiency of the fruit and vegetable wholesale market. In addition to this, we establish the factors that affect the labor-use of the fruit and vegetable wholesale markets.The study found that the labor use of the fruit and vegetable wholesale market actually existed inefficiency. During the study period 1994-2006 the average inefficiency was 38.56%. Inefficiency will be changed with owing to the class, type and market characteristics market. The study identified: the larger market class, a single vegetable market and trading with auctions and bargaining, the higher the inefficiency is. The empirical result of labor-use elasticity is also expected to show that: the price elasticity of labor-use is negative, and the total value of transacted toward labor-use has a positive effect.