A procedure is using the global search optimization of stimulated annealing to design the reference function. First we encode color pattern into a single channel and use the minimum average correlation energy to obtain an initial feasible solution and the state, instead of the state randomly generated. Then substitute the solution into stimulated annealing Algorithm to obtain the optimizing reference function, and utilize the multilevel quantitative analysis of the reference function. Finally we utilize Mach-Zehnder joint transform correlator with simulated annealing to obtain the optimization of the reference function for recognition of color pattern. From results of simulation, the optoelectronic pattern recognition system with simulated annealing is accepted.