This thesis develops a C language and genetic algorithm (GA) based optimization program which can be integrated with HFSS. And we use this program to optimize the ortho-mode transducer structure. There are four kinds of ortho-mode transducer that can be chosen in program interface. We use this genetic algorithm based program to optimize all four kinds of OMT and then compare their performance. Last we real make an OMT by aluminum to measure it and compare with simulation result.