The impacts of casing treatment (CT) with a single circumferential groove on the overall performance and flow stability have been studied based on numerical simulations in a transonic fan rotor. The results indicate that the best position of a single groove should be located at near 10% axial tip chord in terms of stall margin improvement. The peak efficiency is reduced to different extent after CT, while the efficiency is increased at near-stall condition for the effective CT. The entropy generation, blockage effect and leading edge incidence angle are all decreased obviously after the effective CT. Additionally, the interface between the tip leakage flow (TLF) and incoming main flow is pushed more downstream. Quantitative analysis of the tip leakage angle shows that the TLF intensity is reduced remarkably after the effective CT, which indicates the interface spillage and the stall inception are delayed. All the changes are beneficial to the stall margin improvement. Detailed analysis of the flow structures shows that it is more effective to enhance the stable operating range by controlling the TLF released over the 20%-54% axial tip chord.