We reported the fabrication and characteristics of biepitaxial YBCO/CeO2/MgO Josephson junctions and SQUIDs, here CeO2 was used as a seed layer to create the biepitaxial grain boundary. The surfaces of the unannealed CeO2 layer and the annealed one were probed by the atomic force microscope (AFM). The temperature dependent resistance of YBCO/CeO2/MgO thin film reveals that the annealing process for CeO2 layer is crucial for the quality of the YBCO thin film. The voltage-current curves of Josephson junctions exhibit the resistively shunted junction (RSJ) behavior. Furthermore, both the integral and half-integral Shapiro steps were observed under zero applied magnetic field. The magnetic modulated voltage oscillation was also found for the SQUIDs.