Rate-distortion optimized quantization improves the coding performance of video compression. However, most existing methods based on the operational approach require an exhaustive search process that is computationally expensive. In this thesis, we present a novel analytical method that provides a closed-form solution using a rate model of entropy coding. The computational cost of rate-distortion optimized quantization is greatly reduced. Our method achieves 5-7% bitrate reduction with no greater than 1% increase of total encoding time over the H.264/AVC reference encoder for video coded with IBBP GOP structure in normal PSNR range (30-40dB).