Abstract— In this paper, we study the best rate distortionperformance that an H.264 encoder can possibly achieve. Usingsoft decision quantization rather than the standard hard decisionquantization, we first establish a general framework for jointlydesigning motion compensation, quantization, and entropy codingin the hybrid coding structure of H.264 to minimize a truerate distortion cost. We then propose three rate distortionoptimization algorithms—a graph-based algorithm for optimalsoft decision quantization in H.264 baseline profile encoding givenmotion compensation and quantization step sizes, an iterativealgorithm for optimal residual coding in H.264 baseline profileencoding given motion compensation, and an iterative overallalgorithm for optimal H.264 baseline profile encoding—with themembedded in the indicated order. The graph-based algorithmfor optimal soft decision quantization is the core; given motioncompensation and quantization step sizes, it is guaranteed toperform optimal soft decision quantization to certain degree. Theproposed iterative overall algorithm has been implemented basedon the reference encoder JM82 of H.264. Comparative studiesshow that it achieves a significant performance gain, which can beas high as 25% rate reduction at the same PSNR when comparedto the reference encoder
đang được dịch, vui lòng đợi..