Occlusion aware motion compensation for video frame rate up-conversion

Abstract

Since the emergence of high definition (HD) display technologies, video standards conversion problem has become an important issue in storage, transmission and display of video content. Video frame rate up-conversion (FRUC) is considered as a standard task for today's HD displays because these displays reach high refresh rates of at least 100/120 Hz and low video frame rates should be pulled up by a factor of 2 or more before display. Motion compensated FRUC techniques are proposed to avoid motion blur and motion judder at high refresh rates, but these techniques suffer from spatial inconsistencies and artifacts especially in occluded regions of the interpolated frames. This paper introduces a new video FRUC method that aims to remove both motion judder and occlusion artifacts and generate smooth object motion for high quality displays. An occlusion adaptive overlapped block motion compensation (OBMC) technique is proposed, which provides spatio-temporally consistent frame interpolation. Covered/uncovered regions are detected by analyzing the discontinuities of the motion vector field. The occlusion regions are interpolated using this covered/uncovered decision and a new error metric that measures spatial consistency. Compared to existing methods, the proposed algorithm achieves FRUC with fewer artifacts and better spatial resolution especially in occluded areas.