Mobile Video Processing

Several problems related to video quality improvement will be considered in the project: Camera noise and quantization noise reduction, Color constancy improvement, Video quality and noise estimation. The main causes of camera noise are the granularity of sensor characteristics and the influence of thermal noise. Camera noise has spatial and temporal properties. This noise looks like roughness in flat (still) regions which varies from frame to frame. The conventional way of camera noise reduction is to apply spatio-temporal filters. Quantization noise appears after low-bitrate lossy coding applied in mobile video. It is the result of joint process of motion estimation and compensation, independent block transforms (like DCT) and further quantization. Distortions caused by quantization noise are called artifacts. The main types of artifacts are Blocking, Ringing (repeated vertical and horizontal bars) , Mosquito (“clouds of insects” around strong edges). Usually they can be reduced using special post-filtering and in-loop filtering methods. In-loop filtering is a common part of encoder and decoder. Post-filtering methods may be used independently of the codec structure. In-loop filtering is not in the scope of this project because the codec structure is assumed unavailable. Characterization of various types of noise is another research subject. New detection method will be proposed for each type of noise. The task related to the second problem is to improve color rendering of video captured using mobile devices. In many cases the color rendering is degraded due to errors in camera automatic white balance correction and a high dynamic range of the scene illumination. Methods based on color gamut correlation and methods that mimic the human visual system show much promise for quality improvement of such videos. The third problem concerns development of video quality estimation methods. Known objective metrics like PSNR proved to be ineffective in estimation of quantization artifacts and color rendering quality. The goal of the project is to study other methods for quality and noise estimation and introduce some new metrics. Note that method complexity will be taken into account when targeting mobile platforms.