Optimizing Video Signal Processing Algorithms by Evolution Strategies H. Blume, O. Franzen, M. Schmidt Universität Dortmund Lehrstuhl für Nachrichtentechnik, AG Schaltungen der Informationsverarbeitung 44221 Dortmund
[email protected] Abstract Today many kinds of postprocessing are used in digital TV receivers or multimedia terminals for video signals to enhance the picture quality. To achieve this the properties of human visual perception have to be regarded. Because of the nonlinear nature of human visual perception (e.g. perception of edges and objects) many algorithms have been developed and optimized by heuristic methods or by application of rough image models. This is a severe problem as there are sometimes contradictory demands (e.g. detail resolution and alias suppression) and there are many optimization problems which cannot be solved analytically. Furthermore the simulations which have to be carried out in the field of video processing have to take into account a great variety of test sequences and therefore possess a heavy simulation load. In this paper we present the results of evolution strategies (ES) applied to develop and optimize some modules of a video signal processing feature box. The modules we have analyzed are as follows: A proscan conversion module is required to convert incoming interlaced TV signals into a progressive format which is obligatory for computer monitors or LCD and DMD devices (e.g. projectors) as they cannot display interlaced signals [1]. Further linear and nonlinear filter techniques are required for spatial conversion techniques like zooming or a picture in picture reproduction. For high quality temporal scan conversion techniques (e.g. 50 Hz interlace to 100 Hz interlace scan conversion reducing annoying artifacts as large area or detail flicker) motion vector based video processing is state of the art [1]. The motion information is generated by motion estimation algorithms.
Optimizing a nonlinear median based proscan conversion Applying nonlinear filters like rank order filters for picture processing the design process is much more complicated than for linear filters. Whereas linear filters can be described analytically in the frequency domain this is not possible for nonlinear filters. But nonlinear filters offer a lot of advantages like edge preservation, spike removal or robustness which make them very attractive for image processing. There are generally two methods for designing nonlinear filters. These are • design by root signals, which are invariant concerning filtering with a given nonlinear filter, • design by investigation of the output distribution function of a given nonlinear filter.
But each method has its own limitations well known by the experienced user so that still many algorithms are developed by heuristic methods, sequence simulations and subjective evaluations [1]. One of the most interesting applications for nonlinear filters (especially weighted median filters) in the video processing domain is a proscan conversion that means the interpolation of missing lines for interlaced video signals. Many proscan conversion algorithms make use of median filters [1]. We have optimized such a nonlinear median based proscan conversion by evolution strategies. The evolution strategy implemented for this problem represents a median filter arrangement by a vector of possible median window assemblies for proscan conversion. For each possible median window position a weighting factor has to be chosen which is an integer value in the range between 0
and 10. In summary with a total window size (sum of all possible pixel positions within three subwindows) of 50 this yields a search space of about 1150 different configurations. Regarding the time and memory consuming simulation for each individual (filter configuration) it becomes clear that only a tiny fraction of these configurations can be regarded in practice so that a sophisticated search algorithm is required. Fig. 1 depicts the results of an optimization run for a nonlinear proscan conversion. The objective function which is used here for evaluating each individual is the PSNR (Peak Signal to Noise Ratio) which is computed by a comparison between a filtered test sequence and an ideal reference sequence. The evolution strategy which is used for the optimization process of Fig. 1 is a (µ,λ)-ES [3] with µ=10 parents and λ=70 offsprings which are generated each time by mutation and discrete/intermediary recombination of two parent individuals. From the resulting offspring population the best 10 individuals are selected as new parent individuals by their corresponding PSNR values. After that selection the evolutionary loop of mutation and recombination, evaluation and selection starts again (for an exhaustive survey of evolution strategies see also [3]).
Fig. 1: Simulation results for an optimization process with a (10, 70)-ES for a proscan conversion In Fig. 1 also the simulation results of a static (non vectorbased) median proscan algorithm [1], the results of a vectorbased median algorithm with a median mask which has been found heuristically and the results of a vectorbased reference method [2] are depicted. It can be seen that after 6500 function evaluations the new median configuration yields significantly better results than the static and the vectorbased approach with heuristic masks and approaches more and more the reference method which requires much more implementation effort than the method which has been optimized here. Besides this proscan conversion we have investigated further video processing specific problems. Evolution strategies yielded also very good results in optimizing these problems.
References [1] Blume, H.; Schröder, H.: "Image Format Conversion - Algorithms, Architectures, Applications" Proc. of the IEEE ProRISC Workshop, pp. 19-37, Mierlo, Netherlands 27./28. 11. 1996 [2] Ivanov, K.V. : "Ein neues Verfahren zur Konversion von Fernsehbildsignalen für die progressive Wiedergabe", (in German) FKT-Magazin, 48. Jhg.; Nr. 10, pp. 1-7, 1994 [3] Schwefel, H.P.; Bäck, T.: "Evolution Strategies I/II" chap. 6/7 in "Genetic Algorithms in Engineering and Computer Science" ed. by J. Périaux, G. Winter; Wiley & sons, New York, 1995
