Proceedings of the 37th Chinese Control Conference July 25-27, 2018, Wuhan, China
Single Image Super-Resolution via the Implementation of the Hardware-Friendly Sparse Coding Zhekang Dong1, 2, Chun Sing Lai3, Zhao Xu2, Donglian Qi1 1. College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China E-mail: {englishp, qidl}@zju.edu.cn 2. Department of Electrical Engineering, The Hong Kong Polytechnic University Hung Hom, Hong Kong E-mail:
[email protected] 3. Department of Engineering Science, University of Oxford, Oxford OX13LZ, United Kingdom E-mail:
[email protected] Abstract: Sparse representation is one of the effective approaches to perform the feature extraction on the high-dimensional signals and has been widely used in the field of image processing. In this paper, a novel single image super-resolution (SR) reconstruction algorithm based on sparse coding is presented. Specifically, due to the fact that sparse coding has the potential to greatly facilitate the biological-neural system and to deal with a large amount of complex data, while consuming a finite amount of energy, a memristor crossbar array synthesized by a threshold-type memristor model is constructed for the implementation of a hardware-friendly sparse coding method, i.e., the Locally Competitive Algorithm (LCA). Meanwhile, the relevant online dictionary learning method is provided, along with the description of weight programming strategy. Furthermore, the specific description of the image SR via the LCA sparse coding is provided. For the verification purpose, a series of experiments are carried out to illustrate the validity and effectiveness of the entire scheme. Key Words: Image super-resolution, sparse coding, memristor crossbar array, dictionary learning
1
Introduction
Image super-resolution (SR) reconstruction technology has been one of the hot spots in the image processing field over the past decade [1-5]. It mainly focuses on the problem of recovering the high-resolution (HR) image from a set of (a single) given low-resolution (LR) image(s), which opens up a path to overcome the inherent resolution limitations of some lost-cost imaging equipment. Generally, the methods for the implementation of the image SR reconstruction task can be broadly divided into three categories, i.e., the interpolation-based method [6-8], reconstruction-based method [9-11] and the learning-based method [12-15]. Specifically, (i) the fundamental idea of the first class methods is to estimate the missing pixels on the HR image by utilizing different interpolation kernel functions [6-8]. For example, the linear interpolation, bicubic interpolation, and nearest neighbor interpolation are all common interpolation-based SR approaches. Compared with the other methods, the interpolation-based SR algorithms are always simple and easy to implement. However, the problems of blurring and zigzagging artifacts may seriously affect the overall performance of the final HR image [8]. (ii) The principle of the second class methods can be summarized as solving the inverse problem of recovering the original HR image by fusing numerous LR images, based upon some reasonable assumptions or prior knowledge on the expected observation model [9-11]. The typical methods (including bilateral total variation, total variation regularization, non-local methods and so forth) have been proved effective in suppressing additive noises and preserving image edges. However, the performance of these methods may degrade rapidly if the magnification factor is large or the available input LR images are * This work is supported by National Natural Science Foundation (NNSF) of China under Grant Nos. 61571394, 61503341.
8132
insufficient. (iii) The learning-based methods mainly apply different machine learning techniques to establish the mapping relationships between the LR and HR image patches [12-15]. As one of the most commonly used methods, the sparse coding-based methods assume that the LR feature set and the corresponding HR feature set share the same sparse representation coefficients. Then, the missing details can be estimated by conveying the representation of LR feature set to the corresponding HR feature set [12, 15]. Nevertheless, the dictionary learning and the relevant weight storage method may greatly impact the final HR image quality. In 1971, Professor Chua originally predicted the existence of the memristor [16]. Then the relevant physical device is found by researchers of the Hewlett-Packard (HP) lab in 2008 [17, 18]. Notably, memristor is competent for the tasks of sparse coding and dictionary learning, due to its superior device properties such as nonvolatility, high density, low power, nanoscale geometries, nonlinearity and binary/multiple memory capacity and so on [19-23]. In this paper, memristor is utilized for the implementation of the sparse coding algorithm. The elements of dictionary are stored into the memristor crossbar array and can be changed by weight programming operation. The main contributions of this paper are summarized as follows. 1). A novel sparse coding algorithm with the hardware-friendly dictionary learning method is proposed, which has a good compatibility with the biological neural systems and can be utilized to effectively process large-scale sensory data. 2). A single image SR algorithm via sparse coding is provided in detail. Compared with the two existing methods, the presented method has its own advantages in terms of visual effect and objective image quality assessment.
With the development of the research on memristor theory and technology, numerous memristor models with different materials and structures have been presented successively [20]. According to the literature [24], an effective model must be able to reproduce the fundamental properties of memristive devices and does not conflict against various physical implementations. In this work, a threshold-type switching model with the desired memristive characteristics is considered. It can be mathematically expressed by [24]: e2 L (1) M L f0 L where f0 is the fitting parameter. L is the single-state variable and it can be written as:
L
§ N· Lmax ¨ 1 ¸ r¹ ©
dr dt
2 1.5 1
5
Vset
0
-5
Vre 0
1 Time (s)
0.5
2
0 -0.5 -1
-4
-2
.
0 Voltage ( V ) (a)
2
4
250
(2) 200
where Lmax denotes the largest value of the state variable L, κ is a fitting parameter determining the boundaries of the state variable. r is a voltage-dependent parameter and its dynamic function is given by V Vre °D J V V ˈ V0 d V Vre re ° ° Vre d V d Vset ®E V , ° V Vset °D ˈ V V d V0 J V Vset set ° ¯
*******L(V) function ******* .func L(y)={Lmax-Lmax*Kap/y} .ENDS threshold-type memristor Voltage (V)
The threshold-type memristor and its SPICE model
.SUBCKT memristor Plus Minus PARAMS: +rmin=100 rmax=390 rinit=390 al=1E3 be=50 ga=0.1 Vtr=2 Vtn=-2 +Kap=82 Lmax=5 f0=310 y0=1E-4 ******* Differential equation modeling******* Gr 0 r value={dr_dt(V(I(Plus)-V(Minus))*(st_f(V(Plus)-V(Minus))* st_f(V(r)-rmin)+st_f(-(V(Plus)-V(Minus)))*st_f(rmax-V(r)))} Cr r 0 1 IC={rinit} Raux r 0 1E12 ******* Current equation based on Ohm’s Law ******* Gpm Plus Minus value={(V(Plus)-V(Minus))/((f0*exp(2*L(V(r))))/L(V(r)))} ******* Function for the threshold-based behavior ******* .func dr_dt(y)={-al*((y-Vtn)/(ga+abs(y-Vtn)))*st_f(-y+Vtn) -be*y*st_f(y-Vtn)* st_f(-y+Vtr)-al*((y-Vtr)/(ga+abs(y-Vtr)))*st_f(y-Vtr)} *******Smoothing function ******* .func st_f(y)={1/(exp(-y/y0)+1)}
(3)
Memristance ( k : )
2
Table 1: Sub-circuit description of the memristor. * Threshold-type memristor model
Current ( mA )
3). The entire scheme provides a new avenue for the realization of image SR reconstruction in the hardware system. The rest of paper is organized as follows. The Section 2 briefly describes a kind of threshold-type memristor, as well as its SPICE model. In Section 3, the memristor-based sparse coding method and the corresponding online dictionary learning method are illustrated. Then, the image SR algorithm based on sparse coding is provided in Section 4. For the sake of verification, a series contrast experiments with the corresponding subjective/objective analysis are conducted in Section 5. Finally, Section 6 concludes the entire work.
150
100
50
0
where parameters α, β and γ are all constants, they are utilized to determine the slope and magnitude of (3) with the relationship α≫β and 0
