A Multi-Resolution Mode CMOS Image Sensor with a Novel ... - MDPI

sensors Article

A Multi-Resolution Mode CMOS Image Sensor with a Novel Two-Step Single-Slope ADC for Intelligent Surveillance Systems Daehyeok Kim 1 , Minkyu Song 1 , Byeongseong Choe 2 and Soo Youn Kim 1, * 1 2

*

Department of Semiconductor Science, Dongguk University-Seoul, Seoul 04620, Korea; [email protected] (D.K.); [email protected] (M.S.) Department of Information and Telecommunication Engineering, Dongguk University-Seoul, Seoul 04620, Korea; [email protected] Correspondence: [email protected]; Tel.: +82-2-2260-3186

Received: 9 May 2017; Accepted: 24 June 2017; Published: 25 June 2017

Abstract: In this paper, we present a multi-resolution mode CMOS image sensor (CIS) for intelligent surveillance system (ISS) applications. A low column fixed-pattern noise (CFPN) comparator is proposed in 8-bit two-step single-slope analog-to-digital converter (TSSS ADC) for the CIS that supports normal, 1/2, 1/4, 1/8, 1/16, 1/32, and 1/64 mode of pixel resolution. We show that the scaled-resolution images enable CIS to reduce total power consumption while images hold steady without events. A prototype sensor of 176 × 144 pixels has been fabricated with a 0.18 µm 1-poly 4-metal CMOS process. The area of 4-shared 4T-active pixel sensor (APS) is 4.4 µm × 4.4 µm and the total chip size is 2.35 mm × 2.35 mm. The maximum power consumption is 10 mW (with full resolution) with supply voltages of 3.3 V (analog) and 1.8 V (digital) and 14 frame/s of frame rates. Keywords: CMOS image sensor; fixed pattern noise; intelligent surveillance system (ISS); low power consumption; multi-mode pixel resolution; two-step single-slope ADC

1. Introduction Due to their low cost, low power consumption, and easy integration of analog/digital processing functions on a chip, CMOS image sensors (CIS) have driven the market over charge coupled device (CCD). The achievement of such inherent characteristics of CIS is highly desirable for intelligent surveillance systems (ISS) [1,2]. The CIS in ISS observes an environment and then transmits data to monitors equipped with a video recorder for security purposes. Recently, for home and public surveillance, ultralow-power CIS [3–5] that are battery-operated have widely been used owing to their accessibility and easy installation. In addition, since the CIS in ISS is required to always be turned on, even when there are no security-related events (called peace mode), lowering image resolution during peace mode can further reduce power consumption. Therefore, in order to make CIS in ISS more power-efficient, configurable resolution depending on whether security-related events occur or not is necessary to select low-resolution mode and high-resolution mode. In this paper, in order to make images configurable, the proposed CIS supports more sub-sampling ratios from 1, 1/2, 1/4, 1/8, 1/16, 1/32, and 1/64 resolution mode. Smaller resolution modes lead to lower power consumption by disabling some parts of peripheral circuits (such as row control block and column parallel ADC [6–8]). In addition, for the high quality mode when security-related events happen, we improve the performance of a two-step single-slope analog-to-digital converter (TSSS ADC). The nonlinearity and column fixed-pattern noise (CFPN) of TSSS ADCs mainly come from parasitic capacitances on the input node of comparators in correlated double sampling (CDS) located in each column. Since the parasitic capacitances in comparators of conventional TSSS ADC affect the

Sensors 2017, 17, 1497; doi:10.3390/s17071497

www.mdpi.com/journal/sensors

Sensors 2017, 17, 1497 Sensors 2017, 17, 1497

2 of 9 2 of 9

we separate inputduring nodescoarse of comparators for ADC ramp operation, signals forwecoarse ADC fine ADC, slope of rampthe signals ADC and fine separate theand input nodes of respectively 4-inputfor comparators, the reduction of by parasitic capacitances in our comparators by forusing ramp signals coarse ADCleading and finetoADC, respectively using 4-input comparators, proposed The contents of the paper areproposed as follows: in SectionThe 2, contents an intelligent leading to comparators. the reduction of parasitic capacitances in our comparators. of the surveillance system in and a pixel sub-sampling techniquesystem are introduced. circuit design and paper are as follows: Section 2, an intelligent surveillance and a pixelThe sub-sampling technique implementation are circuit discussed in Section 3. Measurement andinconclusions are summarized in are introduced. The design and implementation are results discussed Section 3. Measurement results Sections 4 and 5, are respectively. and conclusions summarized in Sections 4 and 5, respectively. 2. Intelligent Intelligent Surveillance Surveillance Systems Systems 2. Figure 11 provides provides aa brief brief illustration illustration of of aa battery-powered battery-powered intelligent intelligent surveillance surveillancesystem system(ISS). (ISS). Figure The CIS with twotwo different modes: low-resolution modemode and high-resolution mode. The CIS in inthe theISS ISSoperates operates with different modes: low-resolution and high-resolution In the low-resolution mode case, there are no security-related events, andevents, it is called this mode. In the low-resolution mode case, there are no security-related andpeace it is mode. called In peace mode, In CIS inmode, the surveillance system is not required to required capture high-resolution images. However, mode. this CIS in the surveillance system is not to capture high-resolution images. when an emergency situation happens, the images should be captured with high-resolution However, when an emergency situation happens, the images should be captured mode with (called emergency mode) to transmit the detailed information about the event, such as about captured high-resolution mode (called emergency mode) to transmit the detailed information theimages event, and locations to the nearest center. Since the battery-powered CIS should always be turned such as captured images andpolice locations to the nearest police center. Since the battery-powered CIS on to monitor whether security-related events occur or not, the low-power performance is necessary. should always be turned on to monitor whether security-related events occur or not, the low-power Therefore, in is order to reduce the totalinpower CISpower in ISS,consumption we suggest sub-sampling performance necessary. Therefore, order consumption to reduce the of total of CIS in ISS, method during reading out the image datareading in low-resolution mode. we suggest sub-sampling method during out the image data in low-resolution mode.

Figure 1. A brief explanation of an intelligent surveillance system (ISS). Figure 1. A brief explanation of an intelligent surveillance system (ISS).

There are are two two ways ways to to make make the the image image resolution resolution configurable configurable to to achieve achieve low-power low-power CIS CIS during during There low-resolution mode: mode: (1) (1) sub-sampling sub-sampling and and (2) (2) binning binning [9,10]. Sub-sampling Sub-sampling methods methods skip skip multiple multiple low-resolution pixels and and ADCs ADCs during during reading reading out out the image data, leading leading to power-savings power-savings proportional proportional to to the the pixels sub-sampling ratio ratio used. used. On On the the other other hand, hand, binning binning methods methods average average or or adds adds multiple multiple pixels pixels with with sub-sampling additional circuits circuits for for the the functions functions in in the the pixel pixel array arrayor orADCs, ADCs,leading leadingto toextra extrapower powerconsumption. consumption. additional Even though though the the binning binning method method results results in in aa wide wide dynamic dynamic range range while while the the captured captured images images with with Even sub-sampling have have low low resolution, resolution, as as shown shown in in Figure Figure 2, 2, we we choose choose the sub-sampling sub-sampling method method to to sub-sampling maximally reduce power consumption during the peace mode of ISS. It should be noted that we maximally reduce power consumption during the peace mode of ISS. It should be noted that we discuss about about the the sub-sampling of a monochrome sensor in this paper. discuss

Sensors 2017, 17, 1497

3 of 9

Sensors 2017, 2017, 17, 17, 1497 1497 Sensors

of 99 33 of

(a) (a)

(b) (b)

Figure 2. 2. Pixel Pixel sub-sampling sub-sampling technique technique (a) (a) high-resolution high-resolution mode; mode; (b) (b) low-resolution low-resolutionmode. mode. Pixel Figure

3. Circuit Circuit Description Description 3. 3.1. Structure Structure of of the the CMOS CMOS Image Image Sensor Sensor (CIS) (CIS) 3.1. Figure333shows showsthe theblock blockdiagram diagram of the proposed CIS in this this paper. The CIS structure consists Figure ofof the proposed CIS in this paper. TheThe CISCIS structure consists of a shows the block diagram the proposed CIS in paper. structure consists of a pixel array (176 × 144), 8-bit column-parallel analog to digital converter (ADC) including pixel 144), ×8-bit column-parallel analog to analog digital converter including comparators of a array pixel (176 array× (176 144), 8-bit column-parallel to digital(ADC) converter (ADC) including comparators in the the correlated double sampling (CDS), 8-bit static static random access memories (SRAMs), in the correlated double sampling (CDS), 8-bit static random access memories (SRAMs), 8-bit counter, comparators in correlated double sampling (CDS), 8-bit random access memories (SRAMs), 8-bit counter, digital timing control blocks (=row control block), and multiplexer (MUX) for data digital timing control blocks (=row control block), and multiplexer (MUX) for data read-out. The 8-bit counter, digital timing control blocks (=row control block), and multiplexer (MUX) for pixel data read-out. The pixel converts the amount of light into the corresponding voltage, which is the input converts of light the intoamount the corresponding which is thevoltage, input for the ADC read-out.the Theamount pixel converts of light intovoltage, the corresponding which is thewhere input for the the ADC ADC where the pixel pixel output voltage voltage isdigital transformed intorow digital code. The row control control block the pixel output voltage is transformed into ais code. The control block controls the timings for where the output transformed into aa digital code. The row block controls the timingsADC for pixel pixel operation, ADC operation,respectively. and data data read-out, read-out, respectively. In addition, addition, for pixel the operation, operation, andADC data operation, read-out, In addition, pixel sub-sampling controls timings for operation, and respectively. In pixel sub-sampling method makes image resolution configurable to achieve low-resolution with lowmethod makes image resolution configurable to achieve low-resolution with low-power mode andlowthe pixel sub-sampling method makes image resolution configurable to achieve low-resolution with power mode mode and and the the sub-sampling ratio can1/64. be chosen chosen from 1/2 1/2 to 1/64. 1/64. The detailed detailed operation sub-sampling ratio cansub-sampling be chosen fromratio 1/2 can to The detailed operation principle of the operation proposed power be from to The principle of the proposed ADC will be be discussed discussed in the the following following sections. sections. ADC will of bethe discussed in the following sections. in principle proposed ADC will

MUX MUX

Row Row Control Control Block Block

8b 8b SRAM SRAM

8b 8b 8b 8b SRAM SRAM SRAM SRAM

Odd OddOutput Output

Ramp Ramp Generator Generator

8b 8b Counter Counter

Even EvenOutput Output

Pixel Array Array Pixel

Column ADC ADC array array Column

Figure 3. 3. A A block diagram diagram of the the proposed CIS. CIS. Figure Figure 3. A block block diagram of of the proposed proposed CIS.

3.2. Multi-Mode Multi-Mode Pixel Resolution Resolution 3.2. 3.2. Multi-Mode Pixel Pixel Resolution By adopting adopting configurable configurable resolution resolution features, features, CIS CIS can can activate activate different different resolution resolution modes modes such such By By adopting configurable resolution features, CIS can activate different resolution modes such as as either in high-resolution or low-resolution modes. Sub-sampling method is used for as either in high-resolution or low-resolution modes. Sub-sampling is used for either in high-resolution or low-resolution modes. Sub-sampling method is usedmethod for variable-resolution variable-resolution operation (such as 1, 1/2, 1/4, 1/8, 1/16, 1/32, and 1/64), which only processes the variable-resolution as 1, 1/32, 1/2, 1/4, 1/8, 1/16,which 1/32, and which processes the operation (such as 1,operation 1/2, 1/4, (such 1/8, 1/16, and 1/64), only1/64), processes theonly number of pixels number of pixels required for the given resolution. In low-resolution mode, the number of pixels to number of pixels required for the given resolution. In low-resolution mode, the number of pixels to be processed is smaller, resulting in reduced average power consumption for a given time span. For be processed is smaller, resulting in reduced average power consumption for a given time span. For

Sensors 2017, 17, 1497

4 of 9

required for the given resolution. In low-resolution mode, the number of pixels to be processed is Sensors 2017, 17, 1497 4 of 9 smaller, resulting in reduced average power consumption for a given time span. For the configurable resolution feature, CIS consists of a CIS single unit ofof8a×single 8 pixels of 4-shared 4T-active pixel the configurable resolution feature, consists unitconsisting of 8 × 8 pixels consisting of 4-shared sensors (APS), and 4-shared column ADC. 4T-active pixel sensors (APS), and 4-shared column ADC. Figure Figure 4a 4a shows shows the the operation operation of of high-resolution high-resolution mode, mode, where whereodd/even odd/even ADC ADC column column arrays arrays and row control arrays are all operated. It should be noted that a column ADC in this work requires and row control arrays are all operated. It should be noted that a column ADC in this work requires the asas shown in in Figure 4a.4a. Therefore, thethe outputs of A1 the layout layout pitch pitchas assame sameasasthe thewidth widthofoffour fourpixels pixels shown Figure Therefore, outputs of and B1 pixels are are connected to odd column ADC, while those of C1 andand D1D1 pixels areare connected to A1 and B1 pixels connected to odd column ADC, while those of C1 pixels connected even column ADC. After reading outout thethe firstfirst row’s pixel outputs, the the second row’s pixel outputs are to even column ADC. After reading row’s pixel outputs, second row’s pixel outputs read out in consecutive order with row control block. On the other hand, for the operation of 1/16 are read out in consecutive order with row control block. On the other hand, for the operation of 1/16 mode, Figure 4b, mode, one one pixel pixel of of every every four four rows rows and and columns columns is is only only turned turned on. on. As As shown shown in in Figure 4b, since since both both A1 and E1 pixels are connected to odd column ADC, even column ADC can be turned off, leading A1 and E1 pixels are connected to odd column ADC, even column ADC can be turned off, leading to to the of ADC Further, the the reduction reduction of of the the power power consumption consumption of ADC by by half. half. Further, the row row control control block block for for the the first first and and fifth fifth row row only only is is need need to to be be operated, operated, while while those those for for other other rows rows can can be be turned turned off off to to save save power power consumption. Finally, for the case of 1/64 mode, since only A1 pixel is turned in 16 × 16 pixel array, array, consumption. Finally, for the case of 1/64 mode, since only A1 pixel is turned in 16 × 16 pixel half half of of odd odd column column array array and and seven seven row row control control blocks blocks can can be be turned turned off, off, leading leading to to the the maximum maximum reduction of power consumption during the low-power mode of ISS. reduction of power consumption during the low-power mode of ISS. Even Col. ADC

row

A2 B2 C2 D2 E2 F2 G2 H2

3rd row 5th row 6th row

Odd Col. ADC

Odd Col. ADC

(a)

A1

E1

A5

E5

2nd row 3rd row 4th row 5th row

Out

6th row 7th row 8th row

Odd Col. ADC

Odd Col. ADC

Odd Output

row

8th row

Odd Output

7th

Out

1st row

MUX

MUX

4th row

Row Control Block

A1 B1 C1 D1 E1 F1 G1 H1

2nd

Even Col. ADC Even Output

1st row

Even Col. ADC Even Output

Row Control Block

Even Col. ADC

(b)

Figure 4. 4. Simplified Simplified principle principle operation operation of of multi-mode multi-mode pixel pixel resolution resolution with with 16 16× × 16 pixel array array and and Figure 16 pixel ADC array for (a) high resolution mode and (b) 1/16 resolution mode. ADC array for (a) high resolution mode and (b) 1/16 resolution mode.

3.3. Proposed Two-Step Single-Slope ADC (TS SS-ADC) 3.3. Proposed Two-Step Single-Slope ADC (TS SS-ADC) Among the column-parallel ADC architectures, single-slope (SS) ADC is widely used for Among the column-parallel ADC architectures, single-slope (SS) ADC is widely used for commercial high-resolution CMOS image sensor applications. However, SS ADC is not power commercial high-resolution CMOS image sensor applications. However, SS ADC is not power efficient, efficient, compared to other ADC schemes, because ADC operation period is exponentially increased compared to other ADC schemes, because ADC operation period is exponentially increased with ADC with ADC resolution. As for an example of an 8-bit SS ADC, at least 28 cycles (=256 cycles) is required resolution. As for an example of an 8-bit SS ADC, at least 28 cycles (=256 cycles) is required to finish to finish ADC operation. Figure 5a show the operation of 8-bit SS ADC. Each comparator in a column ADC operation. Figure 5a show the operation of 8-bit SS ADC. Each comparator in a column compares compares VRAMP (from ramp generator as shown in Figure 3) and VIN (from APS). When VRAMP starts, VRAMP (from ramp generator as shown in Figure 3) and VIN (from APS). When VRAMP starts, 8-bit 8-bit counter is simultaneously enabled to count output code. Once VRAMP exceeds the VIN, the output counter is simultaneously enabled to count output code. Once VRAMP exceeds the VIN , the output of of comparator is flipped and the counter value at that moment is read out. comparator is flipped and the counter value at that moment is read out. In order to reduce the time for ADC operation, two-step single-slope (TSSS) ADC has been In order to reduce the time for ADC operation, two-step single-slope (TSSS) ADC has been recently recently reported in [11]. TSSS ADC requires only 2M + 2N cycle to finish ADC operation with reported in [11]. TSSS ADC requires only 2M + 2N cycle to finish ADC operation with M + N-Bit of M + N-Bit of coarse and fine ADCs. As shown in Figure 5b, for 8-bit TSSS ADC, coarse ADC converts coarse and fine ADCs. As shown in Figure 5b, for 8-bit TSSS ADC, coarse ADC converts 3-bit of the 3-bit of the most significant bits (MSB) and fine ADC converts 5-bit of the least significant bits (LSB). most significant bits (MSB) and fine ADC converts 5-bit of the least significant bits (LSB). During the During the coarse ADC operation, all comparators use VRAMP that covers full scale of VIN to find upper 3 MSB of the overall ADC. After then, each column selects one of the multiple ramp generators depending on the coarse ADC outputs for fine ADC operation that leads to the output of the fine ADC corresponding to the lower 5 bits of the overall ADC. Finally, TSSS is required to 23 + 25 cycles (=40 cycles) for ADC operation that is 216 cycles less than that of SS ADC.

Sensors 2017, 17, 1497

5 of 9

coarse ADC operation, all comparators use VRAMP that covers full scale of VIN to find upper 3 MSB of the overall ADC. After then, each column selects one of the multiple ramp generators depending on the coarse ADC outputs for fine ADC operation that leads to the output of the fine ADC corresponding to the lower 5 bits of the overall ADC. Finally, TSSS is required to 23 + 25 cycles (=40 cycles) for ADC operation that is 216 cycles less than that of SS ADC. Sensors 2017, 17, 1497 5 of 9 Sensors 2017, 17, 1497

5 of 9

VIN

VIN

VIN

VIN VRAMP VRAMP

VRAMP VRAMP

Counter Counter

Counter Counter

256 Clock Cycles (8-bit) 256 Clock Cycles (8-bit)

(a) (a)

Coarse ADC Coarse(3-bit) ADC 8 Clock Cycles 8 Clock Cycles (3-bit)

Fine ADC Fine ADC 32 Clock Cycles (5-bit) 32 Clock Cycles (5-bit)

(b) (b)

Figure 5. Operation principle of (a) SS ADC and (b) TSSS ADC. Figure Figure5.5.Operation Operationprinciple principleofof(a) (a)SS SSADC ADCand and(b) (b)TSSS TSSSADC. ADC.

Figure 6a shows a schematic of conventional TSSS ADC with an analog correlated double Figure6a6a shows a schematic of conventional TSSSwith ADC ancorrelated analog correlated double Figure shows a schematic of conventional TSSS ADC an with analog double sampling sampling (CDS) block. The holding capacitor (CH) that stores the final coarse analog voltage (VREF) is sampling (CDS) block. The holding (C H ) that stores the final coarse analog voltage (V ) is (CDS) block. The holding capacitor (Ccapacitor ) that stores the final coarse analog voltage (V ) is connected REF drives REF connected to the external ramp signalsH(VRAMP) in a series. When the next fine ramp signal the to the external signals RAMP)When in a series. When next fine drives ramp signal drives the toconnected the external ramp signalsramp (VRAMP ) in a (V series. the next finethe ramp signal the comparator comparator through CH, the fine ramp slope would be distorted by parasitic capacitances that come comparator through CH, the finewould ramp be slope wouldby beparasitic distortedcapacitances by parasitic that capacitances that come through CH , the fine ramp slope distorted come from device from device mismatch, different metal routings, and switching noises, such as clock feed-through from device mismatch, different metal switching noises, as clock feed-through mismatch, different metal routings, androutings, switchingand noises, such as clocksuch feed-through and charge and charge injection. The different parasitic capacitances of each column ADC cause the different and charge injection. different parasitic capacitances of each column cause thegain different injection. The differentThe parasitic capacitances of each column ADC causeADC the different and gain and linearity, finally resulting in column fixed-pattern noise (CFPN). In order to overcome such gain andfinally linearity, finally in column fixed-pattern noise (CFPN). In ordersuch to overcome such linearity, resulting inresulting column fixed-pattern noise (CFPN). In order to overcome limitation in limitation in conventional TSSS ADC, we propose an alternative TSSS ADC structure as shown in limitation inTSSS conventional TSSS ADC, we propose anADC alternative TSSS ADC structure as shown conventional ADC, we propose an alternative TSSS structure as shown in Figure 6b. Figure 7in Figure 6b. Figure 7 shows the timing diagram of proposed TSSS ADC with 4-input comparator that Figurethe 6b.timing Figure diagram 7 shows the timing diagram of proposed TSSS ADC with 4-input comparator that shows of proposed TSSS ADC with 4-input comparator that uses differential uses differential difference amplifier (DDA) for analog CDS operation. The differences of the uses differential difference amplifier (DDA) for analog CDS operation. The differences of the difference amplifier (DDA) for analog CDS operation. The differences of the proposed TSSS ADC from proposed TSSS ADC from conventional one are as following. First, SADC1 and SADC2 are proposed TSSS conventional one are following. First, SADC1 andtoSADC2 are conventional one ADC are as from following. First, SADC1 and as SADC2 are eliminated, leading low offset eliminated, leading to low offset voltage of comparator and reduced mismatch. Second, CH is eliminated, leading to offsetmismatch. voltage ofSecond, comparator and reduced mismatch. Second, H is voltage of comparator andlow reduced CH is connected to VRAMP_C which meansCthat connected to VRAMP_C which means that the fine ramp slope is not affected by the parasitic connected VRAMP_C means the fine ramp slope is not affected by the parasitic the fine ramptoslope is notwhich affected by thethat parasitic capacitances, leading to small variation of the slope capacitances, leading to small variation of the slope of VRAMP_F. ofcapacitances, VRAMP_F . leading to small variation of the slope of VRAMP_F. CH1

s1 VIN

s1

C1

VIN

CH

s2

CH1

s2

C1

C2

CHSADC2 FB SADC2 FB

SADC1 SADC1 VREF VREF

VRAMP VRAMP

(a) (a)

C2

Sync & FB Sync & control FB control

VIN VREFVIN VREF VRAMP_F VRAMP_C VRAMP_F VRAMP_C

s2 CH1 s2 CH1 s1 C1 s1 C1 inn1

FB

s3

C2

inn2 inn1 Outp inp2 inn2 Outp inp1 inp2

FB CH

Sync & Sync & FB FB control control

C2

inp1

Outp

VRAMP_C

CH

s3

VRAMP_F VIN

VRAMP_C BIAS2

VRAMP_F VIN

VREF Outp VREF

BIAS2 BIAS1 BIAS1

(b) (b)

Figure 6. Circuit diagram of (a) conventional TSSS ADC with analog CDS block and (b) proposed Figure6.6.Circuit Circuit diagram conventional TSSS analog CDS block and (b) proposed Figure diagram of of (a) (a) conventional TSSS ADCADC withwith analog CDS block and (b) proposed TSSS TSSS ADC with DDA. TSSSwith ADCDDA. with DDA. ADC

CH

SADC1

VIN BIAS2 BIAS1

VREF

VRAMP

(a)

(b)

6.1497 Circuit diagram of (a) conventional TSSS ADC with analog CDS block and (b) proposed 6 of 9 SensorsFigure 2017, 17, TSSS ADC with DDA.

Sensors 2017, 17, 1497

diagram for for proposed proposed TSSS TSSS ADC. ADC. Figure 7. Timing diagram

6 of 9

4. Experimental Results 4.1. Simulation Results 4.1. Simulation Results and and Chip Chip Photograph Photograph Figure ADC column column Figure 88 shows shows simulation simulation results results of of configurable configurable selection selection of of rows rows and and the the input input of of ADC array with different resolutions. As shown in Figure 8a, every row is selected at normal mode array with different resolutions. As shown in Figure 8a, every row is selected at normal mode (at (at the the top waveform of Figure 8a). For the 1/2 mode, the 1st and 2nd row are selected while the 3rd top waveform of Figure 8a). For the 1/2 mode, the 1st and 2nd row are selected while the 3rd andand 4th 4th With row control, finally pixelresolution resolutioncan canbe beconfigurable configurablefrom from full full resolution resolution rowrow are are not.not. With thisthis row control, finally pixel down to the 1/64 resolution mode depending on the situations of ISS. Like row control signals, down to the 1/64 resolution mode depending on the situations of ISS. Like row control signals, the the input of column ADC array configurable shown Figure While VIN every column input of column ADC array cancan be be configurable as as shown in in Figure 8b.8b. While VIN of of every column is is transferred from pixel array to each column ADC array, only selected column is operated with scaled transferred from pixel array to each column ADC array, only selected column is operated with scaled pixel pixel resolutions. resolutions.

1st 2nd

3rd 4th

1st 2nd

(a)

(b)

Figure 8. 8. Simulation results showing showing (a) (a) row row control control signals signalsand and(b) (b)pixel pixeloutput outputvoltage voltage(V (VIN)) for for the the Figure Simulation results IN input of column ADC array with different resolutions. input of column ADC array with different resolutions.

4.2. Measurement Results 4.2. Measurement Results Figure 9 shows the entire layout and microphotograph of the CIS fabricated with a 0.18 μm Figure 9 shows the entire layout and microphotograph of the CIS fabricated with a 0.18 µm 1-poly 1-poly 4-metal CMOS process. The area of sensor core is 2.89 mm2 (1.7 mm × 1.7 mm) and the chip 4-metal CMOS process. The area of sensor core is 2.89 mm2 (1.7 mm × 1.7 mm) and the chip area is area is 5.52 mm2 (2.35 mm × 2.35 mm). The area of 4-shared 4T-APS in the proposed CIS is 5.52 mm2 (2.35 mm × 2.35 mm). The area of 4-shared 4T-APS in the proposed CIS is 4 µm × 4 µm. 4 μm × 4 μm. In order to minimize CFPN, all columns are arranged to have identical repetition In order to minimize CFPN, all columns are arranged to have identical repetition pattern. Dummy pattern. Dummy pixels are added to each edge of pixel array to more even out the performance of pixels are added to each edge of pixel array to more even out the performance of effective pixels. effective pixels. The number of pixels is 176 × 144 (QCIF resolution) for full resolution (=the highest The number of pixels is 176 × 144 (QCIF resolution) for full resolution (=the highest resolution). resolution).

Figure 9 shows the entire layout and microphotograph of the CIS fabricated with a 0.18 μm 1-poly 4-metal CMOS process. The area of sensor core is 2.89 mm2 (1.7 mm × 1.7 mm) and the chip area is 5.52 mm2 (2.35 mm × 2.35 mm). The area of 4-shared 4T-APS in the proposed CIS is 4 μm × 4 μm. In order to minimize CFPN, all columns are arranged to have identical repetition pattern. Dummy pixels are added to each edge of pixel array to more even out the performance of Sensorseffective 2017, 17, 1497 pixels. The number of pixels is 176 × 144 (QCIF resolution) for full resolution (=the highest 7 of 9 resolution).

(a)

(b)

Figure 9. (a) The chip layout of the proposed CIS and (b) microphotograph of the fabricated CIS.

Figure 9.17, (a)1497 The Sensors 2017,

chip layout of the proposed CIS and (b) microphotograph of the fabricated CIS. 7 of 9

The chip is attached to a PCB via a chip-on-board (COB) process. For measurement, signals for

The chip is attached to a PCB via a chip-on-board (COB) process. For measurement, signals for control operations are generated with a field programmable gate array (FPGA). The measurement control operations are generated with a field programmable gate array (FPGA). The measurement environment is set up so that generated signals activate the CIS whose output is then stored in a environment is setthe upFPGA. so that signals sent activate the CIS whose output an is image then stored register inside Itsgenerated value is eventually to a computer which renders on its in a register inside the FPGA. Its value is eventually sent to a computer which renders an image computer monitor. We observe that power consumption is drastically reduced with the decrease ofon its computer monitor. We observe that is drastically reduced the decrease resolution. Power consumption forpower normal,consumption 1/2, 1/4, 1/8, 1/16, 1/32, and 1/64 modeswith is about 10, 5.2, of 2.7, 1.6,Power 0.9, 0.5,consumption and 0.3 mW, respectively. resolution. for normal, 1/2, 1/4, 1/8, 1/16, 1/32, and 1/64 modes is about 10, 5.2, Figure 10a shows images with different resolution modes. The one with highest resolution is 2.7, 1.6, 0.9, 0.5, and 0.3 mW, respectively. shown, followed by ones withwith 1/4, 1/16, and 1/64 resolutionmodes. respectively. Figure 10b highest shows the results Figure 10a shows images different resolution The one with resolution is from the analysis of testing charts as a way to analyze sample photographs for signal to noise (SNR) shown, followed by ones with 1/4, 1/16, and 1/64 resolution respectively. Figure 10b shows the measurement. With the decrease of resolution from full resolution mode to 1/64 mode, SNR is results from the analysis of testing charts as a way to analyze sample photographs for signal to noise reduced from 47.4 to 39.7 dB along with the decrease of power consumption from 10 to 0.3 mW. (SNR) measurement. With the decrease of resolution from full resolution mode to 1/64 mode, SNR is Table 1 summarizes the detailed specifications of the proposed CIS. And Table. 2 summarizes the reduced from 47.4comparison to 39.7 dB with alongother withworks the decrease ofcomparable power consumption from 10(such to 0.3asmW. Table 1 performance that have image resolutions CIF to summarizes the detailed specifications of the proposed CIS. And Table 2 summarizes the performance QVGA) and technology nodes to this work. The power figure of merit (FOM) [12] is given by: comparison with other works that have comparable image resolutions (such as CIF to QVGA) and Power FOM = . # of merit × (FOM) [12] is ∙ given by: technology nodes to this work. The power figure As shown in Table 2, the power FOM is relatively small and similar as that  of [13,14]. However, Total power consumption W unlike [13,14], the power . with 1/64 mode, Powerconsumption FOM = of CIS in our work can be reduced further # o f pixels × Frame rate pixels · f psof modes (peace mode while the CIS in [13,14] maintains the constant power consumption regardless and emergency mode in ISS).

(a)

(b)

Figure 10. (a) Measured images for multi-mode pixel and (b) measured images for standard chart to

Figure 10. (a) Measured images for multi-mode pixel and (b) measured images for standard chart to obtain SNR. obtain SNR. Table 1. Performance summary of the proposed CIS. Array Format Pixel Size Fill Factor Dynamic Range ADC Resolution Frame Rate

QCIF (176 × 144) 4.4 μm × 4.4 μm 9% 61.8 dB 8-bit 14 frame/s

Sensors 2017, 17, 1497

8 of 9

Table 1. Performance summary of the proposed CIS. Array Format

QCIF (176 × 144)

Pixel Size

4.4 µm × 4.4 µm

Fill Factor

9%

Dynamic Range

61.8 dB

ADC Resolution

8-bit

Frame Rate

14 frame/s

Power Supply

3.3 V (analog)/1.8 V (digital)

Power Consumption

10 mW (High-resolution mode) 0.3 mW (1/64 resolution mode) 11.3 µW (per column) 0.4 µW (per column @power shut off)

SNR

47 dB (High-resolution mode) 39.7 dB (1/64 resolution mode)

Area

5.52 mm2 (2.35 mm× 2.35 mm)

Process

Towerjazz 0.18 µm CIS

Table 2. Performance comparison of the proposed CIS (figure of merit: FOM). Ref.

Pixels

[9] [12] [13] [14] [15]

128 × 128 256 × 256 128 × 128 176 × 144 320 × 240 176 × 144 1/64 mode

This work

1

Technology (µm)

Frame Rate (fps)

Power (mW)

Power FOM (nW/pixels·fps)

Read-Out Method

0.35 0.35 0.13 0.25 0.35

30 30 9 30 15 14 896

30 75 16 20 30 10 0.3

61.04 38.01 108.51 26.31 26.04 28.18 0.85

Single ADC (SAR 1 ) Column ADC (SS 2 ) In-pixel ADC External ADC Column ADC (SS)

0.18

Column ADC (TSSS 3 )

Successive approximation ADC, 2 Single-Slope ADC, and 3 Two-step Single-Slope ADC.

As shown in Table 2, the power FOM is relatively small and similar as that of [13,14]. However, unlike [13,14], the power consumption of CIS in our work can be reduced further with 1/64 mode, while the CIS in [13,14] maintains the constant power consumption regardless of modes (peace mode and emergency mode in ISS). 5. Conclusions A CIS (with QCIF resolution) with an improved TSSS ADC that supports configurable resolutions (1, 1/2, 1/4, 1/8, 1/16, 1/32, and 1/64 resolution mode) is proposed for intelligent surveillance system applications. With the direct connection of fine ramp signals to the comparators, parasitic capacitances causing mismatch and CFPN are reduced. In addition, the measurement results show that figure of merit (FOM) of QCIF CIS is 5.524 µW·fps, which represents a lowest power consumption than ever described in the previous research. The power consumption can further be reduced from 10 to 0.3 mW by lowering the resolution mode from full resolution mode to 1/6 resolution mode. Therefore, the proposed CIS can be used for ultra-low power image sensors in consumer electronics that require always-on sensing features. Acknowledgments: This work was supported in part by the Dongguk University Research Fund of 2017 for author, Soo Youn Kim and in part by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2016-H8501-16-1010) supervised by the IITP (Institute for Information & communications Technology Promotion) for other authors. Author Contributions: D.K. and M.S. conceived and designed the circuits. B.C. and S.K. performed the experiments and analyzed the data. All authors were involved in the preparation of this manuscript.

Sensors 2017, 17, 1497

9 of 9

Conflicts of Interest: The authors declare no conflict of interest.

References 1. 2. 3.

4.

5.

6.

7. 8. 9. 10.

11. 12.

13. 14.

15.

Ibrahim, S.W. A comprehensive review on intelligent surveillance systems. Commun. Sci. Technol. 2016, 1, 7–14. Fernandez, J.; Calavia, L.; Baladr, C. An intelligent surveillance platform for large metropolitan areas with dense sensor deployment. Sensors 2013, 13, 7414–7442. [CrossRef] [PubMed] Choi, J.; Park, S.; Cho, J.; Yoon, E. A 1.36 µW adaptive CMOS image sensor with reconfigurable modes of operation from available energy/illumination for distributed wireless sensor network. In Proceedings of the IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), San Francisco, CA, USA, 19–23 February 2012; pp. 112–114. Couniot, N.; Streel, G.; de Botman, F.; Lusala, K.A.; Flandre, D.; Bol, D. A 65 nm 0.5 V DPS CMOS Image Sensor with 17 pJ/Frame.Pixel and 42 dB Dynamic Range for Ultra-Low-Power SoCs. IEEE J. Solid-State Circuits 2015, 50, 2419–2430. [CrossRef] Choi, J.; Park, S.; Cho, J.; Yoon, E. A 3.4-µW Object-Adaptive CMOS Image Sensor with Embedded Feature Extraction Algorithm for Motion-Triggered Object-of-Interest Imaging. IEEE J. Solid-State Circuits 2014, 49, 289–300. [CrossRef] Snoeij, M.F.; Theuwissen, A.J.P.; Makinwa, K.A.A.; Huijsing, J.H. A CMOS Imager with Column-Level ADC Using Dynamic Column Fixed-Pattern Noise Reduction. IEEE J. Solid-State Circuits 2006, 41, 3007–3015. [CrossRef] Snoeij, M.F.; Theuwissen, A.J.P.; Makinwa, K.A.A.; Huijsing, J.H. Multiple-ramp column-parallel ADC architectures for CMOS image sensors. IEEE J. Solid-State Circuits 2007, 42, 2968–2977. [CrossRef] Lee, D.; Han, G. High-speed, low-power correlated double sampling counter for column-parallel CMOS imagers. Electron. Lett. 2007, 43, 1362–1364. [CrossRef] Artyomov, E.; Rivenson, Y.; Levi, G.; Yadid-Pecht, O. Morton (Z) Scan Based Real-Time Variable Resolution CMOS Image Sensor. IEEE Trans. Circuits Syst. Video Technol. 2005, 15, 947–952. [CrossRef] Huang, H.Y.; Conge, P.A.; Huang, L.W. CMOS Image Sensor Binning Circuit for Low-Light Imaging. In Proceedings of the 2011 IEEE Symposium on Industrial Electronics and Applications (ISIEA), Langkawi, Malaysia, 25–28 September 2011; pp. 586–589. Lim, S.; Lee, J.; Kim, D.; Han, G. A high-speed CMOS image sensor with column-parallel two-step single-slope ADCs. IEEE Trans. Electron. Devices 2009, 56, 393–398. [CrossRef] Choi, J.; Han, S.; Kim, S.; Chang, S.; Yoon, E. A Spatial-Temporal Multiresolution CMOS Image Sensor with Adaptive Frame Rates for Tracking the Moving Objects in Region-of-Interest and Suppressing Motion Blur. IEEE J. Solid-State Circuits 2007, 42, 2978–2988. [CrossRef] Hanson, S.; Sylvester, D. A 0.45–0.7 V Sub-Microwatt CMOS Image Sensor for Ultra-Low Power Applications. In Proceedings of the 2009 Symposium on VLSI Circuits, Kyoto, Japan, 16–18 June 2009; pp. 176–177. Yamada, T.; Kasuga, S.; Murata, T.; Kato, Y. A 140 dB-Dynamic-Range MOS Image Sensor with In-Pixel Multiple-Exposure Synthesis. In Proceedings of the IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), San Francisco, CA, USA, 3–7 February 2008; pp. 50–51. Ham, S.H.; Lee, Y.H.; Jung, W.K.; Lim, S.H.; Yoo, K.S.; Chae, Y.C.; Cho, J.H.; Lee, D.M.; Han, G.H. A CMOS Image Sensor with Analog Gamma Correction Using a Nonlinear Single Slope ADC. In Proceedings of the 2006 IEEE International Symposium on Circuits and Systems (ISCAS), Island of Kos, Greece, 21–24 May 2006; pp. 3578–3581. © 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).