Adaptive Time-Sequential Binary Sensing for High ...

Adaptive Time-Sequential Binary Sensing for High Dynamic Range Imaging Chenhui Hu and Yue M. Lu School of Engineering and Applied Sciences Harvard University, 33 Oxford Street, Cambridge, MA 02138 ABSTRACT We present a novel image sensor for high dynamic range imaging. The sensor performs an adaptive one-bit quantization at each pixel, with the pixel output switched from 0 to 1 only if the number of photons reaching that pixel is greater than or equal to a quantization threshold. With an oracle knowledge of the incident light intensity, one can pick an optimal threshold (for that light intensity) and the corresponding Fisher information contained in the output sequence follows closely that of an ideal unquantized sensor over a wide range of intensity values. This observation suggests the potential gains one may achieve by adaptively updating the quantization thresholds. As the main contribution of this work, we propose a time-sequential threshold-updating rule that asymptotically approaches the performance of the oracle scheme. With every threshold mapped to a number of ordered states, the dynamics of the proposed scheme can be modeled as a parametric Markov chain. We show that the frequencies of different thresholds converge to a steady-state distribution that is concentrated around the optimal choice. Moreover, numerical experiments show that the theoretical performance measures (Fisher information and Cram´er-Rao bounds) can be achieved by a maximum likelihood estimator, which is guaranteed to find globally optimal solution due to the concavity of the log-likelihood functions. Compared with conventional image sensors and the strategy that utilizes a constant single-photon threshold considered in previous work, the proposed scheme attains orders of magnitude improvement in terms of sensor dynamic ranges. Keywords: Adaptive sensing, binary sensing, high dynamic range imaging, quantization

1. INTRODUCTION Compared to the human visual system, current imaging sensors have much narrower dynamic ranges. This limitation becomes evident when one tries to capture scenes containing both very bright and very dark parts. Several solutions, either software-based1–3 or sensor-based,4–7 have been proposed to address this issue. In this work, we consider a binary sensing scheme for high dynamic range imaging that is proposed in several recent work.8–10 Reminiscent of conventional photographic film, the new sensor has a binary pixel response: the output is 1 if the number of photons reaching that pixel during exposure is no less than a given threshold q; and it is 0 otherwise. Essentially, every pixel only provides a one-bit quantized measurement of the local light intensity. To compensate for the loss of information due to coarse quantization, the binary sensor is spatially (or temporally) oversampled, a scheme that is conceptually similar to classical oversampled analog-to-digital conversions.11, 12 The performance of the above binary sensing scheme was studied in detail in Ref. 10, which formulates the image reconstruction task as a parameter estimation problem based on quantized Poisson statistics. The analysis in that work shows that, with sufficiently large oversampling factors, the new binary sensor can have higher dynamic ranges than traditional sensors. In particular, when the quantization threshold is chosen to be q = 1 and when the oversampling factor tends to infinity, the performance of the binary sensor (measured in terms of signal-to-noise ratios) approaches that of an ideal, unquantized, sensor. The intuition behind these results can be easily explained. On the one hand, with a threshold q = 1, the quantized sensor measurement creates ambiguity only when the binary output is 1, in which case the actual photon count can be any integer value from {1, 2, 3, . . .}; On the other hand, photon arrivals at each pixel can be modeled as a Poisson random process whose rate is equal to ctotal /K, where ctotal is the local light intensity and K is the oversampling factor. For fixed ctotal and as K grows to infinity, the rate of the Poisson process E-mail: {hu4, yuelu}@seas.harvard.edu; WWW: http://lu.seas.harvard.edu

becomes infinitesimally small and thus the probability of actually getting two or more photons tends to zero. In other words, with very high probability, the one-bit sensor does not lose information and it operates as if there were no quantization. The above line of reasoning still works well when the oversampling factor K is finite, which is always the case in practice. One just needs to ensure that cmax /K < 1, where cmax is the maximum incoming light intensity. This requirement imposes a constraint on the dynamic ranges achieved by the binary sensor: When cmax becomes greater than K, the probability that a pixel receives two or more photons may no longer be negligible. In this case, the sensor starts losing information in its binary measurements. Furthermore, when cmax ≫ K, it is expected that the sensor outputs (with q = 1) will be mostly dominated by 1’s, which do not provide much information for estimating the light intensity and many measurements are essentially wasted. In this work, we propose an adaptive binary sensing scheme that can update the quantization thresholds in a time-sequential manner. We show that, for any given incoming light intensity level, there exists a corresponding optimal threshold that maximizes the Fisher information of the output binary sequence. The optimal Fisher information is observed to follow closely that of an ideal unquantized sensor over a very wide range of intensity levels. The challenge here is how to achieve this oracle performance: choosing the optimal threshold requires knowledge of the incoming light intensity, which is the unknown parameter we want to estimate in the first place. As the main contribution of this work, we propose a dynamic threshold-updating rule that asymptotically approaches the performance of the oracle scheme. We model the sensing system as a finite-state machine and map every threshold q = 1, 2, 3, . . . to a number of ordered states. The system moves from state to state according to the one-bit measurement the sensor generates in the current observation window. Since the dynamics of the system only depends on its current state and the output of a one-bit sensor measurement, we can model it as a parametric Markov chain.13, 14 We show that the proposed Markov chain has fast mixing and that the steady-state probability distributions of the thresholds concentrate around the optimal (i.e., oracle) choice. Moreover, we show that the theoretical performance measures (Fisher information and Cram´er-Rao bounds) can be achieved by a maximum likelihood estimator, which is guaranteed to find globally optimal solution due to the concavity of the log-likelihood functions. Compared with the constant thresholding scheme considered in the previous work,10 the proposed adaptive binary sensing scheme provides orders of magnitude improvement in dynamic ranges, suggesting its effectiveness and advantages. The rest of the paper is organized as follows. After a brief overview of the binary sensing model in Section 2, we study the optimal threshold choices and discuss the Fisher information of the oracle scheme in Section 3. The proposed dynamic threshold-updating rule is presented in Section 4, where we compare its performance against the oracle scheme. Section 5 concludes the paper. We focus on conceptual ideas and numerical verifications in this paper, and leave the proofs of several theoretical results and more detailed analysis to a forthcoming paper.

2. THE OVERSAMPLED BINARY IMAGING MODEL In this section, we briefly review the imaging model of the oversampled binary sensor studied in this work. In general, the sampling of the light intensities can be done in space (as is emphasized in Ref. 10) or over time. In this paper, we focus on the latter case. Let τ be the total exposure time to capture a single image. We assume that τ ≪ 1 and that the light intensity incident on the pixel stays constant during the entire exposure. As shown in Figure 1, the binary sensor divides [0, τ ] into K subintervals and generates one binary measurement within each subinterval. Let ctotal denote the total light exposure during [0, τ ]. Then, the number of photons arriving at the kth subinterval is the realization of a Poisson random variable with mean equal to c = ctotal /K, i.e., P(Yk = yk ; c) =

cyk e−c , yk !

for yk ∈ N.

(1)

In what follows, we refer to c as the normalized light intensity. The sensor provides one-bit quantized measurements of the photon counts y1 , y2 , · · · , yK by comparing them against a set of thresholds q1 , q2 , · · · , qK

Constant light intensity FWRWDO

Photon counting

y1

y2

yk

yK

Binary quantization

q1

q2

qk

qK

b1

b2

bk

bK

Binary measurements Figure 1. An illustration of the the oversampled binary sensing scheme. The light intensity ctotal remains constant during K observation windows. The number of photons y1 , y2 , · · · , yK received within each window is assumed to be generated by Poisson random variables with mean c = ctotal /K. Comparing the photon counts against thresholds qk , 1 ≤ k ≤ K, the sensor generates a sequence of binary measurements: bk = 0 if yk < qk (shown as shadowed pixels); otherwise, bk = 1 (shown as white pixels).

respectively, yielding a binary sequence

{ 1, if yk ≥ qk ; bk = 0, otherwise.

(2)

Since the photon counts {yk } are drawn from Poisson random variables {Yk }, the binary sensor output {bk } are also realizations of some random variables {Bk }. Defining the following functions def

p0 (c, q) = with q ∈ N, we have that

q−1 m −c ∑ c e , m! m=0

def

and p1 (c, q) = 1 − p0 (c, q)

P(Bk = bk ; c) = pbk (c, qk ),

for bk ∈ {0, 1}.

(3)

(4)

The goal of image reconstruction is then to estimate the parameter c from the K binary measurements {bk }.

3. OPTIMAL THRESHOLDS AND THE ORACLE SCHEME In previous work,10 the thresholds q1 , q2 , · · · , qK are fixed (e.g., they are all equal to one), whereas in this work, we allow the threshold to be adjusted at every subinterval, based on the obtained outcome up to this point. Before presenting the proposed adaptive threshold-updating scheme, we first study the optimal threshold one should choose if the light intensity level is known. Consider the situation where the thresholds q1 , q2 , . . . , qK are predetermined in the experiments. In this case, the outcome of every experiment is fully independent of each other. The Fisher information of the binary output is specified by the following lemma: Lemma 3.1. The Fisher information contained in a binary measurement bk of the normalized light intensity c when a threshold qk ≥ 1 is applied can be written as I0 (c, qk ) =

p′0 (c, qk )2 , p0 (c, qk ) p1 (c, qk )

for c > 0,

(5)

12

Optimal threshold

10

8

6

4

2

0

0

1

2

3

4

5

6

7

8

9

10

Normalized light intensity c˜

Figure 2. Optimal thresholds qopt (c) as a function of the normalized light intensity c.

def

qk −1

∂ where p′0 (x, qk ) = ∂x p0 (x, qk ) = −e−x (qxk −1)! . The Fisher information of the entire binary sequence is thus the sum of the individual Fisher information, i.e.,

IK (c) =

K ∑

I0 (c, qk ).

(6)

k=1

For each normalized light intensity level c, there exists an optimal threshold qopt (c) = argmaxqk ∈N I0 (c, qk ) which maximizes the Fisher information in (5). To see this, we compute the Fisher information with respect to different thresholds and find the optimal choices over a range of intensity levels c ∈ [0, 10]. The relation between the optimal threshold and c turns out to be a simple staircase function, as characterized in Figure 2. We observe that the optimal threshold qopt (c) is either ⌈c⌉ or ⌈c⌉ + 1, where ⌈c⌉ is the smallest integer that is greater than or equal to c. It is easy to see that the aggregate Fisher information in (6) has an upper bound def

IK (c) ≤ KI0 (c, qopt ) = Iopt (c), and the bound can be attained if we let all the thresholds to be equal to qopt (c), which we shall refer to as the oracle scheme. We note that above bound also applies to the more general case when the threshold can be adaptively adjusted according to previous sensor measurements. We omit the proof of this statement. In Figure 3, we plot the Fisher information of three different sensing schemes: the oracle scheme, the ideal scheme (i.e., a sensor that does not do any quantizations), and the constant thresholding scheme with different thresholds (q = 1, 3, 5, 7, 9). The Fisher information for the latter two cases can be shown to be Iideal (c) = K/c

and Iconst (c) = KI0 (c, q),

respectively. We provide two remarks on this figure. First, since the oracle scheme is obtained by choosing the optimal threshold for each c, its Fisher information is the upper envelope of the curves associated with constant threshold schemes. Second, the performance of the oracle scheme is very close to that of the ideal unquantized scheme, despite the coarse one-bit quantization used in the former setting.

4

10

Ideal sensor Oracle scheme 2

10

Fisher information

0

10

q=3 q=5

−2

10

q=7

q=1

q=9 −4

10

−6

10

−8

10

0

1

2

3

4

5

6

7

8

9

10

Normalized light intensity c Figure 3. Fisher information of different schemes with K = 213 : the yellow curve corresponds to the oracle scheme obtained by selecting the optimal threshold for each light intensity; the green dashed curve shows the Fisher information of an ideal unquantized sensor, which gives the best performance for all possible quantized schemes. The remaining curves correspond to constant threshold schemes with different thresholds.

4. ADAPTIVE THRESHOLD-UPDATING SCHEME WITH NEAR-ORACLE PERFORMANCE The oracle binary sensing scheme discussed in the previous section cannot be directly implemented as it requires the knowledge of the incoming light intensity. In this section, we propose a time-sequential adaptive thresholdupdating scheme that asymptotically approaches the performance of the oracle scheme.

4.1 Parametric Markov Chains for Updating Thresholds Although we do not know the incoming light intensity, the outcomes of the one-bit comparisons do contain information about it and thus we can update the thresholds by making use of those results. For example, we may adjust the threshold according to the very last outcome: Seeing a bit 1 suggests that the current threshold may be too low, and we therefore increase it by 1; and vice versa. This simple policy acts like a feedback controller of the threshold, moving it back and forth to track the optimal choice. More generally, we consider the following strategy that determines the thresholds probabilistically. We map every threshold to a series of 2L states, for some integer L. In Figure 4, we illustrate the situation when L = 2. The states {sq,00 , sq,01 , sq,10 , sq,11 } in the system correspond to the threshold q. All the states are placed along a line, with the associated thresholds increasing from left to right. Suppose that at the kth observation window we are at a certain state and have just obtained a new binary outcome bk , then the state of the system at the next time interval is determined as follows: With probability β > 0, we stay at the current state; Otherwise, we move one step to the right if bk = 1 or move one step to the left if bk = 0. Denote by Xk the random variable

p1 (c,1)

1 ! p1 (c,1)

s1,00

"

p0 (c,1)

p1 (c,1)

s1,01

"

p0 (c,1)

p1 (c, q)

p1 (c,1)

s1,10

"

s1,11

"

sq ,00

p0 (c,1)

"

p1 (c, q )

sq ,01

p0 (c, q )

"

p1 (c, q )

sq ,10

p0 (c, q )

"

sq ,11

p0 (c, q )

Figure 4. Representation of the register scheme in a form of parametric Markov chain. Every threshold is mapped to 2L ordered states, e.g. {sq,00 , sq,01 , sq,10 , sq,11 } are corresponding to the threshold equal to an integer q for L = 2. When we are at a certain state and receiving a new binary outcome, the state jumps to its neighbors or stays invariant with probabilities indicated along the corresponding arrows. α, β is the jumping probability and the staying probability, respectively.

encoding the state of the system at step k and let α = 1 − β, we have, for m ≥ 2,   for △m = 0; β, P(Xk+1 = m + △m | Xk = m) = α p1 (c, q(m)) , for △m = 1;   α p0 (c, q(m)) , for △m = −1, where q(m) is the threshold associated with the current state Xk = m. Furthermore, we let the initial state of the system to be drawn from a distribution [ ] π 0 = π0 (s1,00 ), π0 (s1,01 ), · · · , π0 (sq,00 ), · · · , each entry of which denotes the probability at a given state. It is evident that the proposed adaptive threshold-updating scheme can be mathematically represented by a parametric Markov chain. In practice, by assuming that the normalized light intensity is limited within the range (0, cmax ], we can set a maximum threshold value qmax = ⌈cmax ⌉ + 1, in which case the chain contains a total of 2L qmax states. Denoting by P (c) the one-step transition matrix of the Markov chain and by ] def [ π k = P(Xk = 1), P(Xk = 2), . . . , P(Xk = 2L qmax ) the probability distribution of the system states at step k, we have π k = P k (c) π 0 .

(7)

With suitable choices of the “staying probability” β, the “mixing” of the proposed Markov chain, or equivalently, the convergence of the matrix power P k (c), can be accelerated. We demonstrate this behavior in a numerical experiment in which we choose a particular incoming light intensity c = 5.6 and let K = 213 , L = 2, qmax = 10, and β = 0.1. Furthermore, we start the chain from the rightmost state, corresponding to an initial threshold q0 = 10. Figure 5 shows the probability distributions of the quantization thresholds at each step, obtained by using (7) and by summing up probabilities of those states that are associated with a common threshold. The results of Figure 5 confirms the rapid mixing of the Markov chain, with the probability distributions stay almost constant after only 100 steps. Furthermore, we observe that three particular thresholds (q = 5, 6, 7) dominate the steady-state probabilities, whereas other thresholds will hardly be chosen. It is not a coincidence that the optimal threshold for c = 5.6 is equal to 6, as shown in Figure 2. In general, one can show that the steady-state probability distributions of the quantization thresholds will always be tightly concentrated around the optimal choices. We omit rigorous analysis of this property, and just provide the following intuitive explanations. Consider two situations: First, suppose we are at a threshold q that is far away from the optimal one. In this case, we are either going to get

1 0.9 q = 10

Probability of threshold

0.8 0.7

q=8

0.6

q=6

0.5 0.4 0.3 q=5 q=7

0.2 0.1 0

q=4 50

100 150 Number of steps

200

250

Figure 5. Convergence of the probability distributions of the quantization thresholds. Each curve corresponds to the probability of a particular threshold as a function of the time steps. The initial threshold is 10 and the parameters used are c = 5.6, K = 213 , L = 2, qmax = 10, and β = 0.1, respectively.

0 with high probability (when q is “too high”) or 1 with high probability (when q is too low). In both cases, the system will move away from the current state; Second, suppose we are already at the optimal threshold. In this case, the probabilities of getting 0 and getting 1 are more balanced. Since there are 2L states associated with the optimal threshold, we end up moving back and forth among those states, which form a stable “basin of attraction”. The steady-state probability of the Markov chain, denoted by π(c), can be obtained from the balance equation π(c) = π(c)P (c). The performance of the proposed adaptive threshold-updating scheme mainly depends on the steady-state distribution of thresholds. More formally, we present the following theorem related to the Fisher information. Theorem 4.1. Suppose that IK (c) is the Fisher information of the outcome binary sequence when we use the proposed adaptive threshold-updating scheme. Then we have q∑ max IK (c) = π(c, q)I0 (c, q), K→∞ K q=1

lim

(8)

where π(c, q) is the steady-state probability of choosing threshold q and I0 (c, q) is given in (5). The above theorem indicates that,∑ for large values of K, the Fisher information of the proposed time-sequential qmax scheme is approximately equal to K q=1 π(c, q)I0 (c, q). We plot this quantity in Figure 6 and compare it with the Fisher information of the ideal scheme and that of the oracle scheme. The match between the proposed scheme and the oracle scheme is almost exact.

4.2 Image Reconstruction by MLE The output of the proposed adaptive scheme is a sequence of quantization thresholds {q1 , q2 , . . . , qK } and bits {b1 , b2 , . . . , bK }. Denote by Kq the number of thresholds in the sequence that are equal to q, and by Nq the

2

10

Ideal sensor Oracle scheme Register scheme 1

Fisher information

10

0

10

−1

10

−2

10

0

1

2

3

4

5

6

7

8

9

10

Normalized light intensity c

Figure 6. The Fisher information computed from the steady-state distribution of the proposed time-sequential scheme (black dots) and those of an ideal unquantized sensor (red dashed curve) and an oracle scheme (blue curve). The proposed scheme achieves near-oracle performance. The parameters used in generating this figure are K = 213 , L = 2, qmax = 10, and β = 0.1.

number of binary outcomes there are equal to 1 when the threshold is q. With the above notation, the loglikelihood function of the unknown parameter c can be written as ℓ(c) =

q∑ max

[ ] (Kq − Nq ) log p0 (c, q) + Nq log p1 (c, q) + const.

(9)

q=1

The maximum likelihood estimator (MLE) of c can then be obtained by maximizing ℓ(c), i.e., e cMLE = arg max ℓ(c). c

The following theorem establishes the concavity of the log-likelihood function, which ensures that the MLE problem has a unique maximizing point. Theorem 4.2. For an arbitrary experiment results described by the parameters {Kq } and {Nq }, the log-likelihood function ℓ(c) defined in (9) is a concave function on the domain c ∈ [0, cmax ]. The concavity of the log-likelihood function implies that we can solve for the MLE by applying local search/descent algorithms. For those algorithms, it is often helpful to have a reasonable starting point c(0) to accelerate convergence. A simple method is to choose the starting point to be the threshold that is adopted most frequently, i.e., e c(0) = arg max πe (q), q

where πe (q) is the empirical frequency of choosing the threshold q in the output sequence. When q = 1 is the most frequently used one, we can use an even better initial estimate by setting e c(0) = − log(1 − N1 /K1 ),

70

60

SNR (dB)

50

40

30

20

10

0 1 10

2

10

3

10

4

5

10 10 Light exposure values

6

10

7

10

Figure 7. Comparison of the SNRs for different sensing schemes over a wide range of light exposure values ctotal (shown in logarithmic scale). The green dashed line indicates the ideal unquantized sensor; The yellow line illustrates the oracle scheme; The blue dotted curve shows the constant single-photon thresholding scheme with K = 213 ; The two red curves correspond to the proposed register scheme with K = 213 , L = 2, β = 0.1 and different maximum threshold values (from left to right, qmax = 10 and qmax = 100, respectively).

which is the closed-form solution of the MLE in the constant threshold scheme when q = 1 (see Ref. 10 for a derivation). To verify the performance of the proposed time-sequential scheme, numerical experiments are carried out over light intensities within a finite range, i.e. c ∈ (0, cmax ]. We consider two scenarios with either cmax = 10 or cmax = 100. In each case, qmax is set to be equal to cmax and the rest of the parameters are K = 213 , L = 2, and β = 0.1, respectively. For each incoming light intensity level, we solve cMLE by using the simplex search method 15 provided in Matlab. Then, the MLE of the total light intensity is b cMLE = K e cMLE . We measure the performance of the reconstruction in terms of the signal-to-noise ratios (SNRs), defined as SNR(c) = 10 log10

c2 [ ], E (b cMLE − c)2

[ ] where the mean squared error E (b cMLE − c)2 is obtained by the sample mean of Monte-Carlo simulations: For each light intensity c, we repeat the experiment 500 times. We compare the proposed time-sequential adaptive scheme with the constant single-photon thresholding scheme, the oracle scheme, as well as the ideal unquantized sensor in Fig. 7. Suppose SNRmin is the minimum SNR required in a specific application, we can then define the dynamic range of a sensor as the range of c for which the SNR is at least SNRmin . As the figure demonstrates, the proposed scheme greatly outperforms the constant thresholding scheme in terms of the achievable dynamic ranges. If we choose SNRmin = 20dB, then the dynamic range of the constant thresholding scheme with K = 213 is roughly between c = 102 and c ≈ 104.7 , or, if expressed in terms of ratios, 102.7 : 1. In contrast, using the same oversampling factor K = 213 , the proposed scheme with qmax = 10 and qmax = 100 have dynamic ranges equal to 103.25 : 1 and 104.04 : 1, respectively. In the latter case, the dynamic range is about 22-times higher than that of the constant thresholding scheme. We

(a)

(b)

Figure 8. High dynamic range photography using the proposed adaptive binary sensing scheme. (a) Four images taken in front of the Jefferson National Expansion Memorial at night with increasing exposure times.16 (b) The tone-mapped version of the reconstructed radiance map.

can also observe from Fig. 7 that the ideal scheme stands as the limiting performance for all possible quantized sensing schemes and that the oracle scheme follows it closely in a wide range of c. The two versions of the proposed time-sequential scheme match the oracle scheme very well over their respective operating ranges. To further test the performance of the proposed adaptive binary sensing scheme, we simulate the acquisition of a high dynamic range radiance map obtained from a set of four images shown in Figure 8(a). The scene contains both very bright (e.g., the lit courthouse) and very dark regions (e.g., the sky and the ground), resulting a dynamic range of about 3.5 × 106 : 1. Such a high dynamic range is far beyond the capabilities of conventional image sensors. However, the time-sequential threshold-updating scheme handles it very well when we set the parameters to be K = 216 , L = 2, qmax = 100, and β = 0.1. We can see from Figure 8(b), a tone-mapped version of the reconstruction, that details in both light and shadow regions have been faithfully preserved. This suggests the potential of the proposed adaptive scheme in high dynamic range photography.

5. SUMMARY We presented an adaptive binary sensing scheme which is featured by both the utilization of one-bit pixels and the dynamic updating of the thresholds in a time-sequential manner. We first considered the Fisher information of a so-called oracle scheme, which chooses the optimal threshold based on the knowledge of the true light intensity levels. We observe that the performance of the oracle scheme is very close to that of an ideal unquantized sensor, which suggests the potential performance gains that can be achieved by adapting quantization thresholds. As the main contribution of this work, we proposed a time-sequential threshold-updating scheme with near-oracle performance. We showed that the proposed scheme can be modeled as a parametric Markov chain, whose steadystate probability distributions are centered around the optimal thresholds. The unknown light intensities can then be obtained by using the maximum likelihood estimator. The global optimality of local search method is guaranteed by the concavity of the log-likelihood functions. Numerical results verify our theoretical analysis. They show that the proposed time-sequential scheme closely follows the performance of the oracle scheme and outperforms the constant thresholding scheme by significantly expanding the achievable dynamic ranges.

REFERENCES 1. Debevec, P. E. and Malik, J., “Recovering high dynamic range radiance maps from photographs,” in [Proc. of SIGGRAPH ], 369–378 (1997). 2. Mitsunaga, T. and Nayer, S. K., “Radiometric self calibration,” in [IEEE Conf. on Computer Vision and Pattern Recognition ], 374–380 (1999).

3. Kao, W. C., “High dynamic range imaging by fusing multiple raw images and tone reproduction,” IEEE Trans. Consum. Electr. 54(5), 10–15 (2008). 4. Cheng, H. Y., Choubey, B., and Collins, S., “A high-dynamic-range integrating pixel with an adaptive logarithmic response,” IEEE Photonics Technology Letters 19(15), 1169–1171 (2007). 5. Guo, J. and Sonkusale, S., “A high dynamic range CMOS image sensor for scientific imaging applications,” IEEE Sensors Journal 9(10), 1209–1218 (2009). 6. Guo, X., Qi, X., and Harris, J. G., “A time-to-first-spike CMOS image sensor,” IEEE Sensors Journal 7(8), 1165–1175 (2007). 7. Stoppa, D., Vatteroni, M., Covi, D., Baschirotto, A., Sartori, A., and Simoni, A., “A 120-dB dynamic range CMOS image sensor with programmable power responsivity,” IEEE Journal of Solid-State Circuits 42(7), 1555–1563 (2007). 8. Fossum, E. R., “What to do with sub-diffraction-limit (SDL) pixels? – A proposal for a gigapixel digital film sensor (DFS),” in [IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors], 214–217 (June 2005). 9. Sbaiz, L., Yang, F., Charbon, E., S˝ usstrunk, S., and Vetterli, M., “The gigavision camera,” in [IEEE Conf. on Acoust., Speech and Signal Proc.], 1093–1096 (2009). 10. Yang, F., Lu, Y. M., Sbaiz, L., and Vetterli, M., “Bits from photons: oversampled image acquisition using binary poisson statistics,” IEEE Trans. Image Proc. 21(4), 1421–1436 (2012). 11. Candy, J. C. and Temes, G. C., [Oversampling Delta-Sigma Data Converters – Theory, Design and Simulation], IEEE Press, New York, NY (1992). 12. Cvetkovi´c, Z. and Daubechies, I., “Single-bit oversampled A/D conversion with exponential accuracy in the bit rate,” IEEE Trans. Inf. Theory 53(11), 3979–3989 (2007). 13. Ross, S. M., [Stochastic Processes], John Wiley & Sons, New York, NY, second ed. (1996). 14. Pinsky, M. A. and Karlin, S., [An Introduction to Stochastic Modeling], Academic Press, Burlington, MA, fourth ed. (2010). 15. Lagarias, J. C., Reeds, J. A., Wright, M. H., and Wright, P. E., “Convergence properties of the Nelder-Mead simplex method in low dimensions,” SIAM J. on Optimization 9(1), 112–147 (1998). 16. http://www.chaosreigns.com/hdr/photography/.