Neuromorphic Implementation of a Software-defined ...

Invited Paper

Neuromorphic Implementation of a Software-defined Camera that can “See” through Fire and Dust in Real-time Jae H. Cha and A. Lynn Abbott Virginia Polytechnic and State University, Blacksburg, Virginia, USA Harold H. Szu, Jefferson Willey, Joseph Landa Catholic University of America, Washington DC, USA Keith A. Krapels University of Memphis, Memphis, Tennessee, USA ABSTRACT Software-defined Cameras (SDC) based on Boltzmann’s molecular thermodynamics can “see” through visually-degraded fields such as fire, fog, and dust in some situations. This capability is possible by means of unsupervised learning implemented on a neuromorphic algo-tecture. This paper describes the SDC algorithm design strategy with respect to nontrivial solutions, stability, and accuracy. An example neuromorphic learning algorithm is presented along with unsupervised learning stopping criteria. Keywords: software-defined camera, molecular thermodynamics, neuromorphic learning algorithm.

1.

INTRODUCTION

Obscurants such as fire and dust create a challenging environment for imaging: dust particles scatter visible lights, and fire saturates sensors. (Fig. 1). In particular, fire can overwhelm the field of view of a thermal imaging sensor with intensive radiation (~3000 K), and when presented to observers can cause important cues such as humans (~310K) in the scene to go unnoticed due to the limited dynamic range of displays (typically 8 bits per channel) and of the human visual system. To overcome this situation, it is a common practice to apply (linear/nonlinear) histogram-based level adjustment, so that the more intensive radiation signature is suppressed and the lesser one is enhanced. Even such a technique, however, can fail to accomplish the goal when the two objects overlay in space in a way that the fire obscures the human. The concept of the software-defined camera (SDC), discussed in [1], reflects an observation: while sensing is “nature” and could be done via active radar or passive EO-IR, displaying must be “nurture” and is performed for human visual systems (HVS). Our implementation of the SDC based on Boltzmann’s molecular thermodynamics involves manipulation of sensing and displaying at the optical radiation source level. For that reason, we convert the measured spectral data into equivalent entropy/temperature sources. Then after identifying the sources spatiotemporally, we correct estimates for errors caused by dipole/monopole scattering at boundaries (