Modular, uncooled video engines based on a DSP ...

Modular, uncooled video engines based on a DSP processor F. Schapiro, Y. Milstain, A. Aharon, , A. Neboshchik, Y. Ben-Simon, I. Kogan, , I. Lerman, U. Mizrahi, S. Maayani, A. Amsterdam, I. Vaserman, O. Duman, R. Gazit

SemiConductor Devices (SCD), P.O. Box 2250, Haifa 31021, Israel ABSTRACT The market demand for low SWaP (Size, Weight and Power) uncooled engines keeps growing. Low SWaP is especially critical in battery-operated applications such as goggles and Thermal Weapon Sights. A new approach for the design of the engines was implemented by SCD to optimize size and power consumption at system level. The new approach described in the paper, consists of: 1. A modular hardware design that allows the user to define the exact level of integration needed for his system 2. An “open architecture” based on the OMAP™3530 DSP that allows the integrator to take advantage of unused hardware (FPGA) and software (DSP) resources, for implementation of additional algorithms or functionality. The approach was successfully implemented on the first generation of 25µm pitch BIRD detectors, and more recently on the new, 640 x480, 17 µm pitch detector. Keywords: Uncooled engine , VOx technology, 17 µm pitch, TWS, DVE , SWaP

1. INTRODUCTION AND BACKGROUND Over the last decade, uncooled detectors have become increasingly used in a variety of commercial and military applications (1). Due to the large effort required for the design and manufacturing of a high quality uncooled camera core, system integrators are relying more and more on the camera core or “video engine” instead of a detector only, as the basic component of their system. Consequentially, most of the uncooled detectors manufacturers have been adapting to this market demand and concentrated efforts in developing video engines to drive their own detector and offer higher level of product integration. The characteristics of the video engine are tightly defined by the specific application: • Battery operated systems (such as thermal weapon sights and goggles) are seeking low SWaP (Size, Weight and Power) characteristics. • Surveillance systems and remote weapon stations are looking for the highest performance and are less concerned with size and power consumption. • Systems that need real time response will be ready to give up on functionality and intensive image processing in order to minimize the latency between real and processed image. A "general purpose" engine that will provide reasonable response to the specification requirements of most applications would be highly desirable to simplify system design and reduce costs by means of commonality. The requirement from such an engine is to be flexible enough to adapt its configuration and performance to the specific requirements. This can be achieved by: 1. 2. 3.

A modular hardware design that allows the customer to acquire the exact level of integration needed for his system. A single HW/SW that will drive different detector sizes (e.g. 384 x 288, 640 x 480) A DSP based engine with an open software configuration that will allow the system integrator to add or remove functionality and processing tools, according to the application needs.

2. SCD VIDEO ENGINE (VE) ARCHITECTURE 2.1. Video Engine Hardware Architecture The SCD VE has a modular design, allowing system designers the flexibility to acquire the right level of product integration for any specific system. 2.1.1. "Single card" configuration The basic unit, "single card" configuration is basically a detector integrated with a proximity board. The proximity board contains the following functions: • Low noise power supplies to the detector • Thermo-Electric Cooler (TEC) driver and diode temperature sensor feedback, based on a 24 bit analog to digital converter • Shutter driver and optional Shutter • 14 Bit video analog to digital conversion channel The “Single card” configuration, shown in Figure 1, is intended for those who prefer to implement the detector timing, calibration and image processing algorithms with their own electronics.

Weight < 60g Figure 1: "Single card" configuration

2.1.2. "Dual card" configuration. The “Dual card” configuration is a full blown engine that contains all the Hardware and Software required to display an analog or digital IR video. The “Dual card” engine has all the required IR video processing algorithms and is calibrated for operation. The “Dual card” option has the following features: • All “one card” features • Pre calibrated and installed IR image processing: o Non-Uniformity Correction (NUC) o Case Influence Correction (CIC) o Bad Pixel Replacement (BPR) o Dynamic Range Compression (DRC) o Automatic RNU Correction (ARC) • Standard video processing including user configurable symbology overlay • Analog (NTSC or PAL) and digital video output • Single 4V power supply.

A block diagram of the “Dual card” engine configuration is shown in Figure 2:

Figure 2: Block diagram of the "Dual card" configuration

The mechanical structure of the "Dual card" configuration is shown in Figure 3

Weight < 90g, including shutter Figure 3: Mechanical structure of the "Dual card" configuration

2.1.3. "Triple card" configuration The “Triple card” configuration adds an interface or panel board with standard connectors and physical layer interfaces to the "dual card" configuration, intended for product evaluation as well as for stand alone commercial (not military grade) camera applications such as CCTV. The “Three card” configuration has the following features: • All “Two card” features • Auto adjustable input power 5.0V-17V DC • Fast Ethernet PHY and RJ45 standard connector • BNC analog video connector • Mini-MDR26 standard “Camera-Link” connector • GPIO , RS232 / RS422 / RS485 , power connectors • HW provisions for USB PHY and Mini-USB standard connector and MicroSD removable card interface, that can be implemented on demand. An example of the “Three card” configuration application with 50mm F/1.7 optics is shown in Figure 4.

Figure 4: “Triple card” engine configuration with 50mm F/1.7 optics The block diagram of the “Triple card” engine configuration is shown in Figure 5 Vin 5-17V-

Card# 2

Card# 1

Figure 5: Block diagram of the “Triple card” engine configuration

2.2. SCD VE Software Architecture The entire IR video processing chain in the SCD VE is implemented using the Ti OMAP™ 3530 media processor. The OMAP3530 high-performance processor is designed to provide the best-in-class video, image, and graphics processing. The SCD VE application and control module is built over a Microprocessor unit (MPU) subsystem, based on the ARM Cortex™-A8 microprocessor which executes the Linux operating system. IR video processing is performed by dedicated hardware and an optimized code that runs on a C64x+ digital signal processor (DSP) core residing inside the same OMAP3530 silicon die(2). 2.2.1. Open software environment To allow system integrators to add their own software to the engine, all the application layers developed in the VE, and all the development tools that have been used, are based on Royalty free / open source components. The IR video processing algorithms, developed by SCD for optimal performance with the C64x+ digital signal processor, are written as standard Ti xDAIS Framework Components (eXpressDSP-compliant codecs), allowing seamless integration of additional own-developed or third party software components. Multimedia Framework– (Courtesy of Texas Instruments) (3) A major advantage of programmable DSPs over fixed-function devices is their ability to accelerate multiple multimedia functions in a single device. TI multimedia framework products are designed to enable users to share a DSP easily between algorithms, by handling issues like data buffer cache coherency, DMA resource contention, and efficient use of scratch memory. Developers can then leverage their application-specific expertise to create differentiated products. Codec Engine - Codec Engine is a codec execution framework that automates the invocation and instantiation of eXpressDSP-compliant codecs and algorithms. Codec Engine can execute in ARM-only, ARM-DSP, or DSP-only environments and supports concurrent execution of multiple channels and codecs. It does not provide A/V synchronization or manage application I/O, but rather accepts and delivers elementary audio and video streams. Codec Engine is designed to be used in conjunction with higher layer frameworks or middleware that provide A/V synchronization, video port and disk I/O, or network streaming while calling Codec Engine's eXpressDSP-compliant APIs for actual encoding, decoding, and processing. This approach enables system providers to differentiate their applications easily, while still leveraging Codec Engine. Codec Engine performs the following functions: • Automated dynamic • Transparent execution of codecs regardless of where the application resides with respect to the codecs. In some ARM-DSP applications, it may be desirable to execute audio codecs, for example, on the ARM and video codecs of the DSP. • Codec Engine provides APIs for acquiring system resource data, such as DSP CPU and memory usage by codecs. Developers can view this system resource data using the Eclipse-based Data Visualization Toolkit. • Codec Engine provides standardized configuration tooling for creating the codec combinations needed for a specific application. An illustration of the SCD video engine software stack is shown in Figure 6

Figure 6: An illustration of the SCD video engine software stack

3. VE PERFORMANCE The performance of the engine is summarized in the following table: Feature Nominal Power @ 25ºC Dual Card Video Interface Control Interface Shutter IR image processing

"Standard" Image processing

Operating Voltage NETD @300K Scene f/1, 60 Hz Auxiliary interface Operating Temperature Range Time to usable image

Performance < 2.4W , 640 x 480 @ 30 Hz < 2 W, 384 x 288 @ 30 Hz Digital - LVDS or Parallel ,with user programmable timing Analog - PAL (CCIR) or NTSC (RS170) RS232 Ethernet 10/100 Piezoelectric, Military qualified shutter •Case Influence Correction (CIC) •Non-Uniformity Correction (NUC) •Bad Pixels Replacement (BPR) •Dynamic Range Compression (DRC) •Automatic RNU Correction (ARC) •Sun-Burn Cancellation (SBC) •On screen Display •Black hot/White hot •Flip video: Left/Right, Up/Down •Manual / Auto configurable NUC •X2 , X4 Zoom 4 VDC (Dual Card) 5 - 17 VDC , Auto adjustable (Triple Card) ≤ 50° mK • 4 Digital I/Os , User Configurable Functions • RS232 / RS422 master -40C to +70C < 10 Sec

4. SUMMARY AND CONCLUSIONS In this paper we have presented a video engine developed by SCD for its 384 x 288, 25 µm and 640 x 480, 17µm uncooled detectors. In the first part we presented the modular hardware structure of the video engine, which allows system integrators to optimize the size, power, and cost of their system, as well as the overall development effort, including risk, time and cost. In the second part we presented the modular structure of the software, which allows engine users to add their own intellectual property or third party code, to run side by side with SCD's software, in the engine media processor. The engine comes "out of the box" pre calibrated, with all the software algorithms and hardware required to achieve a IR imagery. This allows the user to focus his efforts on system aspects and to build competitive, small foot-print, low power, and cost effective products.

ACKNOWLEDGMENTS The development of the detector and engine was supported by the Israeli Trade & Industry Ministry. We are in debt to the numerous engineers and technicians participating in the project, for their dedicated contribution to the development and production of the detector and the engine.

REFERENCES [1] A. Fraenkel et al. "SCD's Uncooled Detectors and Video Engines for a Wide Range of Applications” Presented at the SPIE Defense & Security Symposium, Orlando, April 2011. [2] "Broad Market OMAP™ 3, OMAP L1x and Cortex M3 Processors Roadmap" , technical presentation , Courtesy of Texas Instruments [3] "Multimedia Framework Products (MFP) - Codec Engine and xDAIS Framework" , Documentation of product , Courtesy of Texas Instruments