Multi-spectrum-based enhanced synthetic vision ...

Multi-spectrum-based enhanced synthetic vision system for aircraft DVE operations Sudesh K. Kashyap*, VPS Naidu, Shanthakumar N. CSIR-National Aerospace Laboratories, PB 1779, Old Airport Road, Kodihalli, Bengaluru, Karnataka 560017, India ABSTRACT This paper focus on R&D being carried out at CSIR-NAL on Enhanced Synthetic Vision System (ESVS) for Indian regional transport aircraft to enhance all weather operational capabilities with safety and pilot Situation Awareness (SA) improvements. Flight simulator has been developed to study ESVS related technologies and to develop ESVS operational concepts for all weather approach and landing and to provide quantitative and qualitative information that could be used to develop criteria for all-weather approach and landing at regional airports in India. Enhanced Vision System (EVS) hardware prototype with long wave Infrared sensor and low light CMOS camera is used to carry out few field trials on ground vehicle at airport runway at different visibility conditions. Data acquisition and playback system has been developed to capture EVS sensor data (image) in time synch with test vehicle inertial navigation data during EVS field experiments and to playback the experimental data on ESVS flight simulator for ESVS research and concept studies. Efforts are on to conduct EVS flight experiments on CSIR-NAL research aircraft HANSA in Degraded Visual Environment (DVE). Keywords: Synthetic Vision System, Enhanced Vision System, Flight Simulator studies, Enhanced vision system field trails.

1. INTRODUCTION The objective of Enhanced Synthetic Vision System (ESVS) is to eliminate or reduce aircraft accidents occur in Degraded Visual Environment (DVE) as well as enhance operational capabilities of aircraft with safety and pilot Situation Awareness (SA) improvements under DVE operations. The ESVS concept is to provide the pilot an unobstructed out of the cockpit window view through the display of computer-generated imagery derived from onboard database of terrain, obstacle, and airport information and enhanced with weather penetrating imaging sensors. Thus ESVS is a combination of “Sensor Vision” and “Synthetic Vision”. The former presents the fused image from multispectral imaging sensors like InfraRed (IR) sensors of different spectral bands and millimeter wave radar and is called Enhanced Vision System (EVS). The latter generates a rendered image of the external scene from the perspective of the flight deck derived from aircraft attitude and high precision navigation solution using database of terrain, obstacles and relevant cultural features and is called Synthetic Vision System (SVS). The perspective images from the vision sensors and the synthetic images will be fused and presented to the pilot on a Head-Up-Display or Head-Down-Display (HUD/HDD). NASA and Federal Aviation Administration (FAA) are jointly conducting collaborative research to ensure effective technology development and implementation of regulatory and design guidance to support introduction and use of EVS/SVS advanced cockpit vision technologies in NextGen operations1. FAA has given clearance for Enhanced Flight Vision System (EVS display with certified flight symbology) to be used on small aircrafts and business jets. There are number of commercially available EVS systems, MaxViz, BAE systems, CMC Electronics, Honeywell to name a few, which have been flying on business jets. Likewise, SVS systems are also integrated to avionics displays by Honeywell. Research Teams at DLR and CAE have demonstrated full functionality of ESVS on a research aircraft and helicopter for brown out conditions respectively. FAA also has issued several advisory circulars which list the various requirements from EVS and SVS systems. Guidelines for certifying the systems for civil aircraft operations are being evolved. Gulfstream, Bombardier, Dassault, CIRRUS, Boeing, Pilatus PC-12 NG, JAPAT's Falcon 900DX etc are using EVS/SVS technologies certified from FAA/European Aviation Safety Agency (EASA). NASA, Gulfstream Aerospace and Honeywell are working on Synthetic and Enhanced Vision System (SEVS) for additional margins of safety and aircrew performance in low visibility surface, arrival, and departure operations. Bombardier Global Vision Flight Deck is *[email protected]; phone (+91-80) 25086556/58; fax (+91-80) 25298293; www.nal.res.in Multispectral, Hyperspectral, and Ultraspectral Remote Sensing Technology, Techniques and Applications VI, edited by Allen M. Larar, Prakash Chauhan, Makoto Suzuki, Jianyu Wang, Proc. of SPIE Vol. 9880, 988017 © 2016 SPIE · CCC code: 0277-786X/16/$18 · doi: 10.1117/12.2223667 Proc. of SPIE Vol. 9880 988017-1 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

the first to develop and certify the new Rockwell Collins Pro Line Fusion with both Enhanced Flight Vision System (EFVS) and SVS on HUD for corporate aviation2-8. Development of ESVS for Indian regional transport aircraft is one of the major activities initiated by National Aerospace Laboratories of Council of Scientific and Industrial Research (CSIR-NAL) recently. ESVS is expected to provide aircraft the capability of operation from all regional airports within India with minimal infrastructure and instrumentation facility under DVE (including night, rain, fog, smoke and other low visibility conditions). The GPS Aided Geo Augmented Navigation (GAGAN) system is expected to provide autonomous CATI landing capabilities at all airports within India. With ESVS it is expected to achieve CAT II and possibly CAT IIIa approach and landing without any additional infrastructure/facilities at most of the Indian airports. As part of the initial work, technology analysis, requirement specification was carried out and a roadmap for the technology development has been evolved. This paper presents the flight simulator that has been developed for carrying out ESVS research at CSIR-NAL and the field experiments carried out with EVS hardware prototype.

2. ESVS FLIGHT SIMULATOR ESVS is primarily a vision based technology. It requires extensive evaluation of ESVS and related technologies in flight simulator and to develop operational concepts for all weather approach and landing and to provide quantitative and qualitative information that could be used to develop criteria for all-weather approach and landing at all regional airports in India. Human factor studies play a vital role in this evaluation of ESVS on flight simulator. There are considerable research challenges that need to be addressed to realize complete benefits of ESVS. Some of the challenges that are planned to address with simulation studies are:  To determine the best way to present ESVS display with symbology to increase field of view and to enhance pilot situational awareness.  To determine the requirements of SVS database and EVS sensors to ensure the integrity of ESVS.  To determine pilot usability/acceptability and situational (terrain) awareness provided by ESVS display concepts.  To evaluate effects of: Sensor latency, Scene content, Design eye point, Fused display insets, Navigation system problems, etc.  Assess the potential effects of component/full system configurations, limits of technology, environmental conditions  To evaluate the effect of advanced pathway guidance (tunnel in the sky) on ESVS head-up-display (HUD) during approach and landing operations.  Following human factor studies:  Cognitive workload on pilot with/without ESVS during approach and landing in DVE conditions.  Influence of SVS and EVS part of ESVS on pilot performance at DVE conditions.  Pilots performance with different ESVS display locations (HUD/HDD)  Influence of pathway guidance information on ESVS display including display clutter on pilot performance  SV database integrity errors on pilot performance.  Sensor latency effect on pilot performance  To develop criteria for all-weather approach and landing at regional airports (with minimal infrastructure) in India. 2.1 ESVS Flight Simulator To carry out the above research studies CSIR-NAL desktop flight simulator ‘NALSim’[9] has been augmented with SVS and EVS simulation hardware and software components. Figure 1 shows the ESVS flight simulator that has been developed and commissioned in CSIR-NAL for ESVS research. The ESVS flight simulator has three hardware compartment namely (i) ‘NALSim’ computing hardware for basic flight model simulation and out-of-the-window (OTW) visual generation, (ii) ESVS computing hardware for simulating EVS and SVS and (iii) Cockpit displays and controls (Pilot station). OTW visuals having total FOV of 150 degrees horizontal and 40 degrees vertical are generated with three window display system as shown in Figure 1. OTW visuals are generated using Open Scene Graph (OSG) library.

Proc. of SPIE Vol. 9880 988017-2 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

EVS sensor simulation is being carried out using Vega-Prime IR sensor simulation software module[10] and Vega-Prime atmospheric module to simulate different spectral IR scene at different atmospheric and visibility conditions. The simulated IR scene is displayed on HUD along with ESVS specific flight symbology on central OTW. Equivalent Synthetic Vision (SV) with relatively larger FOV is simulated using 3D terrain database and 3D models in OSG format and displayed on primary display or HDD as shown in Figure 1.

‘NALSi m’ Basic Flight Simulator Computing Hardware

EVS/SVS Simulation Computing Hardware OTW visuals with EVS HUD and SVS HDD

OTW Visuals

Pilot Station

Figure 1: ESVS Flight Simulator

2.2 Design and Implementation of HUD Symbology HUD is one of the important sub-systems of aircraft avionics suite for enhancing pilot situational awareness for smooth and safe gate-to-gate operations especially during take-off, approach and landing phases of flight. The HUD symbology can be combined or overlaid with EVS/SVS displays to increase pilot situational awareness in DVE operations. Gulfstream is the first aircraft manufacturer to successfully deployed and obtained certification for the Enhanced Flight Vision System (EVS with flight symbology) on its business jet class aircrafts. Gulfstream G450 HUD standard[11] has been used as benchmark reference for the development of EVS specific advance HUD symbology and integrated with ESVS flight simulator. Figure 2(a) shows the snapshot of Gulfstream G450 HUD symbology and Figure 2(b) shows advanced HUD developed and integrated onto ESVS flight simulator.

.nv iss

Altitude Dial

Bank Angle

tat

Ip,

Heading Tape

ár

® .t

U

Runway Airspeed Dial

9

Pitch Ladder 9.

(a) Gulfstream G450 [11],

Local Horizon FPV HSI

(b) HUD symbology page on ESVS flight simulator

Figure 2: HUD symbology developed for ESVS flight simulator 5155 307

The advanced sets of HUD symbology are mainly categorized as conformal symbology and non-conformal symbology. Conformality is characterized by those items in a symbology set that overlay and move in unison with similar far domain counterparts in the environment hence adhering to Gestalt grouping principles of proximity, common fate and good

Proc. of SPIE Vol. 9880 988017-3 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

continuation. For example runway outline overlapping an actual runway is considered conformal. Likewise a HighwayIn-The-Sky (HITS) represents a virtually conformal depiction of the pilot’s flight path. The non-conformal symbology has three major parts, airspeed, altitude and heading indicators. The airspeed part is displayed in left hand and upper side in a form of airspeed dial (Ticks with inset T shape box as shown in Figure 4), secondary airspeed, selected airspeed and airspeed bug. Similarly altitude part is displayed in right hand and upper side in a form of altitude dial (Ticks with inset T shape box), selected altitude, altitude bug and vertical speed. The heading part is represented as i) heading tape on middle and upper side along with heading bug and ii) Horizontal Situation Indicator (HSI) on middle and lower side consisting of heading indicator in dial form and bar indicator for aircraft heading and lateral deviation from designated runway for landing. The other non-conformal symbologies are bank angle, altitude awareness bug, Pitch Limit Indicator (PLI), Course Deviation Indicator (CDI), Instantaneous acceleration caret, Speed error tape, Reference flight path angle, Landing gear etc. Based on G450 HUD symbology norms, some of the symbology like HSI, ticks of airspeed and altitude dials goes ‘OFF’ (de-clutter) when EVS display on HUD is ‘ON’ to provide better view of external scene to the pilot. The conformal symbologies include local horizon, pitch ladder, Flight Path Vector (FPV), airport/runway, HITS etc. The methodology used to make the symbology conformal to real-world is as follows. The central display monitor in pilot station of ESVS flight simulator (see Figure 1) is used to overlay HUD symbology on top of OTW visuals. Based on position of pilot view point the field of view of central OTW visuals is found to be 50 deg (horizontal) and 40 deg (vertical). Since the HUD symbology development is in pixel coordinates, the real world coordinates need to be mapped to pixel coordinates. In present case central display monitor used to overlay HUD symbology is mapped to max value of ± 10 (normalized value) in x and y coordinates with center of monitor in line with pilot eye point. Therefore a factor of 0.5 needs to be multiplied with any angle computed horizontally or vertically to map the real world coordinates to pixel coordinates and hence to make symbology conformal. The following sub-section presents the design and development of conformal symbologies for typical airport/runway and HITS for straight and curved approaches. 2.3 Design of conformal Airport/Runway For approach and landing at Bangalore HAL airport (VOBG) the width of airport and runway is assumed to be 800 and 180 feet respectively and the length of airport and runway is assumed to be 9000 feet for designing conformal airport/runway. The airport symbol starts appearing during the final approach and landing phase of flight when aircraft altitude Above Ground Level (AGL) is less than 2000 feet and remain visible up to 350 feet AGL [11]. The runway starts appearing during this phase of flight when aircraft reaches 400 feet AGL and remain visible till final touchdown. It can be noted that both the airport and runway symbols appear together between 400 to 350 feet during this phase of flight. Figure 3 shows the graphical representation of airport to explain the theory behind creation of runway symbology. Z axis

A

ATL W

E

F

B

C L

Y axis

D

AD

X axis

Figure 3: Real World Graphical Representation for Runway Symbology Creation

‘L’ and ‘W’ represents the length and width of runway respectively, ‘AD’ represents aircraft ground distance from runway threshold , ‘ATL’ represents aircraft altitude above runway threshold, point ‘A’ shows aircraft current position and runway coordinates are represented by four points ‘C’, ‘D’, ‘E’ and ‘F’ ( for e.g. : ‘C’ defined by ( Cx, Cy) in X-Y plane).

Proc. of SPIE Vol. 9880 988017-4 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

In this formulation, it is assumed that aircraft in level flight and is aligned with runway centre line. The near end points ‘C’ and ‘D’ in pixel coordinate are computed as follows: Cx=0.5*tan-1(-W/2, AD), Cy=0.5*tan-1(ATL, (AD*AD + W/2*W/2)1/2) (1) Dx=0.5*tan-1(W/2, AD), Dy=0.5*tan-1(ATL, (AD*AD + W/2*W/2)1/2) (2) The far end points ‘F’ and ‘E’ in pixel coordinate are computed as follows: Fx=0.5*tan-1(-W/2, AD+L), Fy=0.5*tan-1(ATL, ((AD+L)*(AD +L) + W/2*W/2)1/2) Ex=0.5*tan-1(W/2, AD+L), Ey=0.5*tan-1(ATL, ((AD+L)*(AD +L) + W/2*W/2)1/2)

(3) (4)

In case aircraft is not aligned to runway centre line and is maneuvering, then the equations (1)-(4) can be extended to account for aircraft heading, pitch and roll angles and lateral deviation from the runway center line. The same theory can also be used to create conformal symbology to represent airport. Figure 4 (a) and (b) show the conformal airport and runway symbologies during curved approach and landing phase of flight. It can be seen from figure 4(a) that both airport and runway symbologies are present when aircraft altitude is between 400 to 350 feet AGL. Below 350 feet only runway symbology is present as shown in fig. 4(b).

(a)

(b)

Figure 4: Conformal Airport and Runway during Approach and Landing

2.4 Highway-in-the-sky Highway-in-the-sky (HITS) for straight and curved approach for landing are designed and developed. The HITS symbology shape can be of different types such as U shaped, Pathway and BOX shaped etc. but in current work HITS symbology is designed to be of square shape with its dimensions same as width of runway (180 feet). The concept used for designing airport/runway symbology is also applied for the creation of HITS. As compared to airport/runway which is part of terrain, HITS is more like a 3D appearance with origin in the sky which gradually (with slope angle typically same as 3 deg glide slope used for smooth landing) vanishes at the touch down point of the runway. Figure 5 shows the graphical representation of HITS to explain the theory behind creation of symbology. A

Z axis

A/C

B W D C

AGL

W W

Y axis E

X axis

PS P0

F L

AGL n

P1

P2

…….

Pn

AD

Figure 5: Real World Graphical Representation for Straight HITS Symbology Creation

Proc. of SPIE Vol. 9880 988017-5 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Points ‘P0’, ‘P1’,…, ‘Pn’ are the tunnel post points, ‘PS’ is distance between two consecutive post points, ‘AGL’ is the aircraft altitude above ground level, ‘AD’ is the aircraft ground distance from runway threshold and ‘AGLn’ is the altitude of nth tunnel points above ground level computed based on tunnel slope and distance of tunnel point from runway threshold. In this formulation, it is assumed that aircraft is in level flight and is aligned with runway centre line. Since square tunnel is considered in present case, only lower points ‘C’ and ‘D’ are calculated and y coordinate of points ‘B’ and ‘A’ are obtained by adding ‘W’ to y coordinates of respective lower points. The ith points ‘C’ and ‘D’ are calculated as follows: Cx = 0.5*tan-1 (-W/2, AD-i*PS) (5) Cy = 0.5*tan-1(AGL-0.5*AGLi, ((AD-i*PS)*(AD-i*PS) + (W/2*W/2))1/2) (6) Dx = 0.5*tan-1 (W/2, AD-i*PS) (7) Dy = 0.5*tan-1(AGL-0.5*AGLi, ((AD-i*PS)*(AD-i*PS) + (W/2*W/2))1/2) (8) If aircraft is not aligned to runway centre line and is maneuvering then the equations (5)-(8) can be extended to account for aircraft heading, pitch and roll angles and lateral deviation from the runway center line. Also the curved HITS can be drawn by including the radius of curve (based upon the maximum turn rate given aircraft) into the formulation. Figure 6 shows different HITS types that are developed and implemented (a) U shaped tunnel, (b) Rectangle shaped tunnel, (c) Pathway – rectangle shaped tunnels with bottom portion connected to show the path and (d) Box shapedrectangle shaped tunnels with both top and bottom portions connected to show the path. The selection of HITS tunnel types can be part of human factor study with qualitative analysis and feedback from experienced pilots flown in ESVS simulator for different approach (straight/curved) and landing.

(a)

(c)

(b)

(d)

Figure 6: Different HITS tunnel types: a) U Shaped, b) Rectangle, c) Pathway, d) Box Shaped

Proc. of SPIE Vol. 9880 988017-6 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

3. EXPERIMENTS WITH EVS PROTOTYPE As part of ESVS development plan, EVS hardware prototype consisting of Long Wave Infrared (LWIR) sensor and low light CMOS camera and data acquisition system has been developed to carry out EVS field experimental trails initially on ground vehicle and later on research aircraft, playback EVS experimental data on flight simulator to understand issues related to rendering synthetic image from the terrain database using recorded navigation data and study the registration/ overlay/fusion of the EVS sensor image data with synthetic vision and flight symbology. 3.1 EVS Sensor The EVS sensor used in the current experiments is a general aviation dual wavelength enhanced vision sensor supplied by ASTRONICS Max-Viz, USA (Max-Viz EVS 600)[12]. It has following features:    

Sensors of dual wavelength (8-14 microns LWIR sensor with 320 x 240 resolution and CMOS low visible light sensor electronically fused to LWIR image) FOV: 40 degrees Horizontal X 30 degrees Vertical Output: RS-170 Video/NTSC Operational Limits: Altitude: 25000’ Airspeed: 250 kts (Indicated)

3.2 Navigation System To generate the equivalent synthetic vision, navigation data of the test vehicle is required to be recorded during EVS field experiment. Off the shelf Micro Aerial Vehicle (MAV) autopilot board ARDU PILOT MEGA (APM-2.5)[13] is used as Attitude Heading Reference System (AHRS) platform to generate the test vehicle position and attitude parameters. APM-2.5 autopilot board has sophisticated inertial measurement unit (IMU) known as InvenSense MPU 6000. The MPU-6000 device combines 3-axis gyroscope, 3-axis accelerometer and 3-axis magnetometer with an onboard digital motion processor capable of processing complex 9-axis motion fusion algorithms. Proprietary ARDU Direction Cosine Matrix (DCM) estimator algorithm is used to process the IMU data to estimate the angular parameters like roll and pitch and magnetometer heading. External COTS based GPS receiver ‘UBLOX LEA-6H’ is integrated to APM 2.5 autopilot board to generate required angular and position parameters of the test vehicle during the experiments. 3.3 EVS Data Acquisition System Data Acquisition System for EVS field experiment (EVS-DAS) consists of laptop computer (MIL-STD-810G qualified Panasonic tough book), frame grabber unit, power battery, and switch and circuit breaker which firmly enclosed in an aluminum bracket is shown in Figure 7. The EVS-DAS is designed to power the EVS sensor unit and acquire the data from EVS sensor, EO camera, and AHRS unit (navigation data) during the experiments. Data acquisition system software hosted on laptop is designed to acquire EVS (IR and EO) video data through frame grabber and navigation data from AHRS unit, time synchronize the data, store the data on laptop hard disk and then subsequently playback the data on ESVS flight simulator for analysis and research. The laptop is used in tablet mode and data acquisition software is designed with touch screen based graphical user interface for ease of operation. The whole EVS-DAS unit is subjected to structural tests to make it qualify for the flight experiments.

Proc. of SPIE Vol. 9880 988017-7 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Frame Grabber

IPT#1,1 Power Switch

Power Battery

Laptop (tablet) inside aluminum bracket

Figure 7: EVS Data Acquisition System (EVS-DAS)

3.4 EVS Ground Vehicle Experiments Before carrying out expensive flight experiments, several ground vehicle experiments are conducted at Bangalore HAL airport runway/taxiway. The main objectives of this initial ground vehicle trail runs are: (i) To capture runway/taxiway features with EVS sensors and compare with features captured with normal electrooptic (EO) camera at different visibility conditions. (ii) To generate EVS data in time synch with navigation data (iii) Playback the recorded EVS and navigation data on ESVS simulator to study issues related to:  Fusion of multi spectral / multi FOV sensors images  Rendering synthetic view from terrain database using recorded navigation data  Register/Fuse/overlay the EVS image data on rendered synthetic view The experimental setup for the EVS ground vehicle experiment is shown in Figure 8. Aluminum bracket is designed and fabricated to hold the EVS sensor unit, EO camera unit, AHRS unit and GPS unit and mounted on top of the test vehicle as shown in Figure 8. The EVS-sensor unit is powered by 24V DC from Lithium Ion battery. EO camera (GoPro Hero4 Black) has its own power battery. AHRS unit (APM 2.5 board) is powered directly by laptop through USB port. All the individual systems/devices used in this experiment are connected to power supply and laptop as shown in the connection diagram in Figure 9. After initial laboratory tests the equipments are mounted on the ground test vehicle to carry out tests at HAL airport runway. The EVS-DAS with frame grabber, power supply unit and laptop shown in Figure 9 is carried inside the test vehicle for recording the data in real time during experiment. EVS ground vehicle experiments were conducted on runway/taxiway at HAL, Bangalore just before sunset and after sunset and just before sunrise and after sunrise. During the experiment, test vehicle with equipment was driven at approximately 20 KMPH speed on the runway from one end to the other end and return on taxiway in the same speed. Figure 10 show the trajectories of the experiment vehicle (derived from recorded GPS data) during the experiments mapped onto Google map of HAL runway area. During the experimental run, image/video output was captured from both the EVS sensor (IR) and EO camera and recorded on the laptop in time synch with vehicle position and attitude/heading data captured from AHRS and GPS units. The recorded IR and EO video data is then played back on flight simulator with vehicle position and attitude data used to render corresponding synthetic vision using terrain database of the HAL airport area. Initially EO and EVS images/videos are registered using affine transformation and EVS image overlaid on EO image to study their relative field of view and to see the effectiveness of IR sensor in low visibility in recognizing the runway markings. Later EVS images/video and corresponding rendered synthetic view were registered and displayed on a simulator display to see the accuracy at which synthetic view were able to generate with the recorded navigation data using available terrain database/satellite images. Figure 11 show the registered and overlaid EVS images over the corresponding EO images and Figures 12 show the EVS image along with synthetic image rendered using vehicle position and attitude/ heading data. Also few experiments were conducted in fog condition at national highway near Bangalore where mild to moderate fog formation was present in winter month. Figure 13 shows the visual improvements available through EVS sensors when compare to normal vision captured through EO camera.

Proc. of SPIE Vol. 9880 988017-8 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

GPS Antenna

AHRS EO Camera EVS Sensor

Figure 8: Experimental setup for EVS ground vehicle experiments

GPS Antenna

EC) Camera

Video Cable (RS170) I2C

EN'S Sensor

Video Cable (RS170)

Laptop

1/ 21V

EN'S-DAS Frame Grabber USB

able

Power Cable Switch & Fuse

Lithium Ion Battery

Figure 9: Experimental setup for EVS ground vehicle experiments

Proc. of SPIE Vol. 9880 988017-9 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

r J

WPOng MI&V arz, Yí&

IT

JR VIE oLcfÑV

.

.!Q

..1

.E2600020rpel

,a;

Row-7-7-7 Martr MHZ

Alli .11.

tr

rf

.

....

HAL Heritge Museum. --tii...4P,/ sr.t.4,- '=, Pr - . - '

tp

_ -4I

HAL Banoalorp

HAL.

:)010141Z&Tunnel.Rd

111

,

AMBEDKAR

Figure 10: Trace of test vehicle during EVS ground vehicle experiments at runway

EO

EO

IR

CAM01

CAtv10 1

EO

EO

IR

CAM01

CAM01 Figure 11: Registered EVS (IR) images with EO images

Proc. of SPIE Vol. 9880 988017-10 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

CSIR-NAL

Figure 12: Comparison rendered synthetic images with real EVS images

Vision through EVS sensor

Vision through regular EO camera

Figure 13: Comparison of EVS sensor images with EO images in fog condition

Proc. of SPIE Vol. 9880 988017-11 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

3.5 EVS Flight Experiments To support the ESVS concept studies on a flight simulator, few experiments of EVS prototype is being planned on CSIRNAL’s HANSA research aircraft. HANSA is indigenously developed all composite twin seater propeller driven aircraft certified by Directorate General of Civil Aviation (DGCA), Government of India under FAR23 via JAR-VLA. The objective of EVS flight experiments on HANSA is to collect the EVS image data in time synch with EO image data and aircraft navigation data during taxi trails and during runway approach/landing at different visibility conditions and then playback the collected data on ESVS flight simulator and carryout ESVS studies. Preparations are underway for the EVS flight experiments, the installations of EVS sensor and EVS-DAS have been completed and waiting for the procedural clearance from the DGCA for the flight trails. Figure 14 shows the installations of EVS sensor and other equipments on the HANSA aircraft for the experiments. EVS sensor is installed under the starboard wing to avoid aircraft propeller in its field of view. The data acquisition system EVS-DAS is firmly fixed inside the cockpit on co-pilot seat in a way to make it easy for pilot to operate it through GUI touch screen during the flight experiments. AHRS is fixed inside the cockpit near to the aircraft CG. EO camera and GPS antenna are fixed under glass canopy in front of the co-pilot seat. Planned flight experiments on HANSA consists of 1 or 2 taxi trails and 5 to 6 approach/ landing experiments at different visibility conditions including night and in moderate fog (with CAT I visibility conditions, that is runway visibility range of 1800 ft). Efforts are on to get the DGCA clearance at the earliest, complete the flight experiments and present the results.

EO Camera & GPS Antenna

EVS Sensor

EVS-DAS on co-pilot seat

AHRS

_ GPS Antenna

EO Camera

Figure 14: HANSA aircraft with EVS experimental instruments.

Proc. of SPIE Vol. 9880 988017-12 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

4. CONCLUDING REMARKS Research and development on ESVS that is being carried at CSIR-NAL, India has been presented. The main objective of this R&D effort is to study and develop ESVS related technologies and to develop ESVS operational concepts for Indian regional transport aircraft. ESVS flight simulator has been developed to study and develop ESVS operational concepts for all weather approach and landing and to provide quantitative and qualitative information that could be used to develop criteria for all-weather approach and landing at different regional airports in India. Enhanced vision system (EVS) hardware prototype consisting of long-wave Infrared sensor and low light CMOS camera and data acquisition system has been developed and carried out few field trials on ground vehicle at airport runway at different visibility conditions. The EVS experimental data has been played back on ESVS simulator to render equivalent synthetic vision and to study registration and fusion of EVS/SVS images and display concepts. Efforts are on to conduct EVS flight experiments on CSIR-NAL research aircraft HANSA in degraded visual environment.

5. REFERENCES [1] Kramer, Lynda J.; Bailey, Randall E.; Ellis, Kyle K.E.;Norman, R. Michael; Williams, Steven P.; Arthur III, Jarvis J.; Shelton, Kevin J. &Prinzel III, Lawrence J. “Enhanced and synthetic vision for terminal maneuvering area next gen operations”. In Proceeding of SPIE, Vol. 8042, Paper No. 8042 0T (2011) [2] [3] [4] [5] “Honeywell Primus Apex® SmartView ™ SVS for Pilatus PC-12 NG” [6] “ Honeywell Presents: True Stories -- JAPAT's Falcon 900DX” [7] Kevin J. Shelton, Lynda J. Kramer, Kyle Ellis, and Dr. Sherri A. Rehfeld, “Synthetic And Enhanced Vision Systems For Nextgen (SEVS) Simulation And Flight Test Performance Evaluation”, 31st Digital Avionics Systems Conference, October 14-18 (2012) [8] [9] Kamali C, Archana Hebbar, T. Vijeesh, and S. Moulidharan, “Real-time Desktop Flying Qualities Evaluation Simulator”, Defence Science Journal, Vol. 64, No. 1, 27-32 (2014) [10] [11] [12] [13]

Proc. of SPIE Vol. 9880 988017-13 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/03/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx