A Laser Obstacle Detection and Collision Avoidance ...

International Conference on Aeronautical and Astronautical Engineering (ICAAE)

A Laser Obstacle Detection and Collision Avoidance System for Helicopters and Unmanned Aerial Vehicle Applications Assoc. Prof. Roberto Sabatini (RMIT University, Australia)

Prof. Mark A. Richardson (Cranfield University, United Kingdom) Dr. Ermanno Roviaro (SELEX-ES, Italy) Melbourne, 16th December 2013

Scope of the Presentation • Introduction • Operational Requirements • Technical Description • Human-Machine Interface

• Ground and Flight Test • Future Activities

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

2

Introduction MOST HELICOPTER AND UA CRASHES ARE DUE TO IMPACT AGAINST OBSTACLES WHICH ARE SCARCELY VISIBLE EVEN IN DAYLIGHT AND IN GOOD WEATHER CONDITIONS

NEED TO AUGMENT HELICOPTER/UA LOW-LEVEL AND NAP-OF-THE-EARTH NAVIGATION CAPABILITY

OBSTACLE AVOIDANCE SYSTEMS

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

3

Operational Requirements

DETECTION

• ANY OBSTACLE ALONG FLIGHT PATH • EXPECIALLY WIRES

WARNING TIME

• 10 SECONDS

FLIGHT CONDITIONS

HMI

• STRAIGHT FLIGHT (UP TO 260 Km/h) • TURN (UP TO ABOUT 30 deg) • TERRAIN FLIGHT • DAY/NIGHT NVD

• TYPE OF OBSTACLE • LOCATION • AVOIDANCE ADVICE

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

4

Technical Alternatives REQUIREMENTS

MAGNETIC OWS

THERMAL OWS

Wire detection

only energized wires

only energized wires

short small

short as required

Detection range Coverage Area Obstacle position, distance and type determination accuracy Performance/weat her/ flight speed dependency False alarm rate Installability Base technology status

LASER RADAR OWS

all wires preferably perpendicular to flight trajectory as required as required

as required as required

insufficient

good for position and type , no distance provided

medium

very high

good

poor

good

good

high medium

low medium

very low medium

very low medium

mature

mature

state-of-art

state-of-art

LASER “RADAR” SOLUTION A/Prof. R. Sabatini

MILLIMETRIC RADAR OWS

all wires

LOAM

LASER OBSTACLE AVOIDANCE MARCONI

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

5

LOAM Technical Requirements

INSTALLATION PLATFORMS

• ANY TYPE OF HELICOPTER AND SMALL UAV • STAND ALONE / INTEGRATED MODES

OBSTACLES TYPE

• WIRES (5 mm) • PYLONS, POLES, TREES • BUILDING, WALLS

FLIGHT ENVELOPE

• STRAIGHT FLIGHT (UP TO 260 Km/h) • TURN (UP TO ABOUT 30 deg) • LANDING, TAKE OFF

EYE-SAFETY

• STANAG 3606 CLASS I

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

6

LOAM Functions • SCAN BY LASER BEAM THE AREA AROUND THE FLIGHT TRAJECTORY • OBSTACLES DETECTION AND CLASSIFICATION THROUGH ECHOES ANALYSIS

• DELIVERY OF WARNING AND INFORMATION TO THE CREW • COMMUNICATE WITH OTHER ON-BOARD EQUIPMENT

• SELF TEST

SENSOR HEAD UNIT

A/Prof. R. Sabatini

CONTROL PANEL UNIT

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

7

Performance PARAMETERS Vertical FOV Horizontal FOV LOS range Az LOS range El Instantaneous FOV Scan efficiency per one frame Scan efficiency per 0,5 sec. Max range Min range Range resolution Scanning time Detection probability per 1 sec. False Alarm Rate

A/Prof. R. Sabatini

PERFORMANCE > 30° > 40°  20°  20° < 2 mrad 1:15 1:20 2000 m < 50 m < 1.5 m 0.5 s > 99.5 % < 1 per 2 flight hours

SUBSYSTEM

WEIGHT (Kg)

Sensor Head Unit Control Panel Unit Display Unit (optional) Warning Unit (optional)

24 0.5 1.7 0.5

SUBSYSTEM Sensor Head Unit Control Panel Unit Display Unit (optional) Warning Unit (optional)

DIMENSION width x height x depth 320 x 239 x 419 mm 146 x 38.1 x 165 mm 4 ATI or 3x4 ATI 90 x 22 x 100

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

8

Performance Detection range for wires, d=10mm -15 -30

0

Detection range for wires, d=5mm

15

-15 30

-45

45

-60

75

-90

30

-45

60

-75

15

-30 45

-60

0

90 0 100 200 300 400 500 600 700 800 900 1000

60

-75

-90

75

90 0 100 200 300 400 500 600 700 800 900 1000

visibility 800 m visibility 1500 m

visibility 2000 m A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

9

System Architecture WINDOW

TX OPTICS SCANNER ASSEMBLY

LASER ASSY

TELESCOPE SUB-ASSY

Blanking signal

On/Off

GYROSCOPE ASSY

Motors control signals HMD PROCESSING & VDU VIDEO ASSY

PROCESSING ASSY

WMC discrete

VIDEO

RS 422

MIL-STD 1553

ARINC 429

WU C (optional)

VIDEO

CP (helicopter)

RS 422

WU P (optional)

VIDEO

VIDEO

WOW (TBD)

A/Prof. R. Sabatini

MECHANIC

EXTERNAL

SCANNER CONTROL & PREPROCESSING ASSY

POWER SUPPLY & POWER MOTORS ASSY

28 Vdc

OPTIC

DETECTOR SUB-ASSY

OPTICAL ASSY

ELECTRONIC

VDU

HMD/P

HMD/C

FLIR

RS232 Test Link

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

10

Laser Safety

LOAM EQUIPMENT MAKES USE OF ERBIUM FIBER 1.55 µm LASER

LOAM EQUIPMENT OPERATION IS ACCORDING TO STANAG 3606 CLASS I(1) (1) Class I lasers produce radiation that causes no biological damage

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

11

Hardware

Sensor Head Unit Display Unit (for test) Control Panel Unit

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

12

Hardware (2)

Telescope

Laser

Beam expander

Scanning control & Processing board

Power supply

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

13

Hardware (3)

Electrical connector

Scanning mirror

Entrance window

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

14

Scanner Description • BASED ON A SWASHING MIRROR CONCEPT • LIGHT WEIGHT • HIGH RELIABILITY • HIGH ACCURACY

LINE OF SIGHT ORIENTATION

SCAN PATTERN 400 H-FOV

300 V-FOV

• 40.000 LASER SPOTS • DRAWN BY SCANNED ELLIPSES • CAPABILITY TO DETECT OBSTACLES REGARDLESS THEIR ORIENTATION • OPTIMIZED FOR WIRE DETECTION • PRESERVE OBSTACLE SHAPE DURING HELICOPTER MOTION A/Prof. R. Sabatini

• AUTOMATIC FOV LOS RANGE ORIENTATION • ± 20O IN AZIMUTH (1° step) • ± 20O IN ELEVATION (20° step)

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

15

Scanner

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

16

Detection & Classification OBSTACLE DETECTION • ECHO DETECTION (AVALANCHE PHOTODIODE, VARIABLE GAIN AMPLIFIER)

• PRE PROCESSING (SINGLE ECHO POSITION/DISTANCE ANALYSIS)

• PROCESSING (PREPROCESSED ECHOES GLOBAL ANALYSIS)

OBSTACLE CLASSIFICATION • CLASS “WIRE” (HORIZONTAL THIN STRUCTURE “WIRE LIKE”)

• CLASS “TREE/POLE” (VERTICAL SINGLE STRUCTURE “TREE OR PILON LIKE”)

• CLASS “EXTENDED OBSTACLE” (BIDIMENSIONAL STRUCTURE “BUILDING OR HILL LIKE”)

OBSTACLE PRIORITYZATION • BASED ON OBSTACLE RANGE AND FLIGHT PATH • SELECTABLE PHYSICAL ENVELOPE • NEAREST OBSTACLE AND/OR OBSTACLE POSITIONED WITHIN THE FLIGHT PATH HIGHTEST PRIORITY A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

17

Detection & Classification

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

18

Warning Delivery Options VISUAL WARNINGS • OBSTACLES ARE DISPLAYED WITH THREE TYPES OF SYMBOLS (Classes)

SYNTHETIC PRESENTATION OF OBSTACLES

• OBSTACLE SYMBOLS ARE VISUALIZED ON A SCREEN REPRESENTING LOAM FOV • OBSTACLE POSITIONED ACCORDING TO LOAM REFERENCE FRAME • RELEVANT RANGE AND MAX. PRIORITY MARK DISPLAYED

CLASS WIRE SIMBOL

CLASS TREE SYMBOL

CLASS EXTENDED SYMBOL

AUDIO WARNINGS • ANALOG SIGNAL TO DIRECTLY DRIVE HEADPHONE SET

MODULATED SOUND

• DIGITAL SIGNAL TO TRIGGER ON BOARD SOUND GENERATOR

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

19

UAV Integration Architecture Scenario (Altitude)

Manned operations

DETECTION RANGE (NM)

Autonomous operations

FAA Requirements

Nominal Pilot (AC 90-48c)

Auto

LOS

BLOS

Low

2.5

0.5

1

1.5

Medium

3.5

1

2

2.5

High

4.0

1

2.5

3

40° 70° 70º

A/Prof. R. Sabatini

70°

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

20

Display Formats

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

21

Display Formats

Safety Line

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

22

Display Formats

WR / PL

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

23

Display Formats

ALL

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

24

Display Formats

3D

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

25

Display Formats

3D / FLIR

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

26

Display Formats

PPI

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

27

Impact Warning

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

28

Communications & Auto Test COMMUNICATION LINES (MIL-BUS-1553B, RS422 SERIAL LINK, ARINC 429) • OBSTACLE INFORMATION TRANSMISSION Obstacle Class, Priority, Position (Az , El & Distance according to LOAM reference system, useful for evasive manoeuvre) • VIDEO OUTPUT FLIR with obstacle symbols superimposed available • SYSTEM STATUS TRANSMISSION

• EXTERNAL COMMANDS RECEIVE

AUTO TEST FUNCTIONS • POWER ON BIT: AUTOMATICALLY ACTIVATED ON POWER ON CBIT • CONTINUOUS BIT: PERIODICALLY EXECUTED DURING NORMAL OPERATION IBIT • INTERRUPTIVE BIT: ACTIVATED ON PILOT’S REQUEST

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

29

Integration: Standalone Configuration

e.g., NH300

LASER EMISSION

POWER SUPPLY

RETURN ECHOES

SENSOR UNIT INHIBIT HEAD SET

CONTROL PANEL UNIT

A/Prof. R. Sabatini

DISPLAY UNIT

POWER SUPPLY

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

30

Integration: Low-Level Configuration e.g., AB212/AB412 and Small UAV

LASER EMISSION

POWER SUPPLY

RETURN ECHOES

SENSOR UNIT ARINC 429

INHIBIT HEAD SET

IMU PTHROUGH DL IN UAV

CONTROL PANEL UNIT

A/Prof. R. Sabatini

DISPLAY UNIT

POWER SUPPLY

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

31

Integration: High-Level Configuration e.g., EH101 and UAV

LASER EMISSION POWER SUPPLY RETURN ECHOES

EWS

INHIBIT

RS422 (PROVISION)

SENSOR UNIT

FLIR

ARINC 429

MFD

THROUGH DATALINK IN UAV

CONTROL PANEL UNIT

DISPLAY UNIT

CWG

MC IRS

MISSION BUS 1553B A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

32

Integration: High-Level Configuration e.g., NH90

LASER EMISSION

POWER SUPPLY

IRS1,2

RETURN ECHOES

ARINC 429

SENSOR UNIT

VIDEO

FLIR

WOW (TBD)

VIDEO (OPTIONAL)

EWS

INHIBIT

VIDEO (OPTIONAL)

HSM/D P

HSM/D C

RS422

WMC CONTROL PANEL UNIT

RS422

LOAM WU P OPT.

WU C OPT.

VIDEO

VDU

MFD

MISSION BUS 1553B A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

33

HMI2 Optimization

Platform Instantaneous Direction of Flight

LOAM FOV Centre

A/Prof. R. Sabatini

Platform axis

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

34

HMI2 Optimization

2-D Format

Platform Instantaneous Direction of Flight

Altimetric Format

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

35

Detection Performance Model and Test

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

36

Minimum Detection Performance Analysis

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

37

LOAM Ground Test

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

38

LOAM Ground Test Predicted and Measured Signal-to-Noise Ratios Clear Weather

Rain

V=10km

V=12.5km

V=15km

Light

Medium

Heavy

SNRP

4.90104

4.95104

5.02104

3.14104

1.83104

1.45104

SNRE

3.35104

3.80104

4.27104

2.87104

2.47104

2.13104

EP A LT Lr

SNR 

A/Prof. R. Sabatini

4 E p Ar LT Lr e 2R dW 

PD R 2 R  D NEP

= = = =  = dW =  = PD = R =  = D = NEP =

output laser pulse energy receiver aperture transmission losses (including beam shaping) reception losses (including optical filter) atmospheric extinction coefficient wire diameter wire reflectivity pulse duration obstacle range beam divergence initial beam diameter noise equivalent power

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

39

Experimental Flight Test Display Unit (for test)

NH-300

Sensor Head Unit Control Panel Unit A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

40

Experimental Flight Test

AB-212

MCU

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

41

Experimental Flight Test Future Activities AB412

SMALL UAV CONTROL UNIT

SENSOR UNIT

EH101

A129

UNMANNED HELICOPTERS

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

42

Conclusions • Laser Obstacle Detection & Collision Avoidance is a mature technology for low-to-mid dynamics platforms • The LOAM systems was successfully integrated on various helicopters and is been tested on UAVs • A number of low, medium and high-level avionics integration architectures have been developed • All HMI2 aspects were deeply investigated both in manned and unmanned configurations • Ground and flight test activities validated the obstacle

detection, classification and avoidance algorithms • Future activities will address additional manned and

unmanned aircraft applications with mid-to-high dynamics A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

43

References (1) 1.

R. Sabatini and M.A. Richardson. “Airborne Laser Systems Testing and Analysis.” NATO Research and Technology Organization (RTO) – Systems Concepts and Integration Panel (SCI). AGARDograph Series RTO-AG-160, Vol. 26 (Research Monograph Volume). 2010.

2.

R. Sabatini and M.A. Richardson. “New Techniques for Laser Beam Atmospheric Extinction Measurements from Manned and Unmanned Aerospace Vehicles.” Central European Journal of Engineering, Vol. 2, Print ISSN: 1896-1541, Online ISSN: 2081-9927, DOI: 10.2478/s13531-0120033-1. September 2012.

3.

R. Sabatini, M.A. Richardson and T. Jenkin. “A Laser Obstacle Avoidance System for Helicopter Nap-of-the-Earth Flying.” Journal of Defence Science (Classified). Vol. 10 – No 1 (pp. R41-R46). May 2005.

4.

R. Sabatini and M.A. Richardson. “A New Approach to Eye-Safety Analysis for Airborne Laser Systems Flight Test and Training Operations.” Paper published on the Journal of Optics and Laser Technology. Vol. 35 – Issue 3 (pp. 191-198). DOI: 10.1016/S0030-3992(02)00171-8. June 2003.

5.

R. Sabatini and M.A. Richardson. “System Integration and Flight Testing of a Laser Designation Pod and Laser Guided Bombs on the Italian Interdiction Strike Aircraft.” Journal of Battlefield Technology. Vol. 4 – No 2 (pp. 37-48). May 2001.

6.

H. Weichel “Laser Beam Propagation in the Atmosphere”. Second Printing. 1990.

7.

T. Elder and J. Strong, “The Infrared Transmission of Atmospheric Windows”. J. Franklin Institute 255 - 189. 1953.

8.

R. M. Langer, Signal Corps Report n° DA-36-039-SC-72351. May 1957.

A/Prof. R. Sabatini

SPIE Optical Engineering Press.

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

44

References (2) 9.

R. Sabatini, M.A. Richardson, M. Cantiello, M. Toscano, P. Fiorini and A. Gardi. “Experimental Flight Testing of Night Vision Imaging Systems in Military Fighter Aircraft.” Journal of Testing and Evaluation, Vol. 42, No.1, pp1-15. DOI: 10.1520/JTE20120339. July 2013.

10.

R. Sabatini, M.A. Richardson and E. Roviaro, "Development and Flight Test of an Avionics LIDAR for Helicopter and UAV Low-Level Flight.” Journal of Aeronautics and Aerospace Engineering, Vol. 2, No. 4. DOI: 10.4172/2168-9792.1000114. June 2013.

11.

R. Sabatini, M.A. Richardson, C. Bartel, A. Kaharkar, T. Shaid, L. Rodriguez and A. Gardi, “A Lowcost Vision Based Navigation System for Small Size Unmanned Aerial Vehicle Applications.” Journal of Aeronautics and Aerospace Engineering, Vol. 2, No. 3. DOI: 10.4172/21689792.1000110. May 2013.

12.

R. Sabatini, M.A. Richardson, M. Cantiello, M. Toscano, P. Fiorini and D. Zammit-Mangion. “Night Vision Imaging Systems Development, Integration and Verification in Military Fighter Aircraft.” Journal of Aeronautics and Aerospace Engineering. Vol. 2, No. 2. DOI: 10.4172/21689792.1000106. March 2013.

13.

R. Sabatini, C. Bartel, A. Kaharkar and T. Shaid. "Low-cost Vision Sensors and Integrated Systems for Unmanned Aerial Vehicle Navigation and Guidance." ARPN Journal of Systems and Software, ISSN: 2222-9833, Vol. 2, Issue 11, pp. 323-349. December 2012.

14.

R. Sabatini and M.A. Richardson. “Novel Atmospheric Extinction Measurement Techniques for Aerospace Laser System Applications.” Infrared Physics & Technology Journal. First published on-line 16 October 2012. Final journal print on Vol. 56, pp. 30-50, Ref. No: INFPHY-D-1200024R1, DOI: 10.1016/j.infrared.2012.10.002. January 2013.

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

45

References (3) 15.

F. X. Kneizys, E. P. Shuttle, L. W. Abreau, J. H. Chetwynd, G. P. Anderson, W. O. Gallery, J. E. A. Selby, and S. A. Clough, “Users Guide to LOWTRAN 7”. Air Force Geophysical Laboratory Report AFGL-TR-88-0177. Hansom AFB (MA). 1988.

16.

W. E. K Middleton., “Vision Through the Atmosphere”. University of Toronto Press. 1952.

17.

R. D. Hudson, “Infrared Systems Engineering”. Wiley & Sons. 1969.

18.

Strohbehn, J.W. et al., “Laser Beam Propagation in the Atmosphere“. Topics in Applied Physics Series – Vol. 25. Sprienger-Verlag. 1978.

19.

G. G. Keith, L. J. Otten, and W. C. Rose, “Aerodynamic Effects”. ERIM-SPIE IR&EO Systems Handbook (Vol. 2 – Chapter 3). Second Printing. 1996.

20.

A. J. La Rocca and R. E. Turner, “Atmospheric Transmittance and Radiance: Methods of Calculations”. Environmental Research Institute of Michigan Ann Arbor. 1975.

21.

C. B. Kellington, “An Optical Radar System for Obstacle Avoidance and Terrain Following.” AGARD CP-148. 1965.

22.

B. Goldstein and G. Dalrymple, “GaAs Injection Laser Radar.” Proceedings of the IEEE Vol. 55 N° 2. 1967.

23.

G. Hogg, K. Harrison, and S. Minisclou, “The Anglo-French Compact Laser Radar Demonstrator Programme”. AGARD CP-563. 1995.

24.

W. Büchtemann and M. Eibert, “Laser Based Obstacle Warning Sensors for Helicopters.” AGARD CP-563. 1995.

25.

S. L. Holder and R. G. Branigan, “Development and Flight Testing of an Obstacle Avaoidanceb System for the U.S. Army Helicopters.” AGARD CP-563. 1995.

A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

46

Questions & Discussion Thank you for your attention!

Contact Information: Assoc. Prof. Roberto Sabatini Royal Melbourne Institute of Technology (RMIT) University School of Aerospace, Mechanical & Manufacturing Engineering 115 Queensberry Street, Carlton, VIC 3053 (Australia)

E: [email protected], T: +61 3 9925 8015 ; +61 457 126 495 A/Prof. R. Sabatini

RMIT University – School of Aerospace, Mechanical & Manufacturing Engineering

47