Vehicle Architecture for Regular Periodic Fish-cage Net. Inspection. Vagelis Chalkiadakis1, Nikos Papandroulakis1, George Livanos1,2, Konstantia.
HCMR Institute of Marine Biology, Biotechnology and Aquaculture
Designing a Small-sized Autonomous Underwater Vehicle Architecture for Regular Periodic Fish-cage Net Inspection Vagelis Chalkiadakis1, Nikos Papandroulakis1, George Livanos1,2, Konstantia Moirogiorgou1,2, George Giakos3 and Michalis Zervakis2
1 Hellenic
Centre for Marine Research, Institute of Marine Biology, Biotechnology and Aquaculture (HCMR/IMBBC), Greece 2 School of Electronics and Computer Eng., Technical University of Crete (TUC), Greece 3 Department of Electrical & Computer Eng., Manhattan College, USA
Fish-Cage monitoring in aquaculture installations: Challenges Aquaculture constitutes one of the fastest growing sectors of global food production Fish escapes and/or increased death rates -> common reasons for fish-cage dysfunctionalities in aquaculture installations Great challenge to reduce the operational costs Need to guarantee complete coverage of fish-cages during an inspection for damaged net Establish the trend of automation in aquaculture installations
Fish-Cage monitoring in aquaculture installations: Motivation Unmanned Underwater Vehicles (UUVs) have proven to be a safe and cost efficient alternative to divers for net inspection applications Trend to move aquaculture installation sites further offshore -> achieve complete operational /energy autonomy and frequent status reporting Tending to replace most underwater operations within the offshore industry with UUV-based equipment Develop a system to automatically navigate within installation and perform fish-net inspection
State of the Art in
Unmanned Underwater Vehicles UUV technology has gained increased interest and wider applicability within aquaculture Underwater systems have additional communications and environmental condition requirements Technology has not Unmanned Underwater Vehicles been further tailored Remotely Operated Vehicles (ROVs) Manual operation utilizing related software and navigation tools Point search applications, deployment from a platform
for typical farming operations Autonomous Underwater Vehicles (AUVs) Preprogrammed to perform a task or sustain “intelligence” to readjust a specific functionality Capability to cover larger areas
Great potential for improvement towards automation, efficiency, robustness, cost effectiveness
Proposed Work for fish-cage inspection using ROV Objectives Incorporation of sensor and control systems for automatic damage detection in fish-cage nets Implementation of ROV-based tools and routines for automated navigation, motion control and monitoring via optical recognition approaches Development of an overall system for navigation of the underwater system via online video-processing procedure and net inspection through offline analysis of captured material
Proposed Work for fish-cage inspection using ROV Innovation Automate underwater operations in a challenging and highly dynamic environment Reduce risk of escapes through effective and safe inspection of fish-cage nets Reduce costs by combining multiple operations Contribute to improved regularity, portability and timeefficiency of the inspection system
Proposed Underwater System: Equipment Characteristics
ROV PLATFORM SPECIFICATIONS (BLUEROV2): Live 1080p HD video , Highly maneuverable configuration, Stable and optimized for inspection and research-class missions, Easy to use, cross-platform user interface Standard 100m depth rating 6 T200 Thrusters for a high thrust-to-weight ratio Quick-swappable batteries for all-day use
Proposed Underwater System: Experimental Procedure Attach reference targets to the net at known depths and bearings Navigate the AUV inside the fish-cage combining inertia and magnetic sensors with a real-time optical recognition and processing system applied to reference points Program AUV to perform a predetermined course in the cage, in order to record video of the total fish-cage net area target
Perform offline analysis to detect defective areas on the net
AUV Polygonal trajectory
Proposed Underwater System: Navigation Combine optical recognition/validation system with photogrammetry fundamentals Navigate the AUV to detect reference targets of known characteristics attached to the net Translate information from “image plane” to real world under the appropriate mathematical model
Proposed Underwater System: Navigation Recognize reference target based on shape and color intensity characteristics Read the video frame and extract color components in both RGB and HSV color spaces Perform Otsu thresholding and combine information from B, G and V channels in order to detect bright and dark regions of the image (black and white squared targets) Apply morphological opening using a square structuring element Determine square-like segments based on the ratio of the bounding box aspects, the ratio of the region area and its bounding box one and the ratio of the two axes Test under different lighting, environmental and fish-net conditions
Proposed Underwater System: Detection of Defective areas on the net Fish cage monitoring for net inspection assessments, including detection of possible problems (holes, ‘broken’ patterns, etc.) Implementation of appropriate image processing scheme using the video data of the net provided by the AUV Evaluation and computation of imaging parameters that can accurately describe the net condition– Detection of the position of the irregularity on the net - Alert properly
Proposed Underwater System: Detection of Defective areas on the net Proposed Methodology (Initial Approach) At first, two choices: either use the color characteristics of the underwater images or the net pattern Regularity analysis (periodicity) of patterned textures involves two issues: the spatial relationship between intensity values or the repeat distance of a pattern We performed Cross Correlation to robustly estimate correlation between the entire underwater image and the single net hole structure.
Proposed Underwater System: Detection of Defective areas on the net Proposed Methodology At every image position, all possible shifts of the template within a certain window are analyzed, in both positive and negative direction. The result provides a measure for how well the template image fits with each shift. The proposed method is a kind of template matching and it is able to identify defects with differential pixel intensity changes like in the case of broken net pattern structure.
Proposed Underwater System: Detection of Defective areas on the net Results
Fig. 1a: Net image example where the light gray sub-image is the template image
Fig. 1b: Cross-correlation factors where the x-y axis represent the image dimensions while the z axis represent the cross-correlation factor at each image pixel
Fig. 1c: Top view of the Fig.1b
Proposed Underwater System: Detection of Defective areas on the net Results
Fig. 2a: Net image example with artificial defective net areas
Fig. 2b: Cross-correlation factors
Fig. 2c: Top view of the Fig.2b
Proposed Underwater System: Detection of Defective areas on the net Results
Fig. 3a: Net image example with artificial defective net areas
Fig. 3b: Cross-correlation factors
Fig. 3c: Top view of the Fig.??b
Conclusions Development within autonomous and unmanned systems has taken large steps in aquaculture during the recent years The design of cost-effective, functional small-sized AUV for regular periodic fish-cage net inspection in terms of holes and fouling was proposed Proposed platform can be permanently reside in fish-cages and provide near-real-time information about the net integrity status Preliminary results reveal the potential of the proposed framework to provide appropriate corrective measures in order to eliminate fish escapes and minimise related maintenance in fish infrastructure
Current status The proposed UUV is initially programmed for net inspection mission in any underwater environment The current programming version is being tested via incorporation of the code on-chip, enabling on-AUV operation Testing and validation procedures take place in real conditions!!!
Future Work Additional capabilities (sensors or optical recognition of additional parameters) will be considered for the next versions of the device (e.g. biomass estimation, feeding efficiency etc.) More advanced object recognition approaches will be implemented in order to avoid impact of environmental parameters Physics of light in underwater environments will be considered for more accurate quantitative assessments The main issue of cross-correlation techniques in pattern regularity inspection is that they may be vulnerable to the underwater conditions that may cause net fold or net cell rotation in relation with the template image Examine new approaches based on statistical methods
Complete AUV Solution Overview
Dock-station for automated AUV recharge and unhindered operation AUV, consisting of the vehicle and its sensors, the inertia and optical navigation system, the data storage and communication Off-line processing and Communications Room, including storage, image processing and internet communications modules Energy Provision and management system
HCMR Institute of Marine Biology, Biotechnology and Aquaculture
THANK YOU
This work has received funding from the EU H2020 research and innovation programme under Grant Agreement No 678396
