Utilizing Panoramic Views for Visually Guided Tasks in Underwater

MTS/IEEE Oceans 2001, Honolulu, USA

Utilizing Panoramic Views for Visually Guided Tasks in Underwater Robotics Applications∗ S. Negahdaripour, H. Zhang, P. Firoozfam, & J. Oles∗ Underwater Vision & Imaging Laboratory Electrical & Computer Engineering Department, Univ. of Miami Coral Gables, FL 33124-0640 Abstract– Over the last decade, there has been an increasing interest in developing vision systems and technologies that support the operation of unmanned submersible platforms. Selected examples include the detection of obstacles and tracking of moving targets, station keeping and positioning, pipeline following, navigation, and mapping. These developments rely on images from standard CCD cameras. In this paper, we propose the use of these images for the construction of panoramic views that are then utilized for a number of applications, capabilities, and operational modes of underwater vehicles; scenarios that are also common in airborne and space robotics applications. We describe two prototype imaging systems in forward-look and down-look configurations, designed and built from 6 off-the-shelf security cameras, also addressing a number of issues for preprocessing the data to construct these views. We study each of a number of critical capabilities and describe the potential use of the panoramic imaging system, presenting various examples to demonstrate the applications.

can be obtained from the collection of images taken either from the same point as the camera pans, from very nearby viewpoints (relative to the scene distance) using several cameras that simulate a panning camera [24], and special lenses and mirrors [15, 21]. With high-speed image acquisition systems, panoramas can be constructed and utilized as individual frame of a video sequence for real-time vision processing. In addition to video teleconferencing, mapping and robot navigation are natural domain-independent applications for the utilization of panoramic views (e.g., [29]. In the operation of submersible platforms, certain desirable capabilities, including collision detection and avoidance, and tracking targets (for recognition) require the utilization of qualitative information that can be readily extracted from panoramic views. Additionally, quantitative information necessary for certain automatic capabilities of intelligent or (semi-)autonomous vehicles, including accurate localization and positioning, mosaic-based and map-based navigation, and the construction of topographical maps can be determined more robustly from these views than from typical images with a small FOV.

I. INTRODUCTION

In this paper, two prototype panoramic imaging systems are described, and utilized to demonstrate a number of important vision-based capabilities in the operation of underwater vehicles. We explore these in the context of important scenarios in the deployment of unmanned submersible systems, including (semi-)autonomous and operator-assisted/supervised modes of operation. We have skipped details on vision techniques for scene reconstruction that utilize these images, due to the space limitation. Many methods for processing these and (or) standard video images with normal FOV to extract the sought after information are found in the literature on vision, mobile robots, or underwater submersible systems; only a small sample cited here.

Despite research on the realization of automatic sensorbased capabilities, e.g., positioning and station keeping [6, 7, 8, 14, 17, 27, 28], it is likely that many missions involving unmanned submersible platforms, in the near future, may still require human assistance and supervision. In addition to various sensors, the operator relies heavily on the images of the remote scene displayed on his monitor, an thus visualization tools that provide him/her with virtual presence are highly desirable to carry out the task effectively. More precisely, to be able to see the remote 3-D environment surrounding the vehicle, rather than the most current frame of a video, becomes rather important. In this regard and (or) in building a global picture or map of the environment, highresolution images covering a large field of view (FOV) are very informative, motivating work on the construction of accurate photo-mosaics from underwater imagery, particularly online during the mission [5, 7, 27]. Photo-mosaics typically involve the registration of images taken from very distinct view points. Another form of highresolution imagery covering a large FOV is a panorama that

II. PANORAMIC IMAGING SYSTEMS A panoramic view can be constructed from cameras with a finite FOV in one of several ways: Utilizing mirrors and special lenses [15, 21], mosaicking images as a camera pans a full 360 degrees [20], or use of several cameras [24]. Given the complexities in building a water-proof housing for undersea applications, the third scenario provides the simplest design, and an economical solution by employing a set of small off-the-shelf security cameras with an average FOV, positioned symmetrically to cover a panoramic view.

∗ Research done with support from the NSF under grant BES-9711528. Jacques Oles is graduate student at Centre d´e Oc´eanologie de Marseille, France on a summer internship at UVIL with support from R´egion Paca.

MTS 0-933957-28-9

1

Based on the application, these can be adjusted in various, from forward-look to perfect down-look, configurations. A forward-look position gives a full view of the surrounding scene, useful for target tracking, detection of obstacles or free path, and mapping of vertical structures. Specific applications include the inspection and mapping of the internal sections of pipelines and nuclear reactor housings. A slanted down-look setup enables mapping a large area of the bottom surface at each view point, as the platform navigates along the sea floor.

bottom of our water tank, however, the images were taken after emptying the tank since a water-proof housing is yet to be built. The coverage, at this short distance from the bottom, is a length of 7.5’ along the shorter of the two directions (field of view varies with depth, but is no less than about 113 degrees in this direction). No radiometric correction was done to achieve a seamless transition at the boundaries, so we can evaluate the image registration more clearly. The projection onto the conic surface approximately represents the “ideal” slanted down-look panorama if one of the cameras pans about a vertical axis through the optical center, collecting at each camera position (a narrow band around) the vertical scan-line in the center of the image. The primary discrepancies of the generated panorama with the view from the panning system arise from the non-zero baselines between the focal points of neighboring cameras, and diminish for distant targets.

Fig. 1 depicts two prototype panoramic imaging systems developed at the Underwater Vision and Imaging Laboratory (UVIL) at the University of Miami, in forward-look and slanted down-look configurations. Each comprising 6 cameras with nearly 71-degree in FOV (in air) arranged in a symmetric configuration. The forward-look setup provides overlap between two neighboring images within a short distance of less than one feet, though it has not been optimized with respect to binocular cues for 3D reconstruction or 3D visualization in human-assisted operations, but simply targeted for an object tracking or obstacle avoidance sensor. Alternatively, two or more cameras– e.g., those for the front view– may be positioned in a typical parallel stereo configuration when targeted for 3D mapping of vertical structures; e.g., thermal vents or coral reefs. The down-look arrangement, at an angle of nearly 60 degrees from the horizontal plane, provides much larger overlap, and is designed for monocular and binocular stereo imaging over the sea floor.

As for the forward-look system, vertical objects would not be aligned properly within overlapping regions due to the discrepancy in disparity (in comparison to points on the frontal plane that have been brought into registration). A gross analogy can be made with an image where objects at a certain depth and beyond are in focus, but not the others. For this down-look arrangement, the large overlapping regions and their sizes were chosen by design in order to provide binocular cues for 3D reconstruction. Therefore, the misalignments provide us with a strong visual cue for 3D visualization (by the operator is human-assisted/supervised missions) and reconstruction, rather than being an undesirable feature.

Fig. 2a depicts 6 raw distorted images of the forward-look system, utilized in constructing the sample panoramic view in (b). To do this, each raw frame is calibrated and rectified to remove lens distortion [10], mapped to a cylindrical image as shown in (c), and finally unwrapped for display as a 2-D image. The cylindrical image ideally corresponds to a view seen by a continuously panning camera, as the central vertical scan-line of each frame is added to the panorama. Because of the finite, though small, baseline between neighboring cameras due to size constraints and a wide range in depth of the scene objects, not every corresponding image pair within overlapping areas can be registered perfectly. To achieve perfect registration requires solving the stereo depth reconstruction problem over the overlapping regions and consequently re-mapping the scene onto the cylinder by synthesizing a panning motion. Alternatively, two other solutions may be suitable depending on the application. In human-assisted missions, it may be advantageous to maintain, or enforce larger, overlapping areas for 3D visualization by displaying these regions on different channels of a color image. This is suitable for visualization during the operation and reconstruction in post-mission processing, if necessary. Conversely, the relative orientation of the neighboring cameras can be actively adjusted, where the design allows, to control the size of the overlap as target distances change due to the vehicle motion. Doing this automatically requires the extraction some information about the disparity range of the corresponding points from the images.

III. PREPROCESSING A. Calibration and Distortion Correction The 3-D scene reconstruction up to some scale or projective transformation ambiguity by utilizing un-calibrated stereo cameras has become a popular paradigm in vision research over the last decade, tracing back its roots to photogrammetry and earlier motion vision research; e.g., 8-point algorithm of Longuet-Higgins [4, 9, 13]. For absolute 3-D reconstruction, however, the imaging system has to be calibrated to determine the internal/intrinsic parameters, and to remove the lens distortion effect. In this work, we have chosen to utilize from numerous techniques in the vision literature the method in [10] for intrinsic parameter calibration and distortion correction. The external parameters, describing the position and orientation of each camera relative to some common coordinate frame (say, one for a base view or camera) are commonly calibrated by utilizing the matches of some control points. It is noted that intrinsic calibration of the cameras is done once, and so is the registration of the images to construct the panorama for any fixed system configuration, as described next (but has to be repeated if the arrangement of the cameras is modified). B. Construction of Panoramic Views

Fig. 5 shows the raw distorted images of the down-look panoramic imaging system, and the constructed panoramic view after calibration and distortion correction. (Details of the construction have been given in the next section.) The system was set up at a distance of 2ft from the scene at the

The methodology for the construction of the front-look panoramic view utilizes a cylindrical projection described in the vision literature, and is skipped here (e.g., [20]. We provide details for the construction of the down-look panorama. 2

The transformations that map points from each image to the panoramic view can be expressed as a rigid body motion of a plane from one position and orientation to another These can be determined from closed-from solutions that utilize the correspondences or optical flows from a minimum of four image points [9]. Without loss of generality, let us assume that camera 1, once projected to a perfect down-look view defines the plane of the panorama. The matching points to be used are in the overlapping regions of neighboring cameras, k and k + 1 (k = 1, . . . 5). To map points of any image k to the panorama, two options are available: To map from k to k −1, then from k − 1 to k − 2, etc. Alternatively, we could map in the opposite direction. For example, one can either use the mapping computed directly for cameras 6 and 1, or to map from 6 to 5, then from 5 to 4, . . ., which are identical with noise free data. However, there is discrepancy between these two mappings with noisy data, resulting in some misalignment at certain boundaries. To determine the optimal result, we need to enforce the constraint on the equality of these two mappings, which is a ”continuity constraint.” In practice, equality cannot be achieved perfectly and we thus have to minimize the difference, as well as the misalignment errors of corresponding points in neighboring images. This can be formulated as a global optimization method, which is no longer solved in closed form, but iteratively. We have implemented an alternative method to achieve the optimum alignment through another global non-linear optimization, where we exploit a priori knowledge about the system geometry. Assuming a perfectly aligned system according to the desired slanted down-look configuration, achieving a perfect down-look view (alignment of optical axes) involves a rotation by 30 degrees about the X-axis of each camera. Alignment of the XY axes requires rotation about the optical axis (new Z-axis) by 60 degrees. Finally, a fixed shift for each image would align the corresponding points lying on a common frontal plane. Due to camera misalignments, the first rotation may also include a small component about each camera’s Y axes, and various rotation angles may deviate from the assumed values. The global optimization method involves determining various rotation angles and shifts that minimize the total registration error for all matching points over the 6 views. To do this, we arbitrarily choose one camera to define the base frame, say camera 1. The image is warped to a perfect down-look view based on the following model: 





kxp  kyp  =  R k

 x1 0  1  y  0   0  f 1 

The left panorama in fig. 5 has been constructed by utilizing the above constraint equations to determine angles θxi , θyi , and θzi (i = 1, . . . , 6) and the shifts xio and yoi (i = 2, . . . , 6) that minimize the total registration error over manually selected corresponding points, through the application of the Levenberg-Marquardt optimization method. IV. PANORAMIC VIEWS FOR VISUAL TASKS We study a number of capabilities and scenarios for the utilization of panoramic views in the deployment of submersible platform. In human-assisted missions, the operator relies heavily on the video for remote command and control. With no restriction on video transmission, raw images of each camera may be transmitted to the surface, and the processing to construct the panorama is done on a main computer before display on the operator monitor. Alternatively, a local processor on the submersible platform can be dedicated to processing the raw data in order to directly transmit the panorama, instead. This can be done at full resolution, or lower resolutions as described in this section. A. Motion Estimation Various methods, to estimate 2-D image and 3-D camera motions, has been the core of the recent developments of vision systems for underwater applications, including station keeping, positioning and navigation, pipeline following, target tracking, and photo-mosaic construction, among others [5, 6, 7, 14, 27, 28]. The main advantage in utilizing panoramic views for motion estimation is the improved robustness that is offered by a very large field view [19]. In particular, one can overcome inherent ambiguities– including distinction between certain rotational and translational motion components– that present themselves when attention is restricted to a narrow view [1]; e.g., translation in the X/Y direction induces an image motion very similar to that due to camera roll/pitch motion (rotation about the Y /X axis), as depicted through synthetic optical flows in fig. 3, and those computed from real images in fig. 2. In the example of fig. 2 involving the outlook panoramic system (see bottom three panoramas), we compute and compare two optical flows: For two views before and after panning motion of the system (rotation about vertical axis), and for before and after a translation parallel to the scene viewed in the front camera. Both of these two motions induce horizontal flows in the front and back views, but can be readily discriminated in the side views. The motion components contribution to the inherent ambiguities of a narrow view vary with the configuration with respect to the vehicle and the scene. However, any of the six components in translation and rotation induce an image motion pattern that is distinct over the entire panoramic view. B. Mapping



where R = R x R y .

Here, R i denotes a rotation by angle θi (to be determined) about axis i, p 1 = [x1 , y 1 ] is the coordinate in the base frame and p p = [xp , yp ] is the coordinate in the panorama (θx ≈ 30, and θy ≈ 0). The transformation for points p i in camera i is based on the applying the above equation first to arrive at p ip = [xip , ypi ], followed by a planar rotation and translation: ·

xp yp

¸

· =

cos θz sin θz

− sin θz cos θz

¸·

xip ypi

¸

· +

xio yoi

Depending on the system design and size of overlaps of neighboring cameras, where stereo disparity cues are available, we can utilize stereo and (or) motion cues for 2-D (photo-mosaic) and 3-D (topographical) mapping, as the platform moves relative to the scene [12]. The primary advantage in utilizing the panoramic view is data redundancy

¸ ,

where θz ≈ (i − 1) ∗ 60 degrees. 3

and extended coverage offered by a large field of view. Extended coverage at each sample time provides another significant advantage in applications where the continuous use of light sources is prohibited due to power consumption limitations. Here, strobe lighting and reduced frame rate still enables a full coverage in mapping the scene.

In addition to standard still-image and motioncompensated video compression techniques, adaptive spatial sampling based on space-variant representation in constructing the panorama is an effective strategy to reduce the data size significantly before transmission. Particular implementation may rely on the application, operational mode, constraints of the mission, etc. As an example, we demonstrate a selected strategy based on utilizing the logarithmic transformation. We assume that regions of interest (activity areas) has been identified based on processing the panoramic image (say, the four moving targets in fig.2), and these are to be transmitted at a higher resolution that the rest of the image. In fig. 4, three different compressions, down to half the size of a single camera image, and the reconstruction based on cubic-spline interpolation to recover the original size are compared. The difference is mainly the relative degree of compression between the areas of activity and the rest of the image, which can be readily verified over the non-activity areas. This simplified example serves to demonstrate the concept, without any attempt to optimize the reconstruction performance. In practice, various compression and reconstruction methods may be utilized (simultaneously) to improve the compression rate, reconstruction quality, etc.

C. Obstacle Avoidance For detecting obstacles and targets within the field of view, the panoramic imaging system is arranged such that the vehicle motion is dominantly in the plane formed by the optical axes of the cameras. A useful quantity for obstacle avoidance is ”time-to-collision” which can be readily determined from a sequence of images. While there is a scale factor ambiguity in recovering the motion magnitude or distances to scene targets based on motion cues, the ”time-to-collision” is a measure determined from the ratio of the distance to velocity, without ambiguity (e.g., [3, 26]). In a panorama, this can be determined for the entire surrounding scene, as the vehicle moves in any arbitrary direction. In particular, this information is most critical when operating the submersible within enclosed structures, say a pipeline, housing, cave, etc. D. Moving Target Detection and Tracking Motion provides a strong visual cue to identify a moving target in the image [23, 25]. The target typically has a different image motion (size and/or direction) than the background, and segmentation can be done by identifying the discontinuity boundaries of the image motion. Consider the top two panorama views in fig. 2, where the four students move with respect to a stationary background. The estimated optical flow have been superimposed, depicting the image motion of the moving objects. Clearly, a dynamic panorama (or cylindrical projection shown in the last row), actively detecting and marking moving targets, provides the unique advantage to track any such object in any position relative to the vehicle. The last row of fig. 5 shows 2 consecutive frames of the camera in the SE position, as the imaging system moves in the forward direction, while the lobster moves closer to the cylindrical pipe. The estimated optical flow based on a multiscale implementation of the GDIM-based method in [16], within the marked rectangular region, verifies the motion of the imaging system, as well as the independently moving lobster which can be segmented out by the application of the methods in [23, 25].

V. SUMMARY Teleconferencing, mapping, and navigation are primary terrestrial applications of panoramic views. We have covered potential uses of panoramic imaging for a range of visionguided/based capabilities in the deployment of unmanned submersible platforms (or space robotics systems): Target tracking and obstacle avoidance, motion estimation for positioning and navigation, and mapping. We have described two prototype panoramic imaging systems in forward-look and down-look configurations, developed at the University of Miami’s UVIL. We have presented simplified examples, solely to highlight certain unique advantages of panoramas in various application scenarios ranging from remotely-controlled to autonomous operational modes, and for 3-D visualization and reconstruction. Selected advantages include large scene coverage and data redundancy at video rate for unrestricted applications, lower data rate with strobe lighting without compromising coverage where power consumption is limited, and adaptive adjustment of image resolution to meet bandwidth constraints.

E. Adaptive Spatial Sampling

References

In certain operations, channel bandwidth limitations may impose serious restrictions on the transmission of highresolution panoramas. Image compression techniques and (or) multi-resolution representations, including space-variant foveated retina-type architectures and log-polar mapping, may be adopted to achieve the desired data size [2, 11, 22]. In particular, when the panoramic imaging system is primarily used for obstacle detection and target tracking, e.g., in a human-assisted mission, it may suffice to transmit an image at low resolution over most of the scene, excluding the regions corresponding to and surrounding the areas of interest that are to be viewed at full resolution.

[1] G. Adiv, “Inherent ambiguities in recovering 3-D motion and structure from a noisy flow field,” IEEE PAMI, Vol 11(5), May, 1986. [2] G. Bonmasser, E.L. Schwartz, “Space-variant Fourier analysis: the exponential chirp transform,” IEEE PAMI, Vol 19(10), October, 1997. [3] R. Cippola, and A. Blake, ”Surface orientation and toime to contact from image divergence and deformation,” Proc. ECCV, S. Margherita Ligure, Italy, 1992. [4] O. Fuageras, Three-dimensional computer vision, MIT Press, Cambridge, MA, 2001. 4

Figure 1: Images of the forward-look and down-look panoramic imaging systems. [5] S.D. Fleischer, H.W. Wang, S.M. Rock, and M.J. Lee, “Video mosaicking along arbitrary vehicle paths,” Proc. Symp. AUV Tech., Monterey, CA, June, 1996.

[18] S. Negahdaripour, and A. Khamene, ”Motion-based comression of underwater video imagery for the operations of unmanned submersible vehilces,” Computer Vision & Image Understanding, Vol 79, July, 2000.

[6] S. D. Fleischer, and S.M. Rock, “Global position determination and vehicle path estimation from a vision sensor for real-time mosaiking and navigation,” Proc. Oceans’97, Halifax, Nova Scotia, October, 1997.

[19] R. C. Nelson, and J. Aloimonos, , “Finding motion parameters from spherical flow fields (or the advantages of having eyes in the back of your head),” Proc. Workshop on computer vision, November, 1987.

[7] N. Gracias, and J. Santos-Victor, ”Underwater video mosaics as visual navigation maps,” Computer Vision & Image Understanding, Vol 79, July, 2000.

[20] Peleg, S. and J. Herman, “Panoramic mosaics by manifold projection,” Proc. CVPR, June, 1997. [21] Peleg, S., Ben-Ezra, Y. Pritch, “Omnistereo: Panoramic stereo imaging,” IEEE PAMI, Vol 23(3), March, 2001.

[8] N. Gracias, and J. Santos-Victor, ”Underwater mosaicing and trajectory reconstruction using global alignment,” Proc. Oceans’01, Honolulu, HI, November, 2001. [9] R. Hartley, A. Zisserman, Multiple view geometry in computer Vision, Cambridge Univ. Press, UK, 2000.

[22] G. Sandini, F. Bosero, F. Bottino, A. Ceccherini, “The use of an antropomorphic visual sensor for motion estimation and object tracking,” Proc. OSA Topical Meeting Image Understanding and Computer Vision, 1989.

[10] J. Heikkila, and O. Silven, “A four-step camera calibration procedure with implicit image correction,” Proc. IEEE CVPR, Puerto Rico, June, 1997.

[23] A. Spoerri, and S. Ullman, “Early detection of motion boundary,” Proc. ICCV, London, England, 1987.

[11] R. Juday “Log-polar drapped target,” Optics Letters, Vol 20(21) November 1995.

[24] R. Swaminathan, Nayar, S., “Polycameras: Camera clusters for wide angle imaging,” CUCS-012-99, Dept. Computer Science, Columbia Univ., NYC, NY, 1999.

[12] A. Khamene, A., “Motion and structure from multiple cues; optical Flow, shading Flow, and stereo disparity,” Ph.D. Thesis, ECE Dept., Univ. Miami, August, 2000.

[25] W.B. Thompson, and T.C. Pong, “Detecting moving objects,” Proc. ICCV, June, 87. [26] M. Tistarelli, and G. Sandini, ”On the advantages of polar and log-polar mapping for direct estimation of timeto-impact from optical flow,” PAMI, Vol 14(4), 1993.

[13] H.C. Longuet-Higgins, “A computer algorithm for reconstruction a scene from two projections,” Nature, September, 1981. Vol B-208.

[27] X. Xun, S. Negahdaripour, “Vision-based motion sensing for underwater navigation and mosaicing of ocean floor images,” Proc. Oceans, Halifax, October, 1997.

[14] J.F. Lots, D.M. Lane, E. Trucco, ”Application of 2.5D visual servoing to underwater vehicle station keeping,” Proc OCEANS’00, Rhode Island Convention Center, Sept, 2000.

[28] X. Xu, S. Negahdaripour, “Automatic Optical Station Keeping and Navigation of an ROV; Sea Trial Experiments,” Proc. Oceans, Seattle, WA, Sept., 1999.

[15] S.K. Nayar, “Catadioptric omindirectional camera,” Proc. IEEE Conf. CVPR, Puerto Rico, June, 1997.

[29] J.Y. Zheng, Tsuji, S., “Panoramic representation for route recognition by a mobile robot,” International Journal of Computer Vision, Vol 9(1), 1992.

[16] Negahdaripour, S., “Revised Definition of Optical Flow: Integration of Radiometric and Geometric Cues for Dynamic Scene Analysis,” IEEE Trans. PAMI, Vol 20(9), September, 1998. [17] S. Negahdaripour, X. Xu, L. Jin, “Direct estimation of motion from sea floor images for automatic station-keeping of submersible platforms,” IEEE Journal Oceanic Eng., Vol 24(3), July, 1999. 5

Figure 2: Top row shows the 6 raw distorted images, and the next row is the panoramic view constructed from rectified images. Next row shows a lab scene before motion, while the next two rows show images after panning motion -vs- translation parallel to the scene, superimposed with the estimated flow, respectively. While there may be an ambiguity in motion inrerpretation when attention is restricted to the front and back cameras, this is resolved when taking into account the information from the side views. The last row shows the cylindrical view of the first panorama, and the view after the motion of four objects, as marked. 6

N

NW

SW

S

SE

NE

Figure 3: Shown, from left to right, are two superimposed optical flow patterns over the 6 views of the panoramic imaging system (in counterclockwise direction, N, NW, . . . , to NE), induced by vertical translation and pitch motion of the imaging system. Though these patterns may be difficult to distinguished when computed over certain views (cameras 1, 2, and 5 at the front end), they can be readily discriminated when analyzed over the entire panorama.

Figure 4: Three panoramas (left column) compressed to half the horizontal resolution of a single camera image based on Logarithmic transformation, and reconstruction (right column) by inverse transformation and cubic-spline interpolation to recover the original size, maintaining activity areas (moving targets) at high resolution while other regions are constructed at lower resolution.

7

Figure 5: Six raw images recorded by the down-look imaging system, at a distance of about 2ft from the bottom of our 6’ wide by 12’ long water tank. The images were taken in air (emptied tank) since a water-proof housing is yet to be built. Panorama view was constructed from these images after rectification to remove lens distortion, based on the global nonlinear optimization described in section 3 (showing adjacent images over alternating red and green channels, the overlap regions are displayed in yellow). The conic projection (in analogy with cylindrical projection for forward-look setup) approximately represents the image that would be obtained by a camera, in the slanted down-look position, panning about a vertical axis, collecting at each camera position the vertical scan-line in the center of the image. The last row shows consecutive frames during the motion of the imaging system, and the lobster. The estimated optical flow, shown over a window around the lobster, verifies the two independent motions, and can be used to segment out the moving target.

8