Real-Time Stereo Matching Using Orthogonal Reliability-Based ...

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 3, MARCH 2007

879

Correspondence Real-Time Stereo Matching Using Orthogonal Reliability-Based Dynamic Programming Minglun Gong, Member, IEEE, and Yee-Hong Yang, Senior Member, IEEE

Abstract—A novel algorithm is presented in this paper for estimating reliable stereo matches in real time. Based on the dynamic programming-based technique we previously proposed, the new algorithm can generate semidense disparity maps using as few as two dynamic programming passes. The iterative best path tracing process used in traditional dynamic programming is replaced by a local minimum searching process, making the algorithm suitable for parallel execution. Most computations are implemented on programmable graphics hardware, which improves the processing speed and makes real-time estimation possible. The experiments on the four new Middlebury stereo datasets show that, on an ATI Radeon X800 card, the presented algorithm can produce reliable matches for of pixels at the rate of frames per second. If needed, the algorithm can be configured for generating full density disparity maps.

80%

10 20

60%

Index Terms—Dynamic programming (DP), programmable graphics hardware, real-time stereo.

I. INTRODUCTION An intensity-based stereo vision algorithm takes two or more images as input and searches for matches based on intensity similarity. Most existing algorithms perform the following four steps: matching cost computation, cost aggregation, disparity calculation, and disparity refinement [11]. The main difference in many of these algorithms is in the optimization method. Approaches that use winner-take-all optimization or dynamic programming (DP) can generate disparity maps in or near real time, but the results are prone to error [1], [2], [6], [15]. Those that employ graph cuts or belief propagation can produce more accurate disparity maps, but require longer computation time [7], [12]. To generate reasonably accurate disparity maps within a short time, we propose to evaluate the reliabilities of matches found through DP and use them to filter out potential mismatches [3]. The reliability of a proposed match on a scanline is defined as the cost difference between the globally best disparity assignment for the scanline that includes the match, and the globally best assignment that does not include the match. The overall cost of a disparity assignment for a scanline is calculated based on intensity differences as well as on a discontinuity cost , which is charged for each pair of neighboring pixels with different disparities. A modified DP algorithm, referred to as the reliability-based DP (RDP) algorithm, is proposed to search for the best disparity assignment for a given scanline with reliability-based thresholding performed at the same time. Manuscript received April 20, 2006; revised October 23, 2006. This work was supported in part by NSERC, in part by Laurentian University, and in part by the University of Alberta. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Hassan Foroosh. M. Gong is with the Department of Math and Computer Science, Laurentian University, Sudbury, ON P3E 2C6 Canada (e-mail: [email protected]). Y.-H. Yang is with the Department of Computing Science, University of Alberta, Edmonton, AB T6G 2E8 Canada (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIP.2006.891344

For efficiency, the RDP algorithm uses an approximate reliability measure instead of the exact reliability in the thresholding process. The error introduced by the approximation is within the range 6, and, hence, the reliability thresholding process becomes less effective when the value of is large. This limitation is addressed in our later work [4], in which the exact reliabilities are calculated through an efficient forward-then-backward approach. The results show that using the exact reliabilities instead of the approximate ones produces more accurate semi-dense disparity maps in less time. The reliable matches found using a single RDP pass may not be dense enough. To produce both semi-dense and full density disparity maps, an orthogonal RDP (ORDP) algorithm is propose to apply the RDP process along both horizontal and vertical scanlines in alternating order [5]. The reliable matches found in one direction are used to guide the following search along the other direction. This explicitly enforces inter-scanline consistency and, hence, reduces the horizontal streak artifacts suffered by most DP-based algorithms. Although how to implement the ORDP algorithm on both the CPU and the graphics processing unit (GPU) is discussed in [5], the algorithm is not fully optimized for parallel execution. To achieve even faster processing speed, a modified GPU-based ORDP algorithm (or GORDP for short) is presented in this paper. Different from our previous paper [5], the paper presented here focuses on the GPU implementation only and provides new experimental results. More importantly, the new GORDP algorithm is optimized for parallel processing with the following key changes. 1) The path tracing step required by most traditional DP algorithms, including the ORDP, is replaced with a local minimum searching step. This not only simplifies the algorithm, but also makes the parallel implementation on the GPU more efficient. 2) While the matching cost packing scheme used in the ORDP algorithm simplifies the GPU/CPU combined implementation, it complicates the GPU-based DP process along horizontal scanlines. A new packing scheme is used here, which packs together the matching costs of a single pixel with four different disparity hypotheses. 3) In order to communicate between different DP passes, the ORDP algorithm treats reliable matches found in previous passes as ground control points (GCPs) [1]. It is easy to constrain the search using GCPs on a CPU, but not on a GPU. A more efficient strategy is used in this paper to guide further DP search through updating the matching costs. A. Related Works To produce smooth disparity maps, many DP algorithms enforce the ordering (monotonicity) constraint [1], which explicitly models occlusions within the scanline. However, this constraint does not hold when there are thin foreground objects in the scene. Instead of limiting the search using the ordering constraint, the scanline optimization algorithm [11] considers all candidates at the previous pixel. Large disparity jumps are allowed, but are penalized with a discontinuity cost. This idea is adopted in the RDP algorithm [3]. Recent advances in CPU processing power make it possible to perform stereo estimation in real time or near real time with software [9]. The performance can be further improved using either the Multi-Media Extension instructions available on recent CPUs [6] or the processing power of GPUs [15]. The local optimization used in these approaches

1057-7149/$25.00 © 2007 IEEE

880


allows fast processing speed, but also limits the accuracy of the disparity maps generated. For better accuracy, DP optimization is used for in real-time stereo systems as well [2]. To our best knowledge, the ORDP algorithm we proposed is the first one that implements a DP-based algorithm on a GPU [5]. More recently, another CPU/GPU combined algorithm is proposed [14], which calculates matching costs on the GPU using an adaptive-weight cost aggregation approach and performs traditional DP on the CPU.

Fig. 1. Two-dimensional matching cost textures obtained. A single pixel in the texture encodes the matching costs of a pixel under four different disparity hypotheses. As a result, a total of four tiles are used to store the matching costs for the 16 different disparity hypotheses.

II. GPU-BASED ORDP ALGORITHM The outline of the proposed GORDP algorithm is as follows: The algorithm starts with calculating the matching costs of assigning different disparity hypotheses to different pixels based on input images. A RDP pass is then performed to search for reliable matches along horizontal scanlines. The reliable matches found are used to guide the following search along vertical scanlines through updating the matching costs. The above two passes normally find reliable matches for of pixels in the source image. If more matches are needed or the full density disparity maps are required, an additional pass is applied along the horizontal scanlines using updated matching costs. The last pass uses the same procedures as the previous RDP passes except that either the reliability threshold value is reduced by half or the thresholding process is completely skipped. The details about steps are discussed in the following subsections.

60% 80%

A. Matching Cost Initialization and Aggregation Here, we use a triplet hp; q; di to represent the stereo match of assigning disparity hypothesis d to pixel p; q . The objective of cost initialization and aggregation process is to compute a 3-D matching cost matrix , in which the cost of match hp; q; di is stored at location p; q; d . 1) Cost Initialization: The matching cost of a given match is initially set to the truncated absolute difference (TAD) between the corresponding pixels. For two pixels with color I0 and I1 , the TAD is calculated using

( )

[

]

C

TAD (I0 ; I1 ) 2 min = c255 max

I0R 0 I1R

+ I0G 0 I1G + I0B 0 I1B 3

; cmax

Fig. 2. Search result texture (a) and its color channels: The red channel (b) stores the best disparity. The green channel (c) keeps the minimum path cost. The blue channel (d) carries the reliability information. As expected, the minimum path costs obtained for different pixels on the same horizontal scanline are the same, resulting in pixels on the same scanline in (c) to have the same intensity.

fore, necessary. In this paper, the 3 2 3 shiftable window approach is used, which replaces the cost of a given pixel with the average costs within a square window anchored at different locations [11]. The effect of 3 2 3 shiftable window is achieved using two rendering passes: a 3 2 3 box filter pass followed by a 3 2 3 min filter pass. Both rendering passes are performed on the GPU using pixel shaders. B. Disparity Computation and Reliability Thresholding

(1)

where I R , I G , and I B are intensities of the red, green, and blue channels. cmax is the truncation value and is set to 33 in the experiment. The truncated costs are scaled to 255 to make full use of the range that can be represented by a single byte. In the GORDP algorithm, the 3-D cost matrix is treated as a stack of 2-D grayscale images, each of which keeps the matching costs of all pixels with a given disparity hypothesis. To utilize the GPU’s vector processing capacity, the grayscale images of every four adjacent disparity hypotheses are packed into the four channels of a color image, which are then tiled together to form a large 2-D texture (see Fig. 1). To generate the costs texture, a control program running on the CPU steps through different four disparity hypotheses groups each time. A rendering pass is performed at each step, with the two input stereo images serving as input textures and the corresponding tile of the output texture serving as the rendering target. The actual computation is preformed by a pixel shader: a function to be executed by the GPU on per-pixel basis. Since a given rendering pass can handle four disparity hypotheses only, the total number of rendering passes needed equals to a quarter of the number of disparity levels. 2) Cost Aggregation: The matching costs calculated based on individual pixels are sensitive to noise. The cost aggregation step is, there-

Here, we discuss in detail how to search for reliable stereo matches along horizontal scanlines on GPU. The process for vertical scanlines is similar. For a given horizontal scanline q , the RDP procedure searches for a global disparity assignment D that minimizes the following global cost term:

w p=1

(C [p; q; D (p; q)] + S (D (p; q) ; D (p 0 1; q)))

(2)

where w denotes the length of the horizontal scanline. The smoothness function S penalizes disparity changes between adjacent pixels, which is defined using the Potts model

S (d; e) =

0; if d = e ; otherwise.

(3)

To find matches that minimizes the above cost term and to calculate the reliability of matches found, the RDP approach uses a forwardthen-backward process [4]. The details on how to perform each step on GPU are discussed below. 1) Accumulated Costs Calculation: The matching costs, which are accumulated from both left-to-right and right-to-left, are kept in two accumulated cost matrices R and L , with R p; q; d (or L p; q; d )

A

A

A[

] A[

]


holding the cost of the best path that starts from match hp; q; di to the right (or left) end of scanline q . Using the Potts model, the accumulated cost matrices can be calculated using

A [0; q; d] = C [0; q; d] A [p; q; d] = C[p; q; d]+min A [p 0 1; q; d] ; M 01 [q]+ A [w 0 1; q; d] = C[w 0 1; q; d] A [p; q; d] = C[p; q; d]+min A [p +1; q; d] ; M +1 [q]+

881

Using D(p; q ) to represent the optimal disparity assignment at pixel and E (p; q ) the corresponding path cost, we have

(p; q )

2[0;r]

d

E (p; q ) = min (

R

R

R

2[0;r]

d

R p

L

L

L

M

L p

(4)

M

where p01 R [q ] and L p+1 [q ] are the minimum accumulated costs at previous pixel locations, p 0 1 and p + 1, respectively, which are calculated using

M 01 [q] = M +1 [q] = R p L p

min

d

2[0;r]

min

2[0;r]

d

A A

0 1; q; d]

R

[p

L

[p + 1; q; d]

(5)

where r is the maximum disparity value. Because the accumulated costs are computed iteratively, a separate rendering pass is needed for each pixel in the scanline. However, since different scanlines are processed independently, we can still employ parallelism by processing all scanlines in the same rendering pass. The GORDP algorithm takes the matching cost matrix as input and produces an accumulated cost matrix, either R or L depending on calculation direction. Similar to matrix , R and L are 3-D matrices and are represented using 2-D textures. When calculating the accumulated costs for the horizontal RDP pass, the control program running on the CPU steps through different vertical scanlines and conducts two rendering passes for each scanline. Using matrix R as an example, when processing scanline k , the first rendering pass generates a 1-D texture k01 R [q ], which keeps the minimum of R [k 0 1; q; d] under different disparity hypotheses d. The second pass calculates R [k; q; d], which is equal to the smaller value of [k; q; d] + R R [k; q; d] + [k 0 1; q; d]. k01 [q ] + and 2) Search for Global Best Matches for Different Scanlines: In most traditional DP algorithms, including our previous RDP approaches [3]–[5], the best stereo matches for a given scanline are obtained by a path tracing step. Since path tracing is an iterative process, implementing it on GPUs is both difficult and inefficient. In the GORDP algorithm, the path tracing is replaced by a local minimum searching step. The new approach is derived based on the following observations: For a given disparity match hp; q; di, we know the cost for the best path that starts from hp; q; di to the right end of the scanline is R [p; q; d] and the cost for the best path from hp; q; di to the left end of the scanline is L [p; q; d]. Connecting the two paths gives the best path for the whole scanline that passes match hp;Rq; di. The costLof such a path, [p; q; d], can be calculated using [p; q; d] + [p; q; d] 0 [p; q; d]. Since the globally optimal solution for the scanline is the path that gives the smallest cost, the best disparity assignment d for a given pixel (p; q ) should have the smallest [p; q; d] value among different disparity hypotheses. As a result, the optimal disparity assignment at different pixel locations can be found in parallel by locally minimizing the best path matrix .

A CA

C A A

A A

M

M

A

C

A

A

A

A

C

C

A B

B

B

C0 [p; q; d] = min ( 1 C [p; q; d] ; 255) ; C0 [p; q; d] = C [p; q; d] ;

B [p; q; d])

D (p; q ) = arg min (

B [p; q; d]) :

(6)

The above computation is performed on GPUs using two rendering passes. The first pass takes matrices , R , and L as input, calculates the best path matrix , and saves it to a 2-D texture. The second rendering pass uses matrix to compute the best disparity assignment D and the corresponding path cost E at the same time. The result is saved to a texture, with the values of D(p; q ) and E (p; q ) stored at the red and green channels of pixel (p; q ), respectively [see Fig. 2(b) and (c)]. For a given scanline, as long as there is a single best path whose cost is minimal, the matches generated by the local minimization approach and by the traditional path tracing approach are identical. This is because the matches on the best path are the unique local minima at all pixel locations. When two or more paths have the same minimum cost, the traditional path tracing approach will randomly pick one of the paths and select matches on the path, whereas the local minimization approach may select matches from different best paths at different pixel locations. However, the reliable matches found by both approaches are still identical. This is because when two or more paths have the same cost, the reliabilities of matches on these paths are all zero. No matter which matches are selected, they will be filtered out by the following reliability-based thresholding process provided the reliability threshold is larger than zero. 3) Reliability-Based Thresholding: Based on the reliability definition, the reliability of the best match found for pixel (p; q ), denoted as R(p; q ), can be calculated locally using the cost difference between the best alternative path cost and the best path cost. Since the best match for pixel (p; q ) is D(p; q ), the best path cost is E (p; q ), and the best alternative path cost is the minimum path cost value under the constraint d 6= D (p; q ), we have

B

R (p; q ) =

CA

B

min

2[0;r]^d6=D[p;q]

d

A

(

B [p; q; d]) 0 E (p; q) :

(7)

The calculated reliability value is used to filter out potential mismatches. That is, the best match found for a given pixel (p; q ) is considered unreliable if R(p; q ) is smaller than a preset reliability threshold t. The reliability-based thresholding is achieved using the following procedure, which is implemented in a single rendering pass: For a given pixel (p; q ), the pixel shader counts the number of disparity hypotheses d that satisfy the condition [p; q; d] E (p; q ) + t. If the count is larger than one, then the cost difference between the global best path and an alternate path is less than the reliability threshold t. The global best match found for pixel (p:q ) is, therefore, unreliable. To label unreliable pixels, the pixel shader sets the blue channels of the corresponding pixels in the search result texture to 255 [see Fig. 2(d)].

B

C. Costs Update Based on Reliable Matches The semi-dense disparity maps obtained from a single RDP pass may not be dense enough, especially for complex scenes such as the “Teddy” and “Cones” shown in Fig. 3(a). To increase the density of reliable matches, the GORDP approach uses an additional RDP pass to search along vertical scanlines. When full density disparity map is required, a third RDP pass is also used to search along horizontal scanlines again.

6

D (p; q ) is reliable and d = D (p; q )

otherwise

(8)

882


Fig. 3. Disparity maps generated for the Middlebury datasets. The green color represents pixels without reliable matches. In several cases, incorrect reliable matches found in previous pass are corrected by the following passes. For example, in the Tsukuba dataset, the error that shows up in between the arms of the lamp in (b) disappears in (c). TABLE I COMPARISON OF SEMI-DENSE DISPARITY MAPS GENERATED

To guide further search using reliable matches found in previous passes, these matches are treated as GCPs in our previous approach [5]. Here, a more efficient strategy is used to guide further search through updating the matching cost matrix. The idea of the new strategy is simple: Once a reliable disparity assignment D(p; q ) has been found for a given pixel (p; q ), the matching cost matrix will be updated to discourage further DP passes selecting any disparity hypothesis other than D(p; q ). This is achieved by raising the matching costs [p; q; d] for all other disparity hypotheses d by a factor of up to the maximum value 255 [see (8), shown at the bottom of the page], where 0 denotes the updated matching cost matrix. The parameter is set to 8 in all the experiments. The matching cost update process is implemented on the GPU using a single rendering pass. The pixel shader takes the original matching cost texture and the search result texture as input, calculates the updated matching costs, and output the results to a new texture. It is noteworthy that raising the matching cost for a given disparity hypothesis only discourages further DP passes to select this disparity hypothesis but does not strictly forbid them to do so. It is possible that the reliable matches already found are not selected into the best paths in the following RDP passes or are selected but considered as unreli-

C C

able. This is different from the ORDP algorithm [5], which treats reliable matches already found as GCPs and strictly enforces the best paths pass through these matches. Our evaluations suggest that imposing the new soft constraint actually works better overall than the original GCP-based approach. This is because under the new strategy, when an incorrect match is considered as reliable in the previous search, the error may be corrected in the following searches. III. EXPERIMENTAL RESULTS The GORDP algorithm is tested on a 3-GHz P4 computer with 512-MB system memory, running Windows XP Professional and Direct3D 9.0. The graphics card used is an ATI Radeon X800 with 256-MB video memory. All shaders are implemented using HLSL and compiled using pixel shader 2.0 profile. Several experiments are conducted to test both the accuracy and speed of the algorithm. In these experiments, both the discontinuity cost and the reliability threshold t are set to 20, which are the same as in the ORDP algorithm [5]. However, in GORDP, these two values are scaled by a factor of 255=cmax when they are sent to the GPU since all the matching costs are scaled by the same factor.


883

TABLE II COMPARISON OF FULL DENSITY DISPARITY MAPS GENERATED. E , E , AND E DENOTE THE ERROR RATES IN UNOCCLUDED REGIONS, ALL AREAS, AND NEAR DISCONTINUITY REGIONS, RESPECTIVELY

A. Comparison of Semi-Dense Disparity Maps In the first experiment, the GORDP algorithm is used to generate semi-dense disparity maps for the four Middlebury stereo datasets [11]. The results obtained after different RDP passes are shown in Fig. 3(a)–(c). Table I gives the quantitative comparison between our approach and existing algorithms proposed for semi-dense stereo matching [4], [10], [12]. The performance metrics are the same as the ones used in previous approaches: density ( ) is the percentage of matches generated and the error rate ( ) is the percentage of bad matches (those beyond the correct disparity 61). Please note that the error rate evaluates matches found within unoccluded areas only (based on the masks provided on the Middlebury website). The results for Sara’s and Veksler’s approaches [10], [12] are based on their papers and the measurements for the two new datasets (Teddy and Cones) are not available. The running times for our GORDP approach and our previous single-pass RDP approach [4] are measured on the 3-GHz computer described above. The time reported by Veksler is measured on a 600-MHz P3 system. In terms of the match density, the comparison shows that, after the first two passes, our GORDP algorithm already produces more matches than the existing approaches. The additional horizontal pass with lower reliability threshold value further increases the density of the matches, but at the expense of increasing error rate. In terms of accuracy, the results obtained by GORDP with two passes are comparable with Veksler’s approach and are better than Sara’s one. In terms of processing speed, the time needed for our approach is less than 1/10 of that required by Veksler’s approach (after taking into account the speed difference of the computers). The GORDP algorithm is also faster than our previous single-pass RDP approach, thanks to the parallel design and the processing speed of the GPU. The speedup is greater for larger datasets due to the relatively small overhead for setting up the rendering process.

E

D

B. Comparison on Full Density Disparity Maps In the second experiment, the GORDP algorithm is used to generate full density disparity maps for the same datasets. To compare with other stereo vision algorithms, we submitted the full density disparity maps [shown in Fig. 3(d)] to the Middlebury Stereo Vision website and the evaluation results are shown in Table II. Evaluations for other approaches (except [15]) are based on the Middlebury website, whereas that of Yang et al.’s approach are based on their paper [15]. Please note that, for some existing algorithms [2], [6], not all the measurements are available at the Middlebury website. The evaluation provided by the Middlebury site shows that, in terms of accuracy, the overall performance of the GORDP algorithm is slightly better than the ORDP approach. As shown in Table II the two algorithms are ranked between the tensor voting [8] and the tree DP [13] approaches, both of which are offline algorithms. Table II also

shows the performance of the current state-of-the-art offline approach [7] for comparison. When compared to existing real-time and near-real-time stereo algorithms, the GORDP is slightly outperformed by Wang et al.’s approach [14], but does much better than the rest [2], [6], [15]. The GORDP algorithm does have an advantage over Wang et al.’s approach in that it is more versatile—it can generate high accuracy semi-dense disparity maps. Computing full density disparity maps does not require the last reliability thresholding process, and, therefore, the time needed is slightly less than applying three full RDP passes. Our measurement shows that the GORDP approach can achieve 30 60 Mde/s,1 with better performance obtained for larger datasets. This is not as fast as Yang et al.’s and Forstmann, et al.’s approaches [2], [15] (both are about 100 Mde/s), is comparable to Wang et al.’s approach [14] (about 40 50 Mde/s on a more advanced graphics card), and is faster than the others [6], [9]. Since several changes are made for better parallel performance, the GORDP algorithm is also faster than the ORDP algorithm. IV. CONCLUSION In this paper, we present a GPU-based orthogonal RDP algorithm, which extends our previous RDP-based algorithms [3], [4]. The use of RDP passes along both horizontal and vertical scanlines explicitly enforces the inter-scanline consistency and, therefore, improves the density and accuracy of the matches found. The experiments show that the GORDP algorithm can generate results as good as the graph cuts-based semi-dense technique [12], but requires much less computational time. If needed, the GORDP algorithm can also be used to generate full density disparity maps. According to the Middlebury stereo evaluation site, the GORDP algorithm produces better disparity maps than many existing real-time approaches [2], [6], [15] and is comparable to Wang et al.’s approach [14]. Differing from the original ORDP approach [5], the GORDP algorithm is designed for parallel execution on the GPU. The iterative path tracing step required by most traditional DP algorithms is replaced by a local minimum searching step, a different matching cost packing scheme is used to facilitate conducting DP in parallel along both horizontal and vertical scanlines, and a more efficient strategy is used to guide further DP search using reliable matches already found. These changes effectively improve the speed of the GORDP algorithm. ACKNOWLEDGMENT The authors would like to thank the anonymous reviewers who provided exceptionally constructive and valuable comments on their

2

1For a stereo dataset with 320 240 pixels and 20 disparity levels, achieving ten frames per second requires 320 240 20 10 = 15:3 million disparity estimations per second (Md/s).

2

2 2

884


earlier submission. They would also like to thank Dr. Y. Ohta, Dr. O. Veksler, and Dr. D. Scharstein for sharing their datasets and/or experimental results.

REFERENCES [1] A. F. Bobick and S. S. Intille, “Large occlusion stereo,” Int. J. Comput. Vis., vol. 33, no. 3, pp. 181–200, 1999. [2] S. Forstmann, J. Ohya, Y. Kanou, A. Schmitt, and S. Thuering, “Realtime stereo by using dynamic programming,” in Proc. Computer Vision Pattern Recognition Conf., 2004, pp. 29–36. [3] M. Gong and Y.-H. Yang, “Fast stereo matching using reliability-based dynamic programming and consistency constraints,” in Proc. Int. Conf. Computer Vision, 2003, pp. 610–617. [4] ——, “Fast unambiguous stereo matching using reliability-based dynamic programming,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 6, pp. 998–1003, Jun. 2005. [5] ——, “Near real-time reliable stereo matching using programmable graphics hardware,” in Proc. Computer Vision Pattern Recogintion Conf., 2005, pp. 924–931. [6] H. Hirschmuller, P. R. Innocent, and J. Garibaldi, “Real-time correlation-based stereo vision with reduced border errors,” Int. J. Comput. Vis., vol. 47, no. 1–3, 2002.

[7] A. Klaus, M. Sormann, and K. Karner, “Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure,” presented at the Int. Conf. Pattern Recognition, 2006. [8] P. Mordohai and G. G. Medioni, “Stereo using monocular cues within the tensor voting framework,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 6, pp. 968–982, Jun. 2005. [9] J. Mulligan, V. Isler, and K. Daniilidis, “Trinocular stereo: A real-time algorithm and its evaluation,” Int. J. Comput. Vis., vol. 47, no. 1–3, pp. 51–61, 2002. [10] R. Sara, “Finding the largest unambiguous component of stereo matching,” in Proc. ECCV, 2002, pp. 900–914. [11] D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Int. J. Comput. Vis., vol. 47, no. 1–3, pp. 7–42, 2002. [12] O. Veksler, “Extracting dense features for visual correspondence with graph cuts,” presented at the Computer Vision Pattern Recognition Conf., 2003. [13] O. Veksler, “Stereo correspondence by dynamic programming on a tree,” presented at the Computer Vision Pattern Recognition Conf., 2005. [14] L. Wang, M. Liao, M. Gong, R. Yang, and D. Nister, “High quality real-time stereo using adaptive cost aggregation and dynamic programming,” presented at the 3DPVT, 2006. [15] R. Yang, M. Pollefeys, and S. Li, “Improved real-time stereo on commodity graphics hardware,” presented at the CVPR Workshop on RealTime 3D Sensors and Their Use, 2004.