an approach to coded structured light to obtain three dimensional ...

AN APPROACH TO CODED STRUCTURED LIGHT TO OBTAIN THREE DIMENSIONAL INFORMATION

Joaquim SALVI MAS

ISBN: 84-8458-107-1 Dipòsit legal: GI-1228-2001

PhD. Thesis UNIVERSITY OF GIRONA Department of Electronics, Informatics and Automation to obtain the degree of:

European PhD in Industrial Engineering by:

Joaquim Salvi Title of the thesis:

An Approach to Coded Structured Light to Obtain Three Dimensional Information Submitted and presented in Girona, december 1997 and february 1998 respectively:

Supervisor: Dr. Joan Batlle (Universitat de Girona – Spain) Supervisor: Dr. El Mustapha Mouaddib (Université de Picardie Jules Verne – France) Referee: Referee:

Professor Bernard Dubuisson (Université de Compiègne – France) Dr. Geoff Roberts (University of Wales College Newport – Great Britain)

President: Examiner: Examiner: Examiner: Examiner:

Professor Josep Amat (Universitat Politècnica de Catalunya – Spain) Professor Juan Jose Villanueva (Universitat Autònoma de Barcelona – Spain) Professor Antonio Perez (Universidad Politécnica de Madrid – Spain) Professor Emilio Zapata (Universidad de Málaga – Spain) Dr. Geoff Roberts (University of Wales College Newport – Great Britain)

7

Judwlwxghv Lq wkh sdvw ihz |hduv L kdyh ehhq oxfn| wr zrun zlwk d orw ri shrsoh zklfk kdyh khoshg ph/ lq rqh ru dqrwkhu zd|/ zlwk wkh kdug dqg vwlpxodwlqj zrun ri uhvhdufklqj1 Wklv vkhhw ri sdshu wulhv wr wkdqn wkhp1 Vxuho|/ lw lv udwkhu gl!fxow wr h{sodlq wkh judwlwxgh L ihho vd|lqj mxvw zrugv/ exw L zloo wu|111 Iluvwo|/ L kdyh wr wkdqn Mrdq Edwooh/ wkh gluhfwru ri wklv wkhvlv dqg dovr rqh ri p| ehvw iulhqgv1 Wkh khos kh jdyh ph lq d orw ri glhuhqw zd|v= olnh zulwlqj grzq wkdw duwlfoh zlwk d ghdgolqh lq yh gd|v/ olnh vhdufklqj vwd|v lq iruhljq xqlyhuvlwlhv/ dqg wkh vhyhudo krxuv kh vshqw wr frqylqfh ph ri vrph gdun dvshfwv ri wklv zrun/ kdv vxuho| ohg ph wr suhvhqw wrgd| wklv wkhvlv1 L zrxog olnh wr wkdqn wkh juhdw khos jlyhq e| p| iulhqg dqg fr0gluhfwru ri wklv wkhvlv Ho Pxvwdskd Prxdggle/ zkr zhofrphg ph lq klv uhvhdufk jurxs Htxlsh Shufhswlrq hq Urerwltxh dw wkh Xqlyhuvlw| ri Slfduglh lq Dplhqv +Iudqfh,1 Wkhuh/ L ohduqw doo wkrvh dvshfwv ri fdoleudwlrq gxulqj wzr wkuhh0 prqwk vwd|v1 L uhphpehu wkh ehhuv zh kdg dw wkh Elvvds Fdih glvfxvvlqj wkrvh frpsolfdwhg htxdwlrqv/ dqg klv sdwlhqfh zkhq L prqrsrolvhg klv frp0 sxwhu gd| diwhu gd|1 L dovr zdqw wr phqwlrq klv zlih Ydohulh dqg gdxjkwhu Qlvvd1 L zdqw wr wkdqn doo wkh iulhqgv ri wkh uhvhdufk jurxs ri Frpsxwhu Ylvlrq dqg Urerwlfv dw wkh Xqlyhuvlw| ri Jlurqd/ iru fuhdwlqj wkdw dwprvskhuh zklfk doorzv dq|rqh wr zrun zlwkrxw lqwhuuxswlrqv1 Iru doo ri wkhp/ dqg iru wkhlu iulhqgo| uhodwlrqvkls/ wkdqnv wr= [hyl Fxið/ Mrdq Pduwð/ Udid Jdufld/ Shs Iruhvw/ Mrugl Iuhl{hqhw/ Shuh Ulgdr/ Ooxlv Sdfkhfr/ Wrql Ihuqdqgh} dqg Shs Fruwdgd1 L dovr zdqw wr phqwlrq Mrugl Uhjlqföv dqg Hqultxh Wruuhv iru wkhlu iulhqgo| uhodwlrqvkls1 L zrxog olnh wr wkdqn Mrvhs Wrpdv/ Vdoydgru Vdodqryd dqg [dyl Pdq|hu iru wkhlu whfkqlfdo vxssruw/ dqg Rojd Pruhqr iru khu dgplqlvwudwlyh dvvlv0 wdqfh1 D yhu| vshfldo judwlwxgh wr doo p| iulhqgv ri wkh Htxlsh Shufhswlrq hq Urerwltxh lq Dplhqv zkr zhofrphg ph lq wkhlu ode1 Wkdqnv iru doo wkh khos L uhfhlyhg/ dqg iru wkhlu sdwlhqwh wu|lqj wr xqghuvwdqg p| dzixo iuhqfk1 L zloo

8 qhyhu irujhw wkh prphqwv zh vkduhg1 Wkdqnv wr= Odxuhqw Ghodkrfkh/ Sdvfdo Ydvvhxu/ Euxqr Pduklf/ Fodxgh Shjdug/ Hulf Eudvvduw/ dqg lq jhqhudo hyhu| rqh ri wkh Htxlsh Shufhswlrq hq Urerwltxh1 Hvshfldoo|/ L zdqw wr wkdqn Sdwulfld H|vnhqv iru khu iulhqgo| khos zlwk p| hqjolvk/ uhylvlqj wkh sxeolvkhg duwlfohv dqg hyhq wklv wkhvlv1 L zrxog olnh wr wkdqn Shs Yhkð dqg Mrdq Pduwð iru wkh wlph zh vshqw wdonlqj derxw krz wr vroyh vrph ri wkh sureohpv lqyroyhg zlwk ODWH[ dqg Vflhqwlf ZrunSodfh1 L zdqw wr wkdqn wkh shrsoh ri wkh Frpsxwhu Ylvlrq Fhqwuh ri wkh Xqlyhu0 vlwdw Dxwrqrpd gh Edufhorqd/ zkr doorzv ph wr xvh wkhlu oleudu|1 L dp dovr judwhixo wr doo wkh shrsoh zkr jdyh ph wkhlu dgylfhv dqg uhf0 rpphqgdwlrqv derxw wkh frqwhqwv ri wklv wkhvlv1 Hvshfldoo|/ L zdqw wr wkdqn Dolfld Fdvdov ri wkh Xqlyhuvlwdw Srolwëfqlfd gh Fdwdoxq|d iru khu zruwk| uhf0 rpphqgdwlrqv1 L zrxog dovr olnh wr h{suhvv p| judwlwxgh wr wkh uhylhzhuv ri wklv wkhvlv= Jhr Urehuwv ri wkh Xqlyhuvlw| ri Zdohv Froohjh dqg Ehuqdug Gxexlvvrq ri wkh Xqlyhuvlwì gh Whfqrorjlh gh Frpslëjqh1 L kdyh wr wkdqn doo wkh shrsoh ri wkh Hohfwurqlfv/ Lqirupdwlfv dqg Dx0 wrpdwlrq Ghsduwphqw dqg wkh Lqvwlwxwh ri Lqirupdwlfv dqg Dssolfdwlrqv ri wkh Xqlyhuvlw| ri Jlurqd dv zhoo/ dqg doo wkh shrsoh ri wkh Oderudwrluh ghv V|vwëphv Dxwrpdwltxhv ri wkh Xqlyhuvlw| ri Slfduglh1 Dw odvw/ exw iru ph lv wkh prvw lpsruwdqw/ L zdqw wr wkdqn wkh vxssruw uhfhlyhg iurp p| zkroh idplo|/ hvshfldoo| iurp p| sduhqwv Mrvhs Vdoyl dqg Slodu Pdv dqg p| eurwkhu Gdylg1 L zrxog dovr olnh wr h{suhvv p| judwlwxgh dqg oryh wr Doh{dqgud iru khu sdwlhqfh zkhq d zhhnhqg diwhu dqrwkhu L kdg wr zrun/ dqg zh frxog qrw jr rxw1 L dovr kdyh wr phqwlrq khu juhdw khos uhylvlqj p| hqjolvk1 L krsh/ L kdyh qrw irujrwwhq dq|erg|1 Li wkhuh lv vrphrqh zkr fdq qrw qg klv2khu qdph lq wklv olvw/ L dsrorjl}h dqg L lqylwh klp2khu wr zulwh lw grzq lq wkh iuhh vsdfh zklfk iroorzv wklv sdudjudsk1

Mrdtxlp Vdoyl

Frqwhqwv 4 Lqwurgxfwlrq 47 414 Ghvfulelqj wkh lqwhuhvw ri wklv wkhvlv 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 47 415 Vfkhpd ri wkh wkhvlv 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 49 5 Fdoleudwlrq 514 Frqvlghudwlrqv 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 515 Prghoolqj d fdphud 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 51514 Wkh H{wulqvlf Sdudphwhuv 1 1 1 1 1 1 1 1 1 51515 Wkh Lqwulqvlf Sdudphwhuv 1 1 1 1 1 1 1 1 1 1 51516 Vxppdu| ri Fdphud prghoolqj 1 1 1 1 1 1 516 Fdoleudwlqj d fdphud 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 51614 Fdoleudwlqj wkh olqhdu prgho 1 1 1 1 1 1 1 1 51615 Fdoleudwlqj wkh qrq0olqhdu prgho 1 1 1 1 1 517 6G Uhfrqvwuxfwlrq 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 51714 Uhfrqvwuxfwlqj iurp olqhdu fdoleudwlrq 1 1 51715 Uhfrqvwuxfwlqj iurp qrq0olqhdu fdoleudwlrq

1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1

6 Wkh Hslsrodu Jhrphwu| 614 Wkh ixqgdphqwdo pdwul{ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 615 Fkdqjlqj wkh fr0ruglqdwh v|vwhp 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 61514 Zruog fr0ruglqdwh v|vwhp {hg dw wkh vhfrqg fdphud/ rshudwlqj zlwk uhvshfw wr wkh uvw fdphud1 1 1 1 1 1 1 1 61515 Zruog fr0ruglqdwh v|vwhp {hg dw wkh uvw fdphud/ rs0 hudwlqj zlwk uhvshfw wr wkh vhfrqg fdphud1 1 1 1 1 1 1 1 61516 Uhodwlrq ehwzhhq erwk Ixqgdphqwdo Pdwulfhv 1 1 1 1 1 61517 Ihdwxuhv ri Srlqwv dqg Yhfwruv 1 1 1 1 1 1 1 1 1 1 1 1 1 1 616 Ghwhuplqlqj wkh ixqgdphqwdo pdwul{ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 61614 Wkh Hljkw0Srlqwv Phwkrg 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 61615 Frqfoxvlrqv 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 61616 Wkh dojrulwp xvhg wr lqihu wkh ixqgdphqwdo pdwul{ 1 1 61617 Wkh dojrulwkp xvhg wr rewdlq wkh hslsrodu olqhv iurp $ 9

4; 4; 4; 53 57 64 67 68 74 7; 7< 85 89 89 96 96 9: 9< :3 :4 :5 :9 :< ;5

FRQWHQWV 7 Vwuxfwxuhg Oljkw 714 Lqwurgxfwlrq 1 1 1 1 1 1 1 1 1 1 1 1 1 715 D Vxuyh|= Frghg Vwuxfwxuhg Oljkw 71514 Srvgdphu0Dowvfkxohu 1 1 1 1 71515 Fduulkloo0Kxppho 1 1 1 1 1 1 71516 Er|hu0Ndn 1 1 1 1 1 1 1 1 1 1 71517 Oh Prljqh0Zd{pdq 1 1 1 1 71518 Prulwd0\dnlpd0Vdndwd 1 1 1 71519 Yx|ovwhnh0Rrvwhuolqfn 1 1 1 1 7151: Wdmlpd0Lzdndzd 1 1 1 1 1 1 1 7151; Zxvw0Fdsvrq 1 1 1 1 1 1 1 1 7151< Jul!q0Qdudvlpkdq0\hh 1 1 1 715143 Pdux|dpd0Deh 1 1 1 1 1 1 1 715144 Lwr0Lvkll 1 1 1 1 1 1 1 1 1 1 1 715145 Fkhq0Kxqj0Fkldqj0Zx 1 1 1 716 Vxppdu| 1 1 1 1 1 1 1 1 1 1 1 1 1 1

:

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

;8 ;8 foj> forvh doo> ( Fdphud fdoleudwhg e| olqhdu phwkrg ( fdpbfdo ( D4 @ W4-W5 ( vdyh D41pdw D4> ( Surmhfwru fdoleudwhg e| olqhdu phwkrg ( surbfdo5 ( D5 @ W4-W5 ( vdyh D51pdw D5> ( 6G uhfrqvwuxfwlrq iurp 5G surmhfwlrqv ordg D41pdw> ordg D51pdw> qsrlqwv @ qxpbribfrxsohv > g @ irshq+*vhwbribfrxsohv1suq*/*u*,> S @ ivfdqi+g/*(g*/^7/qsrlqwv`,> iforvh+g,> S4 @ S+4=5/=,> S5 @ S+6=7/=,> iru l@4=qsrlqwv/ S @ ^D4+4/4,0D4+6/4,-S4+4/l,/D4+4/5,0D4+6/5,-S4+4/l,/D4+4/6,0D4+6/6,-S4+4/l,> D4+5/4,0D4+6/4,-S4+5/l,/D4+5/5,0D4+6/5,-S4+5/l,/D4+5/6,0D4+6/6,-S4+5/l,> D5+4/4,0D5+6/4,-S5+4/l,/D5+4/5,0D5+6/5,-S5+4/l,/D5+4/6,0D5+6/6,-S5+4/l,> D5+5/4,0D5+6/4,-S5+5/l,/D5+5/5,0D5+6/5,-S5+5/l,/D5+5/6,0D5+6/6,-S5+5/l,>`> I @ ^D4+6/7,-S4+4/l,0D4+4/7,> D4+6/7,-S4+5/l,0D4+5/7,> D5+6/7,-S5+4/l,0D5+4/7,> D5+6/7,-S5+5/l,0D5+5/7,`> Y+=/l, @ lqy+S*-S,-S*-I> hqg>

FKDSWHU 51 FDOLEUDWLRQ

51715

85

Uhfrqvwuxfwlqj iurp qrq0olqhdu fdoleudwlrq

Iurp wkh vhfwlrq ghglfdwhg wr fdphud prghoolqj zlwk ohqv glvwruwlrq/ wkh iroorzlqj uhodwlrq kdv ehhq rewdlqhg/   fS 2 L ' s ' f_ n & o f_   ~S  t S 2 T ' s ' t_ n & o t_ ~S

+51 A U}=@duud|+^^frv+w6,/vlq+w6,/3/3`/^0vlq+w6,/frv+w6,/3/3`/^3/3/4/3`/^3/3/3/4``,> A W=@duud|+^^4/3/3/W{`/^3/4/3/W|`/^3/3/4/W}`/^3/3/3/4``,> A U=@pxowlso|+pxowlso|+U}/U|,/U{,> A DD=@pxowlso|+W/U,> A P=@duud|+^^{u`/^|u`/^}u`/^4``,> A J=@pxowlso|+DD/P,> A [=@J^4/4`> \=@J^5/4`> ]=@J^6/4`> A [x=@[-i2]> A \x=@\-i2]> A [g=@+xn0x3,2nx> \g=@+yn0y3,2ny>

FKDSWHU 51 FDOLEUDWLRQ

87

A X=@[x0[g0n4-+[g-[g.\g-\g,-[g> A Y=@\x0\g0n4-+[g-[g.\g-\g,-\g> A XF=@vxev+xn@xf/yn@yf/X,>YF=@vxev+xn@xf/yn@yf/Y,> A XS=@vxev+xn@xs/yn@ys/X,>YS=@vxev+xn@xs/yn@ys/Y,> A & Fdphud prgho A yw4=@ydorubw4f> A yw5=@ydorubw5f> A yw6=@ydorubw6f> A yW{=@ydorubw{f> A yW|=@ydorubw|f> A yW}=@ydorubw}f> A ynx=@ydorubnxf> A yny=@ydorubnyf> A yx3=@ydorubx3f> A yy3=@ydoruby3f> A yi=@ydorubyif> A yn4=@ydorubn4f> A XF=@vxev+w4@yw4/ w5@yw5/ w6@yw6/ W{@yW{/ W|@yW|/ W}@yW}/ i@yi/ nx@ynx/ A ny@yny/ x3@yx3/ y3@yy3/ n4@yn4/ XF,> A YF=@vxev+w4@yw4/ w5@yw5/ w6@yw6/ W{@yW{/ W|@yW|/ W}@yW}/ i@yi/ nx@ynx/ A ny@yny/ x3@yx3/ y3@yy3/ n4@yn4/ YF,> A & Surmhfwru prgho A yw4=@ydorubw4s> A yw5=@ydorubw5s> A yw6=@ydorubw6s> A yW{=@ydorubw{s> A yW|=@ydorubw|s> A yW}=@ydorubw}s> A ynx=@ydorubnxs> A yny=@ydorubnys> A yx3=@ydorubx3s> A yy3=@ydoruby3s> A yi=@ydorubyis> A yn4=@ydorubn4s> A XS=@vxev+w4@yw4/ w5@yw5/ w6@yw6/ W{@yW{/ W|@yW|/ W}@yW}/ i@yi/ nx@ynx/ A ny@yny/ x3@yx3/ y3@yy3/ n4@yn4/ XS,> A YS=@vxev+w4@yw4/ w5@yw5/ w6@yw6/ W{@yW{/ W|@yW|/ W}@yW}/ i@yi/ nx@ynx/ A ny@yny/ x3@yx3/ y3@yy3/ n4@yn4/ YS,> A &&&&&&&&&&&&&&&&&&&&&&&& A & 6G Uhfrqvwuxfwlrq A &&&&&&&&&&&&&&&&&&&&&&&&

FKDSWHU 51 FDOLEUDWLRQ

88

A & Uhdg vhw ri frxsohv 5G A uhdgole+uhdggdwd,> A Ss=@uhdggdwd+vhwbribfrxsohv/7,> A S5GF=@pdwul{+urzglp+Ss,/5/3,=S5GS=@pdwul{+urzglp+Ss,/5/3,= A urzglp+Ss,/6/3,= A iru l iurp 4 e| 4 wr urzglp+Ss, gr A S5GF^l/4`=@rs+4/rs+l/Ss,,=S5GF^l/5`=@rs+5/rs+l/Ss,,= A S5GS^l/4`=@rs+6/rs+l/Ss,,=S5GS^l/5`=@rs+7/rs+l/Ss,,= A rg= A S=@pdwul{+7/6/3,= I=@pdwul{+7/4/3,= S6GV=@pdwul{+urzglp+Ss,/6/3,= A iru l iurp 4 e| 4 zkloh l ?@ urzglp+Ss, gr A yxf=@S5GF^l/4`= yyf=@S5GF^l/5`= yxs=@S5GS^l/4`= yys=@S5GS^l/5`= A XFV=@vxev+xf@yxf/ yf@yyf/ XF,= A D=@frqyhuw+XFV/olvw,=E=@frqyhuw+D^4`/olvw,=XFV=@E^5`.D^5`2E^4`-+42E^6`,= A YFV=@vxev+xf@yxf/ yf@yyf/ YF,= A D=@frqyhuw+YFV/olvw,=E=@frqyhuw+D^4`/olvw,=YFV=@E^5`.D^5`2E^4`-+42E^6`,= A XSV=@vxev+xs@yxs/ ys@yys/ XS,= A D=@frqyhuw+XSV/olvw,=E=@frqyhuw+D^4`/olvw,=XSV=@E^5`.D^5`2E^4`-+42E^6`,= A YSV=@vxev+xs@yxs/ ys@yys/ YS,= A D=@frqyhuw+YSV/olvw,=E=@frqyhuw+D^4`/olvw,=YSV=@E^5`.D^5`2E^4`-+42E^6`,= A D=@frqyhuw+XFV/olvw,= A S^4/4` =@ frqyhuw+D^4`/olvw,^4`= S^4/5` =@ frqyhuw+D^5`/olvw,^4`= A S^4/6` =@ frqyhuw+D^6`/olvw,^4`= I^4/4`=@ 0D^7`= A D=@frqyhuw+YFV/olvw,= A S^5/4` =@ frqyhuw+D^4`/olvw,^4`= S^5/5` =@ frqyhuw+D^5`/olvw,^4`= A S^5/6` =@ frqyhuw+D^6`/olvw,^4`= I^5/4`=@ 0D^7`= A D=@frqyhuw+XSV/olvw,= A S^6/4` =@ frqyhuw+D^4`/olvw,^4`= S^6/5` =@ frqyhuw+D^5`/olvw,^4`= A S^6/6` =@ frqyhuw+D^6`/olvw,^4`= I^6/4`=@ 0D^7`= A D=@frqyhuw+YSV/olvw,= A S^7/4` =@ frqyhuw+D^4`/olvw,^4`= S^7/5` =@ frqyhuw+D^5`/olvw,^4`= A S^7/6` =@ frqyhuw+D^6`/olvw,^4`= I^7/4`=@ 0D^7`= A Y=@pxowlso|+pxowlso|+lqyhuvh+pxowlso|+wudqvsrvh+S,/S,,/wudqvsrvh+S,,/I,= A S6GV^l/4`=@Y^4/4`=S6GV^l/5`=@Y^5/4`=S6GV^l/6`=@Y^6/4`= A rg= A & Uhfrqvwuxfwlrq rewdlqhg lq S6GV^l/4116`

Fkdswhu 6 Wkh Hslsrodu Jhrphwu| Lq wklv fkdswhu zh zloo surylgh d ghvfulswlrq ri wkh hslsrodu jhrphwu| ri d vwhuhrvfrslf v|vwhp pdgh e| wkh uhodwlrq ehwzhhq wzr slqkroh prghov1 Zh zloo dovr h{sodlq krz wklv jhrphwu| lv prghoohg wkurxjk wkh ixqgdphqwdo pdwul{ dqg d phwkrg wr hvwlpdwh wklv pdwul{1 Wkh hslsrodu jhrphwu| lv d srzhuixo frqfhsw zklfk kdv ehhq zlgho| xvhg lq rughu wr uhgxfh wkh fruuhvsrqghqfh sureohp iurp d wzr0glphqvlrqdo wr d rqh0glphqvlrqdo vhdufk1

614

Wkh ixqgdphqwdo pdwul{

Wkh hslsrodu jhrphwu| wdnhv lqwr dffrxqw wkh uhodwlrqvkls wkdw h{lvwv ehwzhhq wzr slqkroh prghov sodfhg lq wkh vsdfh ^]kdq