Mapvision - the photogrammetric machine vision

applications engineering applications for engineering system for vision system the photogrammetric machine vision Mapvision - the Haggren Henrik Haggrén of Finland Technical Technical Research Centre of SF SF -- 02150 Espoo Finland Abstract

Research Technical Research the Technical by the vision system developed by is aa photogrammetric machine vision Mapvision Mapvision is The applications. The control applications. assembly control and assembly inspection and industrial inspection for industrial Finland primarily for of Finland Centre of Centre especially procameras and an especially solid state cameras system system consists consists of of four four simultaneously simultaneously operating operating solid better is better three-dimensional The three microprocessor. The grammed photogrammetric microprocessor. -dimensional measuring accuracy is dimensional the dimensional example the for example enabling for system enabling is an automated system 1: 5, 000. The than 1:5,000. The Mapvision is feed -backand andinteraction interaction between between computer computer aided aided design design and and manufacturing manufacturing phases. phases. feed-back Introduction viReal -timephotogrammetry photogrammetry has has become become of of increasing increasing importance importance in in industrial machine viReal-time already has already mapping, photogrammetry has topographic mapping, for topographic the general tool for Primarily being sion. Primarily sion. being the it is the applications, as it extreme machine in extreme even in shown shown its its outstanding outstanding usability usability even machine vision vision applications, its predominantly 1 Due shuttle..1 space shuttle the space in the Canadarm in for Canadarm System for case of Space Vision System case Due to its real-time factories, real today's factories, of today's demands of automation demands high automation the high facing the in facing superior competence in -time three-dimenaccurate threein accurate also in role also astounding role likely astounding its likely grounding its photogrammetry is now grounding dimenapplications. sional sional quality quality assurance applications. Real -timephotogrammetry photogrammetry was was first first introduced introduced in in early early 70ies 70ies at at least least by the National Real-time rush The rush Pinkney. 2 ' 3 The and Pinkney.2'3 Kratky and for example by Kratky later reported for Research Council Council of of Canada and later Research single-camera the singlemicro-electronics evolution of modern micro -electronics has has further further improved the camera concepts concepts of of advantage of full advantage the full take the already take ones, which already those times those times to to more more general general and sophisticated ones, recent The recent simultaneously. The several cameras operating simultaneously. the use of several in the inhering in photogrammetry photogrammetry inhering by ElEl-Hakim example by for example outlined for is outlined systems is multi-camera these multiof these development of stage of the development camera systems -Hakim al. 4 .^ et. aí.4,5 Griin et. and Grün

Mapvision in aa origins in its origins has its It has system. It vision system. machine vision Mapvision is a purely photogrammetric machine Centre Research Centre the Technical Research at the image processing at three three years' years' research research program on digital image ConferSPIE Conferthe SPIE during the internationally published during first internationally of Finland. of Finland. The The prototype prototype system was was first version novel version The novel 1985. 6 The in 1985.6 Cannes, France in in Cannes, Measurements" in in Engineering Measurements" "Optics in ence on ence on "Optics in announced in was announced and was for quality assurance purposes became operational and already usable and already and usable for way. under way. are under using Mapvision are experiments using June Just now the the first reference experiment§ 1986. '> 8 Just June 1986.7,8 criteprimary critefollowing primary the following account the into account taking into designed by taking is systematically designed Mapvision is ria : ria: -

free remain free should remain cameras, should the cameras, i.e. the components, i.e. aquisition components, image aquisition The orientation of the image the due to the set-up camera set The camera measured. The be measured. to be objects to in relation to the objects in -up is is thus thus more due geometry. to the measuring geometry. overall manufacturing manufacturing and process specifications than due to and process overall

-

of-the-shelf availableofcommerciallyavailable on commercially mainly on based mainly be based should be system should The system the -shelf components. components. No be should be components should special components of high precision mechanical parts or other special manufacturing manufacturing of systems. real-time of real construction of the construction for the needed for -time photogrammetric systems.

-

should be system should and performing time specifications of the measuring system The The nominal nominal accuracy accuracy and the by the restricted by not be restricted should not individual application and should to each individual adjusted according according to adjusted itself. system itself. basic measuring system

-

applications to manufacturing applications The The measuring measuring system should be be easily adapted and equipped to thorough knowledge of thorough any need of should be and should and be also also maintained by process process personnel without any photogrammetry. of photogrammetry. science of and science art and of of the complicated art

stand-alone The operative stand -alone system system built built by by the the Technical Technical Research Research Centre of Finland is 1). The (Figure 1). constructed constructed specifically specifically for for potential potential application application development purposes (Figure fulfilling in fulfilling already shown its versatility in first practical first practical experiments experiments using using Mapvision Mapvision have have already criteria. these criteria.

Inspection and Measurement Automated Inspection 730 Automated Vol. 730 SPIE Vol. 210 //SPIE Measurement (1986) (1986)

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 12/06/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Mapvision

Figure 1. 1. Mapvision. Mapvision. System design design

Mapvision consists parts, namely one for for the image image aquisition and and one one for consists of two main parts, for the the image processing (Figure (Figure 2). image 2). Image aquisition. aquisition. Fundamentally Mapvision is is designed to make use of any Image any kind of of solid solid state video video cameras. In the the novel novel Mapvision Mapvision there there are are four four Hitachi Hitachi VK VK-M98E 50Hz black state cameras. In -M98E 50Hz black and and white cameras cameras connected to the system. system. Each camera camera contains contains a MOS imager imager chip chip with with an an array array 577 by 388 388 horizontally horizontally resp. resp. vertically of 577 vertically oriented oriented image image elements. elements. The are synchronized with an external The cameras cameras are external pulse generator and and are are operated operated simultasimultain two two pairs. each pair the neously in pairs. Within each the two two cameras cameras are mutually locked locked to to each each other other so so that the the system system has has the the ability ability to to measure measure details details also also in moving objects. that in moving objects. On the contrary contrary the the camera camera pairs pairs have have aa difference difference of of 40 milliseconds in 40 milliseconds in relation to to their timing timing for for frame frame grabbing. All the the cameras are further grabbing. All further synchronized synchronized to to the image image digitizers digitizers and and the the procesprocessor.

Image processing. Image processing. The image image processing processing is is performed performed by by an an especially especially programmed programmed photo photogrammetric microprocessor microprocessor system. The modules modules of of the the processing processing systems grammetric system. The systems are: are: -

The computer (Intel (Intel SYS SYS 310 310-17C) The host computer -17C) -

-

CPU: 8 Mhz 80286 NDP: 8 Mhz 80287 RAM: 1 MB winchester: 19 Winchester: 19 MB floppy: 320 kB floppy: operating operating system: system: BMX RMX 286 286 -286, PL /M -286, ASM286 ASM286 (assembler) compilers: Fortran Fortran-286, PL/M-286, (assembler) bus: bus: Multibus II

Video digitizers digitizers (Matrox) (Matrox) -

-

1. x 512 512 x 8 bit 1. MIP-1024M, MIP- 1024M, 44 image image memories memories of of 512 512 x 2. MIP-1024M, MIP- 1024M, 44 image image memories memories of of 512 512 xx 512 2. 512 x 8 bit image memory of 512 3. MIP-512S, 512 xx 512 512 xx 88 bit bit 3. MIP -512S, 11 image LUT-functions, processors: LUT -functions, hardware hardware ALU fo -time execution execution of of logical logical and fo real real-time and arithmetic operations

SPIE Vol. SPIE Vol. 730 730Automated Automated Inspection Inspectionand andMeasurement Measurement (1986) (1986)// 211 211


space the object in the In the the surveying surveying phase, phase, the the details details to to be be measured measured on on the the object object or or in object space In for is used for laser beam is targets. The specific targets. are signalized with with a laser beam or or with with specific The laser first signalized are first should geometry should the whole the case in the surface in the object projecting discrete discrete spots object surface case the whole object geometry spots on the projecting identithen identiare then signals are object details. distinct object digitized. The The targets targets are details. The The signals are used for distinct be digitized. al coordinates three-dimension the three and the camera and each camera of each arrays of imager arrays the imager fied and and located -dimensional coordinates in the located in fied parameters. calibrated transformation parameters. the calibrated using the coordinates using image coordinates are intersected intersected from the image are data at some preliminary data are some seconds. If is 1.5 seconds. The measuring measuring time time for one If there there are at one point is The For reduced. For be reduced. the measuring time will signals, the will be the image coordinates of the signals, hand for the example the the measuring measuring of of an an averaged averaged observation observation with with ten ten repeated repeated pointings pointings would would take take only only example seconds. 3.6 seconds.

d

_ i

I` - - - -

I .-.

I

I

I Mapvision

C"\

xyz, xyz,

calibrated set up, system. First, Figure 3. The functions functions of of the the Mapvision Mapvision system. First, the the system is is set up, calibrated 3. The Figure al coordinates three-dimension to three two- to from twoand the the transformation parameters from -dimensional coordinates are are solved solved and using XYZ-control XYZ- controlpoints points (a). (a). Under Under the the surveying surveying phase phase the the new new points points to to be be measured measured on on the the using identified and (b). Then object space space are are projected projected onto onto the the image image planes planes (b). Then the the points points are are identified object the points are of the al coordinates of three-dimension the three finally the and finally located on on the the images (c) -dimensional (c) and located (d). parameters (d). calculated using using the the calibrated transformation parameters calculated 213 SPIE Vol. 730 730Automated Automated Inspection Inspectionand andMeasurement Measurement(1986) (1986)// 213 SPIEVol.


space the object in the In the the surveying surveying phase, phase, the the details details to to be be measured measured on on the the object object or or in object space In for is used for laser beam is targets. The specific targets. are signalized with with a laser beam or or with with specific The laser first signalized are first should geometry should the whole the case in the surface in the object projecting discrete discrete spots object surface case the whole object geometry spots on the projecting identithen identiare then signals are object details. distinct object digitized. The The targets targets are details. The The signals are used for distinct be digitized. al coordinates three-dimension the three and the camera and each camera of each arrays of imager arrays the imager fied and and located -dimensional coordinates in the located in fied parameters. calibrated transformation parameters. the calibrated using the coordinates using image coordinates are intersected intersected from the image are data at some preliminary data are some seconds. If is 1.5 seconds. The measuring measuring time time for one If there there are at one point is The For reduced. For be reduced. the measuring time will signals, the will be the image coordinates of the signals, hand for the example the the measuring measuring of of an an averaged averaged observation observation with with ten ten repeated repeated pointings pointings would would take take only only example seconds. 3.6 seconds.

d

_ i

I` - - - -

I .-.

I

I

I Mapvision

C"\

xyz, xyz,

calibrated set up, system. First, Figure 3. The functions functions of of the the Mapvision Mapvision system. First, the the system is is set up, calibrated 3. The Figure al coordinates three-dimension to three two- to from twoand the the transformation parameters from -dimensional coordinates are are solved solved and using XYZ-control XYZ- controlpoints points (a). (a). Under Under the the surveying surveying phase phase the the new new points points to to be be measured measured on on the the using identified and (b). Then object space space are are projected projected onto onto the the image image planes planes (b). Then the the points points are are identified object the points are of the al coordinates of three-dimension the three finally the and finally located on on the the images (c) -dimensional (c) and located (d). parameters (d). calculated using using the the calibrated transformation parameters calculated 213 SPIE Vol. 730 730Automated Automated Inspection Inspectionand andMeasurement Measurement(1986) (1986)// 213 SPIEVol.


is Mapvision. The basic mode is operate with Mapvision. four different modes There There are are presently presently four modes to operate of kind of different kind for different mainly for for research and demonstration purposes and the other three for potenfurther potenof further interactive development of for the interactive is also used for applications. The basic mode is applications. The real-time management, real image management, for image operations for several operations tial application modes. modes. It -time proproincludes several It includes tial hardcopy. and hardcopy. processing, measuring, image processing, cessing measuring, output and images, general image input images, of input cessing of are: modes are: application modes The present application repeated for pointwise pointwise measuring measuring of of signalized signalized object object details details by by using using single or repeated 11 for averaging with seconds. When averaging 2.7 seconds. is 2.7 observations. The The performing performing time for one measurement measurement is observations. seconds. Also aa 33 xx 33 conis 4.9 seconds. time in this ten observations observations the the total total measuring time this mode mode is ten seconds. 2.8 seconds. is 2.8 one measurement is for one time for total time the total volution might might be be used. used. Then the volution

-

No.

-

No.

-

beam. laser beam. scanning laser by aa scanning signalized by surfaces signalized object surfaces for continuous continuous digitization of object No. 33 for seconds. 1.5 seconds. is 1.5 interval is sample interval The sample

The perbeam. The laser beam. surfaces signalized by the laser object surfaces 2 pointwise measuring of object for pointwise 2 for seconds. 1.5 seconds. is 1.5 forming time for one measurement is forming time

operand opernew and to new respect to with respect further modified with The selection selection of application modes will be further ative applications. as aa defined as was defined Mapvision was for Mapvision Specifications. accuracy primarily designed for Specifications. The measuring accuracy 0.1 resolution of 0.1 image resolution an image relative and was was 1:5,000. This accuracy corresponds to an 1:5,000. This accuracy and relative accuracy in' each of the three pixels and and results results in better position accuracy than 1 1 mm in 55 meters in'each in a better pixels shown to be single measurement has practically shown coordinate directions. The The uncertainty of a single coordinate directions. explained by is explained even better better and and is is presently presently about about 0.02 0.02 pixels. pixels. This This resolution resolution enhancement enhancement is by even intersection. cameras simultaneously for the intersection. four cameras the high redundancy redundancy when using four the high mainly are mainly and are itself and limited by the Mapvision itself not limited are principally not The measuring ranges are conditions. working conditions. overall working and overall specifications and due due to the camera specifications Applications real-time As a three -dimensional measuring measuring domain the real -time photogrammetry photogrammetry intervenes the gap three-dimensional Mapvision Generally, Mapvision systems. Generally, theodolite systems. digital theodolite and digital between coordinate coordinate measuring machines and is aa non-contact non -contactand andoptical optical measuring measuring device device owing owing aa high high automation automation potential potential in in closing closing is as-built the bidirectional information information loop between designed and as -built data data bases bases of of modern as-designed between asbi-directional applicaactual applicacomputer systems. Introductory there are now two actual computer integrated manufacturing systems. namely dimensional is used for automated inspection and measurement, tions measurement, namely tions where Mapvision is still under are still earlier the reference experiments are and quality control quality assurance. assurance. As As stated earlier control and way. way quick the quick is the control is dimensional control Dimensional control. control. The preliminary application within dimensional out by check of of symmetry symmetry deformations deformations of of airplanes. airplanes. The The rigging rigging checks checks are are carried carried out by measuring measuring check structure. The comparison diagonal check points of the body structure. symmetry distances between diagonal symmetry distances regular intervals or occasionally after hard repeated at regular measurements will will be be individually repeated measurements landings. experiments have been first experiments the first Quality assurance. assurance. Within quality assurance applications the semi-automatic develope aa semi to develope is to aim is The aim done -automatic processes. The manufacturing processes. and manufacturing ship building and done for ship elements. body elements. ship body on -site inspection inspection system system for for verification verification of curved ship on-site distinct some distinct in some used in suitably used to be suitably already proved to outlook. Mapvision has already Applicational outlook. account into account taken into and actual practical practical applications. the Mapvision we have taken developing the applications. In developing and actual requirements. The whole applications with new requirements. that the system can be be further further used for new applications that the detection, robot qauging, flaw detection, applications including gauging, variety of of traditional traditional machine vision applications variety dimension. Because etc. are now becoming to third dimension. guidance, assembly control, control, surveillance surveillance etc. guidance, assembly aquisition image aquisition the image applications are more due to the the potential cameras the of the the use video cameras potential applications use of video high-speed like high applications like the applications Principally, the systems than than to the Mapvision itself. -speed or or itself. Principally, to the systems reach. close reach. relatively close within aa relatively also within are also underwater photogrammetry are Conclusions

real-time the real The developing of the -time photogrammetric photogrammetric systems systems at at the the Technical Technical Research Research Centre Centre obstacles today technical obstacles stage. There are no technical first maturity stage. the first of Finland has has now reached the vision. The to apply apply photogrammetry photogrammetry to to aa really really fascinating fascinating extent extent to to metric metric machine machine vision. The accuaccuto real-timeness 1:5,000, the real racy racy limit limit of of 1:5,000, -timeness of of about about one one second second and and the the applications applications for for autoautothis paper are to be considered only as pracin this mated inspection inspection and measurement dealt with in steadily afthem and tical milestones milestones on on this this way. way. The The real real challenges challenges lie lie far far beyond beyond them and are are steadily aftical electro-optical say electroto say better to or better electronic, or of electronic, fected by fected by the rapid development of optical component component technology. and processing technology. (1986) Measurement (1986) and Measurement Inspection and AutomatedInspection 730Automated 214 //SPIEVol. SPIE Vol. 730


References 1. 1.

1985. pp. 77-15. 5/1985, pp. No 5/1985, Steklasa, R., -15. 1985. Dimension, No Science Dimension, See," Science That See," "Machines That R., "Machines Steklasa,

2.

Cont.rol," Three-Dimensional Dynamic Three of Dynamic Kratky, -Time Photogrammetric Photogrammetric Support of -Dimensional Control," "Real-Time V., "Real Kratky, V., 1979. 1231-1242. pp. 1231 9, pp. No 9, 45, No Vol 45, Sensing, Vol Photogrammetric Engineering -1242. 1979. Remote Sensing, and Remote Engineering and

3. 3.

On-Line an On of an F. L., -Line 30 30 Hz Hz Video Video Photogrammetry Photogrammetry Development of and Development "Theory and L., "Theory H. F. Pinkney, H. Pinkney, -Time 3-Dimensional 3- DimensionalControl," Control," International International Archives Archives of of Photogrammetry, Photogrammetry, Real-Time for Real System for 1978. pages. 1978. 38 pages. V.2, 38 Part V.2, XXII, Part Vol. XXII, Vol.

4.

El- Hakim, S.S. F., F., "A "A Real-Time Real -TimeSystem System for forObject Object Measurement Measurement with with CCD CCD Cameras," Cameras," InternaEl-Hakim, 363-373. pp. 363 5, pp. Part 5, 26, Part Vol. 26, tional Archives of Photogrammetry -373. Sensing, Vol. Remote Sensing, and Remote Photogrammetry and tional 1986.

5. 5.

"Real-Time A., "Real H. A., Beyer, H. Grün, A. A. W. W. and Beyer, -Time Photogrammetry Photogrammetry at at the the Digital Digital Photogrammetric Griin, Real-Time on Real symposium on ISPRS symposium the ISPRS at the presented at paper presented -Time Zurich," aa paper ETH Zürich," (DIPS) of ETH Station (DIPS) 1986. 16-19, June 16 Ottawa, June -19, 1986, 1986, 14 14 pages. pages. 1986. Challenge, Ottawa, New Challenge, Photogrammetry -- A New

6. 6.

real-time " Photogrammetric prototype prototype system system for real -time engineering engineering applications," H., "Photogrammetric Haggren, Haggrén, H., 1986. 330-335. pp. 330 599, pp. Vol. 599, Proceedings, Vol. SPIE Proceedings, Optics in -335. 1986. Measurement, SPIE Engineering Measurement, in Engineering

7. 7.

RealApplications,"Real Vision Applications," Machine Vision for Machine Used for as Used Haggrén, -Time Photogrammetry as "Real-Time H., "Real Haggren, H., and Photogrammetry and of Photogrammetry International Archives of Challenge, International Time Photogrammetry - A New Challenge, 1986. 374-382. Remote Sensing, -382. 1986. pp. 374 5, pp. Part 5, 26, Part Vol. 26, Sensing, Vol.

8. 8.

for 33-D Cameras for State Cameras Solid State of Solid Leikas, E., -D Measuring," Digital Use of the Use in the "Experiences in E., "Experiences Leikas, be to be Workshop, to IFAC Workshop, the IFAC Image Processing in Industrial Applications, Applications, Proceedings of the Image 1986. pages. 1986. Oxford, 66 pages. Ltd., Oxford, published by Pergamon Books Ltd., Pergamon Books

986) // 215 SPIE SPIE Vol. Vol 730 Automated Automated Inspection Inspection and andMeasurement^ Measurement (1986)
