POLLACK PERIODICA An International Journal for Engineering and Information Sciences DOI: 10.1556/Pollack.5.2010.2.5 Vol. 5, No. 2, pp. 49–56 (2010) www.akademiai.com
DEVELOPING A WEB BASED PHOTOGRAMMETRY SOFTWARE USING DLT Bence MOLNÁR Budapest University of Technology and Economics Department Photogrammetry and Geoinformatics, Budapest, Hungary e-mail:
[email protected]
Received 1 January 2010; accepted 17 February 2010
Abstract: With the appearance of digital cameras photogrammetry entered a new era. Simple digital cameras emerged and new methods of three-dimensional measurements became available (e.g. lasers-scanner) in addition to the basic methods of photogrammetry. Cost plays major role in development of new methods. At the same time new user requirements emerge: applying commercial digital cameras and limited knowledge need about the technology. With a new, web based photogrammetry application the number of users can be increased. If the cameras are not calibrated, Direct Linear Transformation is a well adaptable calculating method. To enable robust processing, Huber method is an efficient way to eliminate gross errors. The paper discusses the development of the WebDLT application from the planning through the realization to the accuracy issues. Keywords: Photogrammetry, Direct Linear Transformation, Huber method, Web application
1. Introduction Nowadays everybody has a digital camera. Since these cameras are widely used and became even more affordable, they are used for more and more tasks. For example many people take photos before buying a real estate. The decision can be supported by the help of the photos. It would be beneficial, if they could perform simple measurements or even draw a map, without visiting the real estate again. Photogrammetry provides a good solution for this claim; it allows performing spatial measurements based on photos. Classical photogrametry applies expensive equipment, thus it was used only by few professionals. Technical and computing development
HU ISSN 1788–1994 © 2010 Akadémiai Kiadó, Budapest
50
B. MOLNÁR
broadened the application field of photogrammetry. One missing component for largescale usage is the inexpensive and simple post-processing software. In order to fulfill this demand photogrammetric software operating in a web browser was developed. With this application the user is able to make spatial measurements based on photos. In this paper the method of photogrammetric calculation, filtering gross errors made by inexperienced users and the technical background of the application are discussed. For calculations due to uncalibrated cameras Direct Linear Transformation (DLT) was used. Earlier attempts were published for Internet based photogrammetry [1], the presented WebDLT provides solutions for several shortcomings of these applications.
2. Photogrammetry It often happens to engineers that they have to record geometric data quickly and the measurements cannot be performed directly on the object. In this case, photogrammetry provides good solution, which means performing spatial measurements on 2D photos. The main disadvantage of photography is the dimension loss by the projection of the 3D reality into a planar photo. Central projection causes several distortions that have to be corrected on the photos. Because of the rapid data acquisition and the application of commercial digital cameras, it is a reasonable way to perform spatial measurements based on photos. To measure in 3D, two or more photos are needed from different camera stations (like the eyes in human vision). If camera positions and beams can be reconstructed with high accuracy, the intersections of the beams, therefore the original spatial coordinates of the points can be calculated (Fig. 1). Since all beams go through the projection center, if an object is far from the camera, the points of it will appear relatively closer on the projected images, as shown in Fig. 2. The determination of unknown points is possible if some control points are available. In case of control points both the spatial and image coordinates are known. They represent the connection between the images’ and the real objects’ coordinate systems. Photogrammetry has been used for a long time; the first calculations have been based only on the geometry of photography. The camera stations (position, orientation) were reconstructed, mainly with analogue methods. It is only feasible, if the projection center is known in the image coordinate system with high accuracy; this was only available using costly, special cameras. In addition, the analogue post-processing instruments were also expensive, and operation needed professional users and special skills. These instruments separately projected the image pairs in the eyes of the evaluating person enabling the spatial measurements. These equipment were expensive that limited the application fields. Photogrammetry entered in a new era. This is due to technological development (computers and cameras) and the evolution of the mathematical model describing the physical model. Computing revolution and the development of digital cameras also had major effect. Because of the recent computer’s processing capacity there is almost no limitation to solve equations, thus analogue instruments can be replaced by computers executing complex mathematical calculations. Theoretical photogrammetric equations Pollack Periodica 5, 2010, 2
WEB BASED PHOTOGRAMMETRY SOFTWARE USING DLT
51
can be solved even with a simple PC. As real-time 3D representation on computer can be problematic, image coordinate measurements can be performed separately on the photos. Digital cameras enabled simpler image coordinate measurements. The mathematical models are continuously developing; therefore the special photogrammetric cameras can be replaced by simple, commercial ones. One of these mathematical models is the Direct Linear Transformation.
Fig. 1. Projection of two photos in digital photogrammetry
Fig. 2. Central projection
Pollack Periodica 5, 2010, 2
52
B. MOLNÁR
3. Direct linear transformation - DLT The Direct Linear Transformation describes direct connection between image coordinates and coordinates on the object side. Therefore the interior and exterior orientation parameters of the camera stations are not required, i.e. no calibrated camera is needed. The method uses 11 (not independent) parameters to describe the connection [2], [3], L X + L2Y + L3 Z + L4 , x= 1 L9 X + L10Y + L11Z + 1
(1)
L X + L6Y + L7 Z + L8 y= 5 , L9 X + L10Y + L11Z + 1
(2)
where x,y are image coordinates, X,Y,Z are object space coordinates and Li, i = 1,2, L ,11 are DLT parameters. The main advantage of this formula is that the calculations can be done by solving linear equations that leads to fast and efficient processing. The calculation can be done involving Least Squares Method [4]. First the 11 DLT parameters have to be calculated for each photos using the control points. If all the parameters are known, unknown points can be calculated v PLL v = min .
(3)
Recently, photogrammetry has been changed a lot. In addition, various measurement methods became available on the market for the same or similar tasks e.g. laser scanner. To remain competitive, photogrammetric solutions have to be affordable and easy to use. Currently everything is given, but the inexpensive and user-friendly application. If software fulfilling these requirements it would be available, lot of new photogrammetric users would appear using commercial digital camera and PC. As a practical example, prior to buying a real estate, the buyer can create a simple map and carry out some basic measurements; a digital camera and a computer are the only required equipment.
4. WebDLT More and more office applications are available over the Internet, in a simple web browser. This is because people are getting more and more mobile, and broadband Internet is available almost everywhere. The advantage of web applications is that no program installation is needed and working is possible everywhere by a regular user interface. Similarly, with a web based photogrammetric application, photogrammetry could be used in much wider scale. Therefore a web based photogrammetric application was created.
Pollack Periodica 5, 2010, 2
WEB BASED PHOTOGRAMMETRY SOFTWARE USING DLT
53
Obviously during the calculations, supporting inexperienced users also have to be considered. They can cause gross errors during image coordinate measurements that should be automatically filtered out. The inexperienced users cannot perform high accuracy measurements, like in classical photogrammetry, however they probably do not need high accurate results. Of course, experienced users are able to carry out precise measurements; the mathematical calculation does not have any wrong effect on accuracy. Other remarkable advantage of web applications is that they do not need to be upgraded on the client side; the user could use always the newest version. Furthermore, the potential user should not have special rights on the computer; no extra plug-ins and add-ons are needed to the browser. Since the jobs are stored on a remote server, the work can be always continued at another location. Advantages and limitations are summarized in Table I. Table I
Advantages and limitations of WebDLT Advantages No limit of operating system (multi-platform) No need of software installation No specific rights are needed No need to upgrade, always the newest version is available Location and time independent Regular user interface Inexpensive
Limitations Broadband internet connection required Large data amount causes high network traffic Browser limitations
5. Planning and realization Prior to the realization of WebDLT, the application has to be carefully planned. User demands and the limitations according to web realization have to be considered. The application should be independent of computer architecture, operating system and browser. To achieve these goals, all web standards must be fulfilled. Photos typically mean huge amount of data, the transfer of photos through network is limited and bandwidth-dependent. Therefore network traffic must be planned, and the necessary jobs must be shared between client and server computer. Images should be tiled, and network traffic should run in the background. To keep the project on low budget, it was an important issue to develop the entire application using free and open source programs. These kinds of software are rapidly developed, errors (bugs) are quickly corrected, source can be edited and free to use, and they are free enabling the wide scale usage. All parts of WebDLT are based on open source projects. The realization is divided into two main parts. The first is the
Pollack Periodica 5, 2010, 2
54
B. MOLNÁR
photogrammetric calculations; the other one is the web user interface (including the digitizing module). The DLT calculations are made by Octave mathematical programming language. It is very similar to the well-known MATLAB that is specialized for matrix operations. Calculations have two steps, first the DLT parameters need to be calculated, and then the unknown point coordinates should be calculated. The measurements contain errors and inconsistency to each other; therefore Least Square Estimation is used. This method is very sensitive to gross errors; to minimize the effect of gross errors, WebDLT uses Huber method [5]. It declares that gross errors do not have definitely normal distribution, and takes a threshold (based on errors), where distribution becomes different ⎧ x, ⎪ ψ ( x) = ⎨ x a , ⎪ x ⎩
x ≤ a,
(4)
x > a,
where a is the threshold. The method modifies the weight matrix in iteration steps; gross errors will have approximately zero weights as shown in Fig. 3.
Influence function 4 3 2 1 0 -4
-3
-2
-1
-1
0
1
-2
2
3
4
Normal distribution
-3
Huber function
-4
⎡1 ⎢0 ⎢ ⎢0 ⎢ 0 P=⎢ ⎢0 ⎢ ⎢0 ⎢0 ⎢ ⎢⎣0
0 1 0 0 0 0 0 0
0 0 1 0 0 0 0 0
0 0 0 0 0 0 0 0 0 1 0 0 0 0.008 0 0 0 0.011 0 0 0 0 0 0
0 0 0 0 0 0 1 0
0⎤ 0⎥⎥ 0⎥ ⎥ 0⎥ 0⎥ ⎥ 0⎥ 0⎥ ⎥ 1⎥⎦
Fig. 3. Huber function and the final weight matrix
With Huber method, the estimation will become robust. Based on the final weight matrix and the root mean squares, the final accuracy can be determined. During the calculations only linear equations should be solved, i.e. the process is very fast and efficient. No linearization is needed and iteration is only used for re-weight the measurements. The user interface (Fig. 4) is also developed with open source software. On the server side PHP web programming language was used to generate web pages. The database backend is MYSQL. On client side the visualization is made by HTML and
Pollack Periodica 5, 2010, 2
WEB BASED PHOTOGRAMMETRY SOFTWARE USING DLT
55
JavaScript. The client side applies W3C web standards; therefore all browsers (e.g. Firefox, Explorer 7+, Opera, Chrome) will display the pages in the same format. Because of the bandwidth limitations, the photo transfer must be carefully planned [6]. First of all, the photos are to be cut into tiles; the user interface can be fast and geometrically reliable (e.g. in handling images). All image tiles are stored in browser cache transferred on network once. The upload method of 4 5MB photos takes in an average home connection in Hungary about 2 minute. The client downloads the tiles in the background to save time. In the digitizing module there is a big place to process digitizing, the other photos in current project are visible on the right side, and an enlarged part of currently digitized point on other photos.
Fig. 4. Digitizing module
6. Accuracy The final accuracy of measurements depends on multiple factors. An important one is the accuracy of control points’ spatial coordinates. The critical factor in WebDLT’s aspect is the digitizing accuracy. In digital photogrammetry sub-pixel digitizing is used to increase accuracy. It means that all pixels are divided to more pixels, and intensity is calculated for all sub-pixels depending on the neighbor pixels’ intensity values. Modern browsers (e.g. Firefox) use this method if a photo is zoomed: WebDLT uses this method if available in browser. The digitizing gross errors are filtered as described above. As mentioned before, the user’s experience is also an important factor. It also covers the photogrammetric network planning, i.e. the determination of camera positions and control point positions. The accuracy also depends on the quality of photos, the important factors are: geometric and radiometric resolution, optical distortion, and sharpness.
Pollack Periodica 5, 2010, 2
56
B. MOLNÁR
7. Conclusions With WebDLT the number of potential photogrammetric users can be increased. Users only need to have a simple digital camera and a computer with a web browser and broadband Internet connection. Measurements can be performed anytime all over the world, since user data are stored on server. Users can work fast and be effective due to well-planned data transformation and calculation methods. Accuracy depends on users’ experience, but the available accuracy is comparable to that of other photogrammetric methods. The program combines DLT, Least Squares Estimation and Huber method to get the most accurate result for low experienced users. The developed photogrammetric tool is free to use, enables rapid, efficient and accurate processing. Since the application is readily accessible and cheap, it can be useful in education too. It is already used for photogrammetry education at Budapest University of Technology and Economics. The web based software can be reached at: http://dlt.fmt.bme.hu.
References [1] [2] [3] [4] [5] [6]
Grussenmeyer P., Drap P. Possibilites and limits of Web photgrammetry - Experiences with the ARPENTEUR web based tool, Photogrammetric Week 01, Wichmann Verlag, Heidelberg, 2001. Abdel-Aziz Y. I., Karara H. M. Direct linear transformation from comparator coordinates into object space coordinates, ASP Symposium on Close-Range Photogrammtery, Falls Church, VA, 1971, pp. 1–18. Fekete K. Close range photogrammetry and some medical practice, (in Hungarian), PhD Thesis, Budapest university of Technology and Economics, Budapest, 2006. Detrekői Á. Adjustment calculation, (in Hungarian), Tankönyvkiadó, Budapest, 1991. Huber P. J. Robust statistics, John Wiley, 1981. Cârstea A., Macariu G. Towards a grid enabled symbolic computation architecture, Pollack Periodica, Vol. 3, No. 2, 2008, pp. 15–26.
Pollack Periodica 5, 2010, 2
