Developing Specialised Software for Real Estate ... - CiteSeerX

Pacific Rim Real Estate Society Conference, 23rd-25th January 1996

Page 1

Developing Specialised Software for Real Estate Applications A Case Study of the Development of UPmarket Paul J. Kershaw, M.Sc., Principal Lecturer Robin R. Kooymans, B.App.Sc.(PRM), Grad.Dip.Ec.(NE), AVLE(Ec), AAII, Lecturer Peter A. Rossini, B.App.Sc.(Val), Grad.Dip.Econ.Stats.(NE), Lecturer

S CHOOL OF E CONOMICS , F INANCE AND P ROPERTY F ACULTY OF B USINESS AND M ANAGEMENT U NIVERSITY OF S OUTH A USTRALIA

Abstract This paper examines the development of a software package which is used for property transactions, data retrieval and analysis in South Australia. This software was first demonstrated at the 2nd Australasian Real Estate Educators’ Conference in Adelaide in 1992 in the authors’ paper titled “Microcomputer Based Real Estate Decision Making and Information Management - an Integrated Approach”. While the concepts of the system were popular with potential users, the process of producing the software as a full working Windows program and the difficulties of co-ordinating the access to data and marketing and distribution have been great. These difficulties are examined and useful suggestions provided for other potential software developers

Introduction At the 2nd Australasian Real Estate Educators’ Conference in Adelaide in 1992 in the authors’ paper titled “Micro-computer Based Real Estate Decision Making and Information Management - an Integrated Approach”, it was argued that the foundation of informed Real Estate decision-making is the availability of data in an easily accessible and usable format. Our premise was, and still is, that without access to data immediately it is needed, and in the format in which it is needed, the bestintentioned of analysts in possession of the most sophisticated analysis tools will be defeated by deadlines, costs and the sheer tedium of data collection. To support our contention that it is possible to achieve ready access to data and to analyse it in a relatively short time frame, the DOS version of UPmarket was demonstrated. This version had been two years in development, although that development took place very much on a part-time basis. Since this paper was presented UPmarket has been re-written as a Windows system, has been released for sale and is now being used by real estate professionals in South Australia for their sales retrieval and analysis needs. The process of moving from an experimental version to a release version which has been taken by a significant number of users has been a difficult one. In this paper the authors detail how Upmarket has progressed from the working version in 1992 to the market release in 1995. The future of UPmarket is also examined.

What is UPmarket UPmarket is an integrated system to retrieve property transaction data, provide direct analysis and a variety of reporting formats. Data comes directly from the Department of Environment and Natural Resources (DENR) each month and includes sales settled up to the end of that month. Data is preprocessed and distributed to clients within 5 working days so that users always have access to recent sales. Data is stored in a highly compressed format so that each year’s data for the State of South Australia occupies about 10 megabytes of hard disk space. (Average 50,000 sales per year with some 47 fields). The retrieval system is Windows based and provides very fast search times with up to 30 search parameters. Typical searches take one or two seconds. Searching can be done on eight different locational parameters (LGA, Street, Postcode, Country Towns, Regions, Street, Map Reference and valuation number references). Similarly, searches can be filtered on land use, land area, price, date of sale, zone and on residential characteristics such as building area, condition, age, style and cladding.


Page 2

Records are reported to a brief details screen where they can be sorted, deleted and tagged. The record can then be viewed in full, a statistical summary viewed or analysis carried out on the data. Results can be saved to disk or printed in a number of formats. UPmarket comes with full help facilities, a tour quide and onscreen decoding and glossary facilities. The first release version of the software was released in June 1995 through the Real Estate Institute of South Australia. The current experimental version includes a wider range of sales and time series analysis tools. Figure 1 shows a Upmarket search and parameters screen. Figure 2 shows the full record listing. Figure 1

Figure 2

Page 3


The Need for Speed As soon as an Anaytical Engine exists, it will necessarily guide the future course of science. Whenever any result is sought by its aid, the question will then arise - by what source of calculation can these results be arrived at by the machine in the shortest time? Charles Babbage 1864 (as quoted in Knuth 1973)

One of the most significant design criteria of Upmarket was the need for speed. In order to make retrieval, analysis and reporting effective, the system would need to deliver results in real time in personal computers. Similarly the data base must sit on that computer. The original DOS version of UPmarket was developed on a 386 machine when hard disk space was expensive. While this space is now cheaper and processors are faster, there are still big advantages if the system is lighting fast and uses only small amount of space. The database of South Australian property sales comprises some 50,000 records per annum with a single record having a length of 580 characters. To store the whole series of 25 years would require some 725 million bytes. Indexing on, say 20 fields, adds another 200 million bytes. Whilst most users would only require access to the prevoius three years worth of data this still produces a searchable file of 111 million bytes Early experiments quickly showed that proprietary databases were to slow and produced files that were too large for the computers currently in use by the client market. Proprietary databases, necessity have to be a compromise between speed and usability and offer a wide range of features that are redundant in a project of this type. These include record editing, deletion, addition and repair. It was also considered necessary that UPmarket work speedily over a network. Speed over a network is inversely proportional to the size of the data flow. The decision was made to construct our own database engine that would address the specific needs of the project. Experiments showed that optimal performance could be achieved by compressing the data, performing pre-indexing and choosing the most efficient algorithms. The database was compressed to 34 million bytes for a three year holding of data ( a compression ratio of approximately 3:1) and work on searching , sorting and file handling routines carried out. An indication of the savings that can be made are presented in the following tables and charts. Sorting Sorting is an essential element in determining the median price and the correct choice of routine brings immense savings. Various algorithums are available. Three sort routines were trialed on typical searches utilising different numbers of records. Results of these trials are shown.

Time

1000.00

Shell Sort QuickSort

900.00

50

3.23

1.59

20.83

100

12.31

4.68

21.09

800.00

B u b b le S o r t

150

29.06

7.32

21.28

700.00

S h e ll S o r t

200

51.86

11.57

22.24

250

81.97

13.98

22.24

300

118.55

18.01

22.85

500.00

350

161.95

21.80

22.80

400.00

400

210.40

31.71

22.80

500

334.04

35.93

23.91

600

480.46

48.26

24.98

200.00

700

658.26

56.20

25.95

100.00

800

861.78

83.22

27.27

900

1092.51

93.63

27.65

1000

1354.30

103.40

29.07

Q u ic k S o r t

600.00

300.00

500

350

250

150

50

0.00 900

Bubble Sort

700

Table 1 Records

Page 4


Time

1200

Records

Shell Sort

QuickSort

1500

78.16406

16.73438

2000

122.8789

18.97656

2500

169.9102

21.82031

3000

200.5859

24.27344

3500

249.5547

26.86719

4000

316.4688

29.59375

5000

423.5195

35.21484

6000

511.9453

40.88672

7000

619.1523

46.51953

8000

830.0078

52.46094

9000

897.1758

58.25391

10000

1071.469

64.82031

1000

Shell Sort

800

QuickSort

600 400 200

10000

9000

8000

7000

6000

5000

4000

3500

3000

2500

1500

0 2000

Table 2

The conclusions that were drawn from the charts is that there is no overall optimum sorting algorithm and that tests need to be conducting over the whole range of possible requirements.However given the furture use of Upmarket for dealing with Time series data, the use of quick sort routines were considered most appropriate as very large volumes of data would need to be handled when working with any long series. File Handling The retrieval of data requires the manipulation of a large databases. This requires a large number of file routines. As with the sort handling various file handling routines could be used. Several of these were trialed the results shown below. Table 3

Seconds Sequential Input

Table 3 below gives specimen timings for handling a 512KB text file.

Time to load 512KB File Normalised

135.39

752.2

Sequential Line Input

3.95

21.9

Random Get

0.91

5.1

Binary Get

0.77

4.3

System Call

0.18

1

Table three demonstrates the effectivness of more complex file handling proceedures. Traditional sequential input proved to be just over 750 times slower than system call. These faster file handling systems have greatly added to UPmarkets speed and efficiency.

Speed of the Finished Product A series of complex searching, sorting and file handling routines makes Upmarket very quick. On a 486 DX 66 a search of the whole state using multiple search criteria for 5 years (250,000 records) takes less than 3 minutes. The same search for a large Local Government Area (LGA) (more than 2.5% of total sales) takes less than 13 seconds whilst a small LGA or suburb takes less than 4 seconds. Typical searches using multiple search criteria for several months is almost instantaneous.

The Development of UPmarket After the demonstration of the DOS version of UPmarket at the 1992 PRRES Conference, no further development was carried out on this version, despite the fact that it was fully operational and quite robust. The reason for this was largely a change in philosophy on the part of the development team, a change that was not reached without a good deal of debate. The change in philosophy arose for two related reasons, market focus and acceptance of technology. The following sections describe the process and the problems in moving from the experimental DOS version to the Windows release version described above.


Page 5

Market Focus While the DOS version of UPmarket was designed with an eye to its potential application in the property professions, it was first and foremost intended as the basis of analysis tools for use by the Property staff and students at the University of S.A. The University had access to the S.A. Department of Environment and Natural Resources’ property sales data in electronic format at a reasonable price, for use in teaching and research. The property professions had access to the same data on microfiche or in printed format at a reasonable price. In hindsight, it appears as if we initially somewhat vaguely and naively assumed that there would be a natural progression to provision of data to the property professions in electronic format at a reasonable price. At that point, the DOS version of UPmarket would be available to make the best use of the data, its introduction to the profession being led by graduates who had become familiar with it during their property studies at the University of S.A. The natural progression of DENR and the property professions to electronic data had not occurred, and did not look like occurring, at the time UPmarket was demonstrated in 1992. The reasons for DENR not pushing the change seemed to lie in a commitment to main-frame computer technology, with staff having a vested interest in going with the technology they knew, and a perceived lack of demand by the property industry. The reasons for property professionals themselves not demanding change are a little harder to gauge. Certainly we discovered a much higher than expected take-up of personal computers in real estate offices, both in the literature and in our own observations (Newell and MacFarlane, 1991; Hargreaves, 1994; McGregor Marketing Pty. Ltd. 1994; Lilburn, 1995), and many of the property professionals in S.A. had a good grounding in technology through the University of S.A. Property Courses. We suspect that there was a lack of appreciation of the potential advantages of being able to rapidly search desired sections of the sales data base on a wide range of criteria - not having seen such a system, it was hard for practitioners to imagine one. There was undoubtedly an element of inertia - “the present system works fine, so what is the problem?” Importantly, there was the cost element. At that time, business for valuers and land agents was not going well, with fee-cutting making it more difficult to run a viable business. The industry had had the recent and ongoing experience of the introduction of the Torrens Automated Title System (computerised land titles) which, as far as the average property professional was concerned, provided no more information at a much greater cost. The possible introduction of computerised sales data was perceived in much the same light. It became increasingly apparent that a catalyst was needed if we were serious about providing the property professions, not just ourselves, with modern, usable tools for property analysis. It was also apparent that we were going to have to be that catalyst and, in the process, shift the short-term focus of the whole UPmarket project to professional use. This was certainly in accordance with one of the main objectives of the University of S.A. - to work within the community and to disseminate the products of research as widely as possible.

Acceptance of Technology Reading of the literature (Newell and MacFarlane, 1991; Hargreaves, 1994; McGregor Marketing Pty. Ltd. 1994; Lilburn, 1995) and observation of the market for computer software led us to the inescapable conclusion that the future of personal computer technology lay with Windows-based products. It is easy to see with the benefit of four years’ further experience how Windows-based, mouse-operated software has almost completely supplanted its DOS-based keyboard-operated equivalents in business and home computing environments. In 1992, however, there were plenty of people willing to discredit Windows environments as being for computer-illiterate game players rather than for “serious” applications. The view we reached after a lot of consideration was that the era of proving of one’s “intelligence” by being able to operate a fiendishly complicated program and to memorise an illogical set of commands was nearly over. The professional of the future was going to judge a piece of software on its ability to do a job which could not be done before; on its contribution to a healthier bottom line and greater client satisfaction; on the ease with which it could be mastered and used. A lot of the criticisms of Windows-based programs, like being relatively slow and memory-hungry were quite justified. However, it was only necessary to look at the rapid development of personal computer hardware to realise that memory capacity and speed in relation to price would eliminate such problems. In any event, development of the first version of UPmarket had made us realise that you could nearly always find a better way to do things, and that Windows-based software was probably going to be no exception.


Page 6

There was a strong practical reason for going to a Windows environment, which in the event probably proved decisive. If we were going to market a program to the property industry at large, we had to be able to cater for all sorts of printers, screens and program interfaces. The chances were that no two office computer set-ups would be alike, and it was certain that new products would be released and sold faster than we could know, let alone develop an appropriate UPmarket interface. The Windows environment talks to virtually all available PC hardware and software and is upgraded to be compatible with new products as developed. By adopting the Windows environment for UPmarket, all these compatibility problems could be solved by Bill Gates’ people, freeing the development team to concentrate on the real work.

Market Research With the temporary switch of focus from use of UPmarket within the University for our own requirements, to use by property professionals, it was necessary to find out what the property profession wanted in a sales data base system. In the development of the DOS version, we had relied on the individual experiences of the development team in the property profession, the academic knowledge of our specialist disciplines and informal canvassing of people in the real estate industry that we knew well. For a system that people would actually buy and use, however, we required more detailed input. In this regard, the DOS version proved to be invaluable, because its demonstration gave property professionals something to focus on. We had proven that fast access to sales data by various criteria with minimal disk space requirements could be achieved, thus removing the initial scepticism. Property practitioners were then able to criticise aspects of the DOS version and the first Windows versions in terms of what they would like to see changed, added or deleted - an altogether more satisfactory process than asking people to imagine what they would like an imaginary sales data base system to do. As Windows versions were developed, potential users ranging from large firms with PC networks to individual valuers volunteered to act as evaluators of UPmarket, making suggestions and noting problems in a non-judgmental manner. This occurred both pre- and post-launch and has worked extremely well. At the same time, Windows versions of UPmarket have been included on the Staff and Student PC Pools at the University of S.A., with the aim of testing its robustness under everyday use. Development of UPmarket has taken into account suggestions made and problems found through this process, which has enhanced the confidence of system users.

Data Access When it came to marketing UPmarket to the professions, we had two related problems - marketing of the software and marketing of the data. Clearly, the software would be useless without up to date sales data in electronic format and the sales data in that format would be useless without a program to access it. The software was under our control, and we had access to the sales data for University purposes as mentioned previously. However, with the exception of one or two major real estate offices who received limited sales data on magnetic tape for use on mainframe computers, we were the only people receiving regular sales data in electronic format. That data was not ours to sell (or give away for that matter), so the crucial issue became the availability of sales data in electronic format. Although other States, notably Queensland, had made progress in making electronic sales data available to the property professions, South Australia’s Department of Environment and Natural Resources had made little progress in investigating the practicalities, and virtually none in determining a policy. Therefore, between mid-1992 and mid-1995, the development team had to mount a time consuming campaign to get the sales data available to the profession in electronic form at a viable price. In respect of things like multi-user licences negotiations are continuing. As indicated already, the property profession at large was generally satisfied with the mode of delivery by microfiche and suspicious of change because it would probably cost more money for no perceived advantage. The development team had to negotiate alone for a large part of the time, virtually on behalf of a reluctant profession. Compared with a nominal fee (less than $500 a year) for the microfiche, DENR originally proposed a fee in the order of $30,000 per annum for a year’s sales data (50,000 sales) in electronic form. By the development team researching the fees charged interstate, and the method of licensing and distribution, we eventually reached the present situation where property professionals are charged $1,200 per annum for a year’s sales for the State.


Page 7

Following the Queensland model, we proposed that they make the data available through a software development organisation with a licence agreement between the end data user and DENR and the licence fee to be collected and remitted by the software marketer. DENR now offer the price and arrangement we negotiated to any software developer. The negotiation process was very lengthy and time-consuming. One of DENR’s major problems was S.A. Public Service restructuring following the large public sector deficits incurred in meeting State financial guarantees for failed quasi-government financial institutions. These were compounded by the election of a new government in late 1993, leading to changes in personnel, structures and something of a paralysis in decision making. The involvement of the Real Estate Institute of South Australia in political lobbying during the latter stages of negotiations was most helpful in resolving what became the most difficult part of the whole project. It is probably fair to say that, had we known from the outset what we now know about negotiating the availability of sales data at a viable price, we would not have gone ahead with developing UPmarket for the profession. We would certainly counsel other developers to ensure the availability of data to their market before putting in too much development effort and, if negotiating with government, to enlist the aid of influential industry lobbies sooner rather than later.

Marketing and Distribution From the outset, we had to be very clear on our objectives for this particular part of the UPmarket project. Were we out to make our fortune, or were we carrying out our academic responsibilities in “spreading the gospel” of sound objective sales analysis to as many people within the property professions as possible? Very early in the piece, we concluded that the latter was the case, and that any returns would be ploughed back into the project, which is very much open-ended. In the event, DENR’s sales data pricing policy meant that the software had to be sold very cheaply indeed to ensure wide distribution. If one looks at the pricing of business software generally, it becomes apparent that pioneering into a very small market is no way to make a living. UPmarket’s development and marketing has been very much a “spare time” operation, on top of the development team’s normal teaching, administration and research duties and without the resources to purchase computer programming, administrative and research support. We did not intend to become part-time marketing and support people as well, so looked for an appropriate marketing body. As it happened, DENR had commenced supplying printed sales data for distribution through the REISA. REISA were impressed enough by UPmarket to offer to market it on our behalf and were, as indicated, helpful in concluding sales data price negotiations. As the REISA were in touch with virtually all real estate agencies in S.A. - the “mass” market for UPmarket - and had many years’ experience in marketing all kinds of goods and services to real estate agents, they seemed to be the appropriate body. Certainly, determining the identity of the market was the least of our problems. Despite our best intentions, it quickly became obvious that we would have to become involved in the marketing effort, particularly to major users, because we were the only ones who knew a lot about the product. REISA has a fairly efficient distribution set-up, although we had to become rather heavily involved in designing a secure and timely distribution of monthly data updates, and in training REISA staff in its operation. The distribution seems to have settled down quite well. The biggest ongoing bugbear, and one which all intending software developers need to take into account is support. We were persuaded to take on a company which supports some other REISAdistributed software, after they agreed to modify their original fee demands which would have seen them taking all the income from the software sales. This has not been the most productive relationship. The lesson is to assume nothing about your support, except that it will consume more of your time than you expect it to.

Future Developments - Direct Analysis


Page 8


Page 9

References Hargreaves R.V.: Does the Computer Have a Place in Your Office? New Zealand Valuation Journal, March 1994 Kershaw P.J., Rossini,P.A, Kooymans, R.R., Direct Real Estate Analysis - The UPmarket Approach to Real Estate Decision Making. 3rd Australasian Real Estate Educators Conference, Sydney, 1993 Kershaw P.J.,Rossini,P.A, Kooymans, R.R., Micro-Computer Based Real Estate Decision Making and Information Management an Integrated Approach. 2nd Australasian Real Estate Educators Conference, Adelaide, 1992 Knuth,D.E. The Art of Computer Programming - Volume 3 - Sorting and Searching, AddisonWesley 1973 Lilburn, I.D.: Software and its Use in the Real Estate Industry. Unpublished student paper, University of South Australia, Faculty of Business and Management Resource Centre, December, 1995 McGregor Marketing Pty. Ltd.: Marketing Report Prepared for the Real Estate Institute of South Australia, July, 1994 Newell, G and MacFarlane, J: Use of Computers in Valuation. The Valuer, May, 1991 Rossini,P.A, Kooymans, R.R., Kershaw P.J.: Constant Quality House Prices in an Australian Context - A Case Study of Port Pirie, South Australia. Pacific Rim Real Estate Society Conference RMIT January 1995