A Survey of Web Engineering in Practice

A Survey of Web Engineering in Practice Andrew McDonald and Ray Welland Department of Computing Science, University of Glasgow, Glasgow, Scotland. G12 8QQ 1 March 2001

Abstract During October, November and December 2000, we conducted interviews with a number of people within organisations in the United Kingdom who are involved in the development of Web-based applications. The goals of the survey were to identify more clearly the major issues facing the development of Webbased systems, and to see which, if any, traditional software engineering practices and techniques were being successfully applied. Fifteen interviewees from seven different organisations took part in the survey. This report details the background and results of our survey, and the conclusions that can be drawn about the practice of Web Engineering. We also discuss the major characteristics that describe Web-based application development, and the issues that a successful Web engineering process will have to address.

© University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 1

Foreword The following report details the results of a survey of Web Engineering teams during October, November and December 2000. The report comprises the following sections. The first section is essentially a copy of a draft paper that has been accepted for the Web Engineering Workshop at WWW10. This summarises the results of the survey and discussed their significance. The following three sections describe the survey and detail the design, raw results and analysis for each of our three classifications of Web development organisations: Contractors, Outsourcers and In-house. The three appendices contain copies of the questionnaires used for each of these three classifications of Web development organisations. Our objective in publishing this technical report is make the detailed survey results, upon which our paper is based, available to a wider audience. We also hope that perhaps some other research group might carry out a similar survey elsewhere for comparison.


Page 2

Contents 1. Introduction........................................................................... 4 1.1 Web Application Development ........................................................................ 4 1.1.1 Web Development Team Demographics ................................................................................... 5 1.1.2 Characteristics of Web Development Projects .......................................................................... 6 1.1.3 Web Engineering Processes in Practice ..................................................................................... 8

1.2 Conclusions...................................................................................................... 11

2. Contractors.......................................................................... 13 2.1 Design ............................................................................................................... 13 2.2 Results .............................................................................................................. 13 2.3 Analysis ............................................................................................................ 23

3. Outsourcers ......................................................................... 28 3.1 Design ............................................................................................................... 28 3.2 Results .............................................................................................................. 28 3.3 Analysis ............................................................................................................ 32

4. In-house ............................................................................... 37 4.1 Design ............................................................................................................... 37 4.2 Results .............................................................................................................. 37 4.3 Analysis ............................................................................................................ 41

5. Aknowledgements ............................................................... 44 6. References ............................................................................ 45 Appendix A - Contractor Questionnaire............................... 47 Appendix B - Outsourcer Questionnaire .............................. 49 Appendix C - In-house Questionnaire .................................. 51


Page 3

1. Introduction Over the past five years many software engineers have become concerned about the way Webbased applications are being developed [3, 14, 15 & 17]. These concerns are highlighted by the somewhat unusual characteristics that describe Web application development in comparison to traditional software development, and the seemingly poor suitability of traditional software engineering approaches and techniques to the development of Web-based systems. In order to try to gain a better understanding of the challenges facing Web Engineering we approached nine independent organisations involved in Web development to take part in our survey. Each organisation that was approached had Web-based development operations in the North of Britain. We placed each organisation into one of three categories: contractors, outsourcers, or in-house. Contractors are vendors who service Web engineering projects that are put out to tender by organisations who are classified as outsourcers. The in-house category describes organisations that primarily develop their Web applications within the organisation. It was decided that the three categories contractors, outsourcers, or in-house would suffice as a basic classification for organisations who are involved in Web development. The survey was conducted in a qualitative manner using an in-depth one-to-one interview technique. All the answers were recorded on paper by the interviewer conducting the survey. In order to help ensure access to potentially commercially sensitive information it was decided to present the results of the interviews anonymously, by individual and organisation. Five of the organisations that participated can be classified as contractors. One organisation that participated was classified as an outsourcer, which puts out to tender millions of pounds worth of Web contracts annually. In addition, three organisations that for one reason or another primarily build their Web sites in-house were also approached. Of the three in-house organisations that were approached, only one was willing to participate. In both cases, despite personally knowing people involved in Web development within the organisations, and providing them with advanced copies of the questionnaire, no official reasons were given for their unwillingness to participate. In total fifteen interviewees were involved in the survey. Where possible within each organisation that participated, we attempted to interview more that one type of developer. Each different type of developer was classified under one of the following roles: software engineer, creative designer, team manager, business expert or domain expert. It was felt that this basic classification of developers roles, would enable classification of all interviewees. Each of the five basic Web developer classifications can be subdivided into a number of other roles. For example, database administrators, Web masters and programmers would all be classified under software engineer. Eleven of the interviewees worked for organisations categorised as contractors, two of the interviewees worked for one outsourcing organisation and two interviewees worked for one in-house organisation. While there were questions that we felt were appropriate to all organisations involved in Web development, it was felt that a deeper insight would be gained by tailoring small parts of each questionnaire to suit each different category of Web development organisation.

1.1 Web Application Development The results from the survey can be broken into three sections. Section 1.1.1 describes the type of


Page 4

people, and the structure of the teams involved in Web-based development. Section 1.1.2 focuses on describing the characteristics of Web engineering projects. Section 1.1.3 addresses the features common to the Web engineering processes being used in industry, their shortcomings and their perceived advantages.

1.1.1 Web Development Team Demographics During the survey, we found inconsistency between organisations with respect to the titles being given to Web-based developers. However, we did find a number of similarities in the tasks developers from different organisations were responsible for during the life-cycle of a Web development project. For example, two of the interviewees belonging to different organisations in the contracting category had the job titles Producer and Senior New Media Designer. The job titles Producer and Senior New Media Designer conjure up radically different perceptions of what responsibilities would accompany the role, however when each interviewee was asked to identify their responsibilities within their respective Web development teams their answers were almost identical. A similar scenario repeated itself with two developers, again from the contracting section who had the titles Head of Production and Programming Team Leader and again almost identical responsibilities within their respective Web development teams. According to our results, the average age of a Web-based developer was observed to be 26 years. With respect to the gender breakdown of a typical Web development team 70% of the teams involved in this survey were male, 30% being female. One of the major differences between traditional software development and Web-based development is seen to be the small size of Web development teams [14 & 17]. Indeed this survey shows the average size of a Web-based team to be approximately six1 developers. It has also been noted that one of the other differences between traditional software development and Web-based development is the nature of the project deliverables [14 & 15]. In traditional software development most of what is delivered comprises a series of related software components with supporting systems. Whereas in Web application development, each delivered solution is bespoke, with the project deliverables comprising software components and data that are very inter-dependent. We consider the creation, integration and delivery of data to be just as important as the design, construction and delivery of software components in a Web application. Because of the emphasis on the data, we classified those people responsible for the creation of data, into one of two Web developer categories, business and domain experts. Domain experts provide data or content for Web applications. In addition to providing content, business experts are seen to provide guidance on achieving the business objectives of the Web application and are more involved in contributing to the overall structure of the Web presence. It should be noted that the majority of business and domain experts reside out with contracting organisations, and are most frequently found in different departments to the core Web development team, within inhouse organisations.

1

Note the interviewees in the outsourcing organisation were unable to put an accurate figure on the average size of a Web development team. They instead gave figures for the number of people under their respective control who are involved in Web development. The outsourcers answers were therefore ignored in this calculation. © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 5

The multidisciplinary nature or wider diversity of the types of developers within Web development teams is another factor that distinguishes Web development from traditional software development. The wider diversity of disciplines involved in Web development has also been noted by others [14]. At present there is little in the published literature that attempts to detail what different types of developers are required within Web development teams [8 & 14], and in what quantity they are represented. We attempted to identify what different types of developer are involved within Web development teams, and in what quantity they are represented. We did this by asking each interviewee to classify every member of a standard Web team under one of the following six categories: software engineer, creative designer, team manager, business expert, domain expert or other. According to the answers given1 , the members of a typical Web development team can be broken down as follows: 31% software engineers, 31% creative design, 20% management, 9% business experts and 9% domain experts. Put another way, given a Web development team with eight members2 , two will be contributing technical skills, two will be contributing creative design skills, two will manage the team, one will provide domain specific business or market advice and one will be providing domain information. Eight of the interviewees questioned had been involved in Web application development for less than three years, the other seven interviewees had been involved in Web development for less than six years. Five of the Web development teams involved in the survey had been in operation for less than three years, the other two being involved in Web application development for less than six years.

1.1.2 Characteristics of Web Development Projects Many consider the biggest hurdle to the successful adoption of a software engineering approach to Web development to be the short development life-cycle time, or time to market pressures that are associated with the development of most Web applications [15 & 17]. Indeed this is very true, in the experience of all interviewees no Web applications had a development life-cycle time longer than six months, the average development life-cycle time of a Web application was observed to be just under three months. One of the issues concerning the development of Web-based applications we were keen to get information regarding, was the number of Web-based projects that run over budget and/or over predicted time scales. 1

These figures are based on the answers given by the interviewees in the contracting section. Three of the interviewees in the outsourcers and in-house organisations were either unable to accurately answer this question, or they simply gave a breakdown of the personnel directly under their management. The fourth interviewee within the in-house and contracting organisations gave a breakdown that was dramatically different from the rest of the answers. As it was not possible to verify these answers, they were omitted. 2 The average size of a Web development team from the answers given to the breakdown was two developers higher than the average size of a Web development team. This was due to two factors. Firstly, we rounded the answers to the nearest whole developer. Secondly, one of the interviewees broke down the developers in an average team using one of the organisations current projects, that totalled seventeen developers, rather than using the figure of six developers previously given for the average size of a Web development team. © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 6

A number of practitioners in the field of cost and effort estimation have highlighted the difficulties in applying traditional software cost and effort estimation techniques to Web-based development projects [11, 12 & 17]. We tried to investigate the problems associated with cost and effort estimation by asking every interviewee the following questions: 1. How often do your Web development projects run over time? 2. What are the reasons for projects running over time? 3. How often do your Web development projects run over budget? 4. What are the reasons for Web engineering projects running over budget? The formal answers given to questions 1 and 3 did not correspond with the informal discussions held with members of the respective organisations in this survey. We concluded that we were often being given political answers to questions 1 and 3. In addition, the answers often differed between interviewees within the same organisation and were generally too widely spread to draw any valid conclusions. In our opinion one of the prime reasons for going over budget on a software or Web project is due to a failure to deliver on time. Our scepticism regarding the accuracy of the answers to questions 1 and 3 were supported by a number of interesting anomalies, discovered when we compared answers by the same interviewees to questions 1 and 3. For example, one interviewee stated that approximately 62% of the projects that they have worked on run over predicted time efforts, and then stated that only 5% of projects run over budget. Another interviewee stated that 25% of projects run over budget but that 0% of projects run over time. Further, another interviewee stated that 25% of projects ran over time, but that 0% of projects ran over budget. Questions 2 and 4 showed a lot of consistency in the interviewees answers1 . All the interviewees in the contracting organisations, mentioned one of the reasons for Web engineering projects running over time to be because of poor communication between themselves and their clients, late project changes by their clients, or poor understanding of the process of building a Web application on the behalf of the client. In our opinion these answers clearly indicate a problem with the requirements and analysis phases of most contractors Web engineering process. In addition when discussing reasons for projects running over time, the outsourcing organisation also acknowledged problems in capturing requirements and both in-house interviewees mentioned problems in controlling requirements creep. Other reasons given for projects not delivering on time include poor project management on behalf of the contractors, poor project effort estimation techniques, and inadequate testing procedures. The primary reason given for projects running over budget was also down to problems with the requirements phase of the Web engineering process. Other reasons given for projects running over budget include lack of resources, poor delivery of data/content, poor management, lack of professionalism and unforeseen costs. The interviewees in both the in-house and contracting organisations were asked if they ever found themselves short of development expertise during the development of a Web application. All interviewees answered the question2 . Two interviewees mentioned a lack of business experts, 1

Eleven out of the fifteen interviewees answered question 2. Nine out of the fifteen interviewees answered question 4. 2 The two outsourcers were not asked this question as it was felt that they would not be outsourcing the development of their Web applications if they had the skills in-house. © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 7

two interviewees mentioned a lack of domain experts, seven interviewees mentioned a lack of technical expertise, one interviewee mentioned a lack of creative design skills and one interviewee mentioned a lack of management skills. The interviewees were asked the following two questions, regarding the success rate of Web application development and the reasons for project failure. 5. How many Web projects proposed result in a delivered system? 6. What are the prime reasons for projects not resulting in a delivered Web system? Question 5 suffered similar problems to questions 1 and 3 previously. In that the formal answers to both questions did not match the informal discussions with many of the members of the organisations interviewed. Also, like questions 1 and 3, the answers varied in order to draw any valid conclusions. However like questions 2 and 4, question 6 was very illuminating, with six interviewees claiming lack of budget and four interviewees claiming problems with analysis and requirements phases in their Web engineering processes as a reason for project failure. The answers to question 6 further strengthen our claim that Web engineering processes being used in industry need to focus more on analysis and requirements phases if they are going to increase the rate of successful projects.

1.1.3 Web Engineering Processes in Practice Many practitioners in the field of Web engineering and software engineering have commented on the lack of suitable software engineering processes that can be used to build Web applications [14, 15 & 17]. We decided to investigate the way industrial Web engineering is being carried out by asking a number of questions relating to the development process being used to develop Web applications. All interviewees were asked the following question. 7. Do you have a well-defined and documented development process for building Web-based projects? Seven of the fifteen interviewees claimed to have a development process in place for building Web applications. However, of the seven interviewees who answered yes to this question, only two interviewees, who both belonged to the in-house organisation, were actually using an industry standard software development process for building Web applications. The other six organisations were using a development method that had been created in house. The industry standard software development process in question, used by the in-house organisation, is Dynamic Systems Development Method (DSDM) [5]. DSDM is a Rapid Application Development approach that focuses on delivering solutions in short time scales and professes close integration with business processes. DSDM is independent of any software engineering techniques and is a truly high level approach to building software systems. When asked to describe the process of building a Web application all the interviewees processes started with the development of a project Scoping document, that covered the requirements and


Page 8

design phases of a Web project. The Scoping document was one of only two deliverables that were created in the vast majority of Web engineering processes used by the organisations in this survey. The Web application being the second deliverable. If one considers the major stages in a number of widely used software development process [1, 2, 10, & 18], regardless of what order or how often one wishes to address these phases, to be Analysis, Requirements Capture, Design, Implementation, Testing (verification), Evaluation (validation), Training and Maintenance. Then one would hope that every Web engineering process would at least try to address each of these stages at some point in the development lifecycle. Sadly though the majority of the development processes seem to focus on Implementation with Requirements Capture and Design being carried out as one combined phase at the beginning of the project life-cycle, and Testing being carried out in parallel with Implementation, if at all. Only two interviewees mentioned an Analysis stage, and no interviewees mentioned Evaluation, Training or Maintenance stages in their Web development process. We believe the number of problems highlighted by the interviewees with respect to the requirements definition and capture to be a result of combining the design phase with the requirements phase. Often the stakeholders are making low-level decisions regarding how they will realise the design of a system before they understand what goals or problems the system should be addressing. When asked what features of their Web development process interviewees considered successful, three mentioned the Scoping document, four mentioned good relations with clients and four mentioned good management. When asked what features of the process interviewees considered problematic, nine mentioned problems with managing requirements or poor communication mechanisms with their clients, three mentioned problems with managing the development process, two mentioned poor testing procedures and one mentioned poor documentation. At the end of the questionnaire all interviewees were asked the following two questions. 8. What mechanisms do you employ to measure the success of a project? 9. At what stage in the development process do you employ these mechanisms? Ten of the interviewees considered a project to be successful primarily if the client was happy with the deliverable, seven interviewees mentioned achieving budget and time estimates as a mechanism for measuring a project’s success. Only three interviewees mentioned passing a Testing phase as a mechanism for measuring project success and not one interviewee mentioned involving end users as a mechanism for validating the success of a project. Since the majority of interviewees were using prototyping or user-centred design techniques during the development of their Web applications, it was alarming to hear developers mention clients rather than end users as a mechanism for validating the project deliverables. As others have noted [16], one of the main problems currently facing Web engineering is developers striving to satisfy their clients desires rather than their clients needs. All the interviewees measured success at the end of a project’s life-cycle, or after the project had ended. There were no mechanisms in place to properly validate the project deliverables during development. One of the most disturbing issues facing Web engineering is the poor attention paid to analysis, evaluation, training and maintenance in industrial Web engineering processes [6]. Without proper analysis of the problems or challenges to be addressed, how do developers know if they are


Page 9

building the right product? The lack of an evaluation stage leaves developers no mechanisms to properly validate the deliverables, and therefore no mechanisms for checking to see if they have built the right product. If one considers the simple hypertext paradigm of the Web then it is easy to ignore training, however if one does not require training to use a software system this does not imply that there is no need for training for the people responsible for maintaining the system. Indeed the complete lack of attention paid to maintenance issues in general during Web engineering processes must reduce the quality and longevity of the systems being developed, and enforce the growing opinion of many that the Web is going to be where most of the legacy problems of the future will be found [19]. Each interviewee was asked to identify which tools and technologies they used to implement their Web applications and at what stage in the development process these tools were used. The results showed little consistency between or within organisations. Often there were many competing tools and technologies being used within organisations with no clear rationale, with the exception of developer preference, for the selection of one solution over another. All we conclude with confidence from the answers to the questions relating to tools and technologies was that Microsoft’s Outlook Express was by far the most popular email client across all major platforms (Mac OS, Win32, and the major Unix flavors), and that Adobe Photoshop was the most popular raster image manipulation tool. As was shown previously two of the major differences between software development and Webbased development are the multidisciplinary nature and smaller size of teams. If one considers large-scale software development (100 developers, or more) and large Web engineering projects (100 developers, or more), then the overall management of developers becomes very similar in structure. In both cases developers are managed in small teams, however this is where the similarity ends. In software engineering, the teams are broken down into smaller units who will address different problems and tasks. Consider a large software engineering project, with the goal of building a new operating system, teams could be broken down as follows, one team responsible for the development of the kernel, one team responsible for the windowing system, another responsible for standard device drivers, and so forth. On a large Web engineering project teams will be broken down into multidisciplinary groups, who will build different sections of the Web application, but in general will produce very similar deliverables and work on very similar problems. In the course of developing a large software system each small team will have to interact with other teams, this is normally done through predefined interfaces, each team mostly viewing other team’s deliverables as black boxes. In Web engineering, not only do teams have to communicate information regarding their deliverables through predefined interfaces, but in order to reduce duplication of effort and ensure consistency, good communication amongst similar types of developer in different teams is just as essential as good communication within teams and between different teams.

Our personal experience of developing Web applications in both industry and academia have made us aware of the conflicts that often arise between different types of disciplines within a Web development team. To try to gain a better understanding of these conflicts we asked every interviewee the following two questions.


Page 10

10. What types of conflict arise between different developers in a Web development project? 11. How are conflicting views resolved between different developers in a project? According to the answers given to question 10, each different type of developer seems to poorly appreciate the contributions made by every other type of developer. Sadly according to the answers to question 11, there seems to be little policy in place for resolving conflicts, only one organisation mentioned using the Scoping document to resolve conflicts with the best interests of the project at hand.

1.2 Conclusions The impact of the World Wide Web on commerce, education, governments, and entertainment over the past five years has been significant. The ad-hoc and chaotic manner in which the majority of web applications are developed is a real cause for concern. Since 1998 a growing community with members in both academia and industry have come together to try to face the challenges this new field presents [21]. Giving the area the title Web Engineering, the members hope to address the challenges of developing, quality, robust, reliable Web applications that are delivered on time, on budget, to specification, and ultimately satisfy the project’s objectives. A number of members of the Web engineering community have commented on the lack of a suitable Web engineering process for the development of Web applications [7, 8, 9, 13, 15 & 17]. At present most of the industry focus is on tools that aid and assist the implementation of Webbased systems. Yet, as this survey has shown the greatest problems facing the field are at the Analysis, Requirements, Testing (verification), Evaluation (validation) and Maintenance stages of the Web engineering life-cycle. If a Web engineering process is to be successful then it must address the following: 1. Short development life-cycle times. According to our results, the average development lifecycle time is less than three months. This figure is dramatically lower than that of traditional software development. If a Web engineering process is going to be successful, then it has to cope with exceptional time pressures. 2. Delivery of bespoke solutions. Unlike traditional software engineering, Web engineering must cope not only with the development of software components but also with the development of data, and the inter-dependencies between them. 3. Multidisciplinary development teams. Many have commented on the diverse variety of disciplines involved in the development of Web-based systems [9 & 20]. A Web engineering process must take into account the different types of developer required to build a successful solution. Ensuring that all involved understand their roles and responsibilities and where overlap occurs how to resolve conflict in the best interests of the project in question. 4. Small development teams working in parallel on similar tasks. Like traditional software development, large numbers of Web developers are split into smaller teams. Each team providing an interface to the deliverables that they produce, enabling other teams to view the deliverable as a black box. A Web engineering process should also allow different types of developer to communicate amongst their peers, beyond particular projects and teams. This will help ensure consistency and will prevent duplication of effort amongst teams. © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 11

5. Analysis and Evaluation. There is a clear need for focus on Analysis and Evaluation stages in Web engineering processes. At the moment, there is little attention paid to either of these stages. A Web engineering process should encourage developers to address the following questions: a.) Why are we going to develop a Web application? b.) What problems or goals will the Web application address? c.) How will we know if the solution addresses these problems or goals? d.) How will we measure and validate the deliverables? 6. Requirements and Testing. In addition to understanding what issues the Web-based solution should address, and how we will measure the success of the deliverables in tackling these issues, a Web engineering process needs to focus more on Requirements and Testing. A Web engineering process should force it’s users to ask the following questions: e.) Are we building the product right? f.) How will we ensure that we have built the product right? 7. Maintenance. If the longevity and quality of Web-based solutions are to be improved, then more attention needs to be paid to the issue of Maintenance. While a well-documented system may not be essential during the development of a Web application, it is certainly necessary for ensuring the proper maintenance and update of the deliverables. Not only must the process address the points 1-7 above, but it should also be independent of tools and technologies. This is not to say that supporting CASE tools are not important. But such is the diversity and rapid change of technologies used to build Web applications, that a Web engineering process should remain as distant as possible from mechanisms used to implement Web applications.


Page 12

2. Contractors 2.1 Design Section 2 of this survey was conducted using eleven Web developers from five companies who service Web engineering contracts that are put out to tender, by organisations classified as outsourcers. Each organisation was assured of anonymity in order to assist in gaining information that is of a commercially sensitive nature. The companies are either based or have Web-based operations in Scotland. Company A is one of the largest companies servicing Web engineering contracts in the North of Britain with over one hundred employees. Company A specialises in thin client business-tocustomer e-commerce solutions for organisations primarily in the United Kingdom. Company B specialises in developing front-end solutions, playing to their strengths in the creative design field. Company B employs nine people in their new media team, and has clients throughout the UK. Company B often brings in outside IT contractors to assist with the more technical challenges some large projects present. Company C is a recent start up company in Central Scotland. Company C specialises in e-business solutions to clients in the North of Britain, focusing on business-to-business and business-to-customer solutions. Company D is an Internet Service Provider (ISP) with a small Web development team. Company E is a large IT Consultancy firm with over 300 employees based throughout the United Kingdom. Currently 25% of all company E’s business is in the development of Web applications. Company E’s customers are primarily made up of government and financial service sector organisations. Each interview was conducted following the questionnaire in Appendix A. The sample consists of eleven Web developers: three employees from company A, three from company B, one from company C, three from company D and one from company E. The results of the interviews are presented anonymously in order to assist in gaining information that is of a commercially sensitive nature.

2.2 Results NOTE: The numbers 1 to 11 will refer to the interviewees. Question 1 - What is your current job title? Answers 1: Programming Team Leader (software engineering background) 2: Technical Research & Development Manager (software engineering background) 3: Producer (creative design background) 4: Account Manager (management background) 5: Technical Programmer (software engineering background) 6: Senior New Media Designer (creative design background) 7: Head of Production (software engineering background) 8: Programmer (software engineering background) 9: Managing Director (medical background) 10: Web Designer (creative design background) 11: Principal Consultant (software engineering background) © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 13

Question 2 - What are your responsibilities within your Web development team? Answers 1: Assist in the development of each project’s scoping document (requirements and design specification) with clients. Responsible for managing technical developers throughout the rest of the project life cycle (implementation, testing). Not involved in the maintenance stage of any project. 2: Responsible for exploring, developing and providing middleware solutions. 3: Responsible for the day to day management of the creative design aspects of a Web engineering project. 4: Account management, client liaison, process management (preparation of documents between client and company) 5: Responsibilities include the development of technical solutions in each Web engineering project. 6: Responsibilities include management of creative design and direction of each Web engineering project, as well as client liaison and new business direction in the team. 7: Project Management 8: Client liaison, requirements capture, design, implementation and testing. Note most sites are designed in such a way that each client can maintain their own presence. 9: Sales and project management occasionally do some development. 10: Provide Web design artwork, and to an extent programming. 11: Chief technical developer on each Web engineering project undertaken by organisation. Question 3 - How many developers (inclusive of third party or external developers) make up your average Web development team? Answers 1: 8 people 2: 8 people 3: 9 people 4: 2-5 people 5: 7 people 6: 7 people 7: 5 people 8: 5-6 people 9: 5 people 10: 4 people 11: 8 people Question 4 - What is the average age of a member of your Web development team? Answers 1: 30 years of age 2: 27 years of age 3: 25 years of age 4: 24 years of age 5: 25 years of age


Page 14

6: 26 years of age 7: 25 years of age 8: 24 years of age 9: 24 years of age 10: 25 years of age 11: 29 years of age Question 5 - What percentage of males and females make up your average Web development team? Answers 1: Male: 66% 2: Male: 66% 3: Male: 66% 4: Male: 80% 5: Male: 100% 6: Male: 90% 7: Male: 100% 8: Male: 60% 9: Male: 66% 10: Male: 75% 11: Male: 50%

Female: 34% Female: 34% Female: 34% Female: 20% Female: 0% Female: 10% Female: 0% Female: 40% Female: 34% Female: 25% Female: 50%

Question 7 - What is the average length of a Web development project, from inception to first delivered working system? Answers 1: 1-4weeks _____ 2: 1-4weeks _____ 3: 1-4weeks _____ 4: 1-4weeks _____ 5: 1-4weeks _____ 6: 1-4weeks _____ 7: 1-4weeks _____ 8: 1-4weeks _____ 9: 1-4weeks _____ 10: 1-4weeks _√___ 11: 1-4weeks _____

1-3months_√__ 1-3months____ 1-3months____ 1-3months_√*_ 1-3months____ 1-3months_√__ 1-3months____ 1-3months_√__ 1-3months_√__ 1-3months____ 1-3months____

3-6months______ 3-6months__√___ 3-6months__√___ 3-6months______ 3-6months__√___ 3-6months______ 3-6months__√___ 3-6months______ 3-6months______ 3-6months______ 3-6months__√___

6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______

* Normally it only takes 1-3 months to develop a project from requirements to deliverable of final Web application. However it often takes as long to successfully get a contract as it does to develop the project. Question 6 - Break down the number of developers (inclusive of third party or external developers) in an average Web development team into the following roles!


Page 15

* Often there can be two or three business experts in a Web project. ^ A software engineer is required only in one third of projects undertaken. º Figures are rounded to the nearest person. Question 7 - What is the average length of a Web development project, from inception to first delivered working system? Answers 1: 1-4weeks _____ 2: 1-4weeks _____ 3: 1-4weeks _____ 4: 1-4weeks _____ 5: 1-4weeks _____ 6: 1-4weeks _____ 7: 1-4weeks _____

1-3months_√__ 1-3months____ 1-3months____ 1-3months_√*_ 1-3months____ 1-3months_√__ 1-3months____

3-6months______ 3-6months__√___ 3-6months__√___ 3-6months______ 3-6months__√___ 3-6months______ 3-6months__√___

6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______ 6+ months______


Page 16

8: 1-4weeks _____ 9: 1-4weeks _____ 10: 1-4weeks _√___ 11: 1-4weeks _____

1-3months_√__ 1-3months_√__ 1-3months____ 1-3months____

3-6months______ 3-6months______ 3-6months______ 3-6months__√___

6+ months______ 6+ months______ 6+ months______ 6+ months______

* Normally it only takes 1-3 months to develop a project from requirements to deliverable of final Web application. However it often takes as long to successfully get a contract as it does to develop the project. Question 8 - How often do your Web development projects run over time? Answers 1: Did not answer. 2: 75% of the time. 3: 50% of the time. 4: 25% of the time. 5: 0% of the time. 6: 0% of the time. 7: 100% of the time. 8: 50-74% of the time. 9: 50-74% of the time. 10: 50-74% of the time. 11: 0-24% of the time. Question 9 - What are the reasons for projects running over time? Answers 1: Did not answer. 2: Changes made by clients to project scoping document during development of project. 3: Lack of preparation and understanding on behalf of client, major reason for projects running over is poor communication between client and internally between different members of the development team. 4: Changing scope of project by clients, due to lack of knowledge of effort required in order to build a Web system. Poor allocation of resources on clients behalf for provision of content. Also conflict arises early in project between educating client and providing free Consultancy. 5: Not applicable. 6: Not applicable. 7: Getting content from clients. 8: Mostly due to clients not producing content, sometimes though due to poor project management. 9: Poor prediction of effort, heavy workload, and clients not producing content, also not testing properly. 10: Mainly lack of content from clients. 11: Poor co-operation from clients, often content is not delivered in time. Also two contracts can conflict in terms of time.


Page 17

Question 10 - How often do your Web development projects run over budget? Answers 1: Did not answer. 2: Did not answer. 3: 50% of the time. 4: 0% of the time. 5: 0% of the time. 6: 25% of the time. 7: 50% of the time. 8: Don’t know. 9: 5% of the time. 10: 25% of the time. 11: 0-24% of the time. Question 11 - What are the reasons for Web engineering projects running over budget? Answers 1: Did not answer. 2: Did not answer. 3: Client changes to scoping document, during the project development. 4: Not applicable. 5: Not applicable. 6: Client changes in scoping document. Often these take the form of additions rather than changes. 7: Poor allocation of resources on client’s behalf. Most clients have a fixed budget! 8: Not applicable. 9: Clients not delivering. 10: Lack of discipline on behalf of clients and developers. 11: Projects rarely run over budget as every project is fixed price. Question 12 - Do you ever find yourself short of development expertise in a Web development project? Answers 1: Yes. Getting domain expert to provide content in a suitable format and to agreed time scale. 2: Yes. Getting domain expert to provide content in a suitable format and to agreed time scale. 3: Yes. Technical expertise is often spread pretty thinly across a number of projects in development in parallel. 4: Yes. Technical experts who can work in team centred on creative design skills. 5: Yes. Short on technical expertise. 6: Yes. Often short on technical expertise. 7: Yes. Lack of understanding of business practices on behalf of management, technical and creative design developers. 8: No.


Page 18

8: No. 9: Yes. In all areas. 10: No. 11: Often there is not enough technical expertise to deal with all concurrent competing projects. Question 13 - How many Web projects proposed result in a delivered system? Answers 1: Not Sure. 2: Not Sure. 3: Did not answer. 4: 51-75% of projects. 5: 76-100% of projects. 6: 51-75% of projects. 7: 76-100% of projects. 8: Don’t know. 9: 51-75% of projects. 10: 76-100% of projects. 11: 26-50% of projects. (Includes projects that go to other vendors) Question 14 – What are the prime reasons for projects not resulting in a delivered Web system? Answers 1: Clients not having the budget. 2: Not applicable. 3: Did not answer. 4: Lack of budget on client’s behalf, changes on client’s behalf, or loose on pitch situation. 5: Clients scope is not clearly defined. 6: Lack of budget on client’s behalf, lack of understanding in terms of the cost of technology. 7: Did not answer. 8: Not applicable. 9: Lack of budget 99.9% of the time. 10: Not applicable. 11: Pre selected supplier, price is too high and requirements are often unclear. Question 15 - Do you have a well-defined and documented development process for building Web engineering projects? Answers 1: Not really. Still in development, although some procedures are in place primarily a scoping document that includes requirements and design. Both parties sign off on scoping document before project can continue. 2: Not really. Primarily focusing on backend solutions after involvement in scoping document tends to remove oneself from specific projects. Uses UML to model design of middleware solutions.


Page 19

3: Not really. Primarily uses site maps for front-end and backend implementation guide from scoping document. 4: Yes. Phases include requirements document, creative design proposals, design development, creation and implementation. No explicit testing stage, testing is integrated into creation and implementation stage. Maintenance is not considered as part of a projects development life cycle. 5: Not really involved in any stage except creation and implementation, often uses design documentation and templates. 6: Assists with definition of Brief, Costing Model, and Design Development. 7: Well-defined process, not well documented. The only documentation is the requirements and design information that is delivered in one document. 8: Not really, mostly requirements, technical and design specifications. 9: We have a ‘Document Pack’ which includes: quotation and project definition, contract, design and implementation. 10: Research -> Development -> Conclusion -> Finalise. 11: Yes. Bespoke process. Business Analysis stage (sometimes already done by client) followed by requirements specification and risk analysis that comes in the form of a Project Initiation Document, and finally Test Document. Question 16 - What features of your Web development and design process do you consider successful? Answers 1: The scoping document. 2: Breaking up the development teams into smaller groups. 3: Task management and site maps. 4: Use of templates. 5: Creative process works well. 6: Good communication with clients. 7: All works well. 8: Small teams, good communication with clients. 9: Client content summary sheet. Documenting and getting clients to agree to the project specifications. 10: Them all. 11: Independent Project Advisor (who is not part of the development team), and a separate Quality Analysis Team. Question 17 - What features of your Web development and design process do you consider problematic? Answers 1: Need for more flexibility in the process. Although one of the problems is that if you allow clients to change one aspect of the scoping document, they think they can change everything. 2: Major problem communicating with clients. 3: Need for better communication with clients, most clients don’t understand the process. 4: No clear definition of process for clients to view, lack of communication with clients. 5: Managing the development process, keeping focused on which points of the job are crucial. 6: Better technical expertise, and a need to be stricter with clients. 7: Biggest problem is client management. © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 20

8: Don’t think we produce enough documentation. 9: Getting clients to stick to the process. 10: Too many ideas. 11: Creative Design Aspects of Web design. Question 18 - If you do not use a Web development process, why not? Answers 1: Not relevant. 2: Not relevant. 3: Not relevant. 4: Not relevant. 5: Not relevant. 6: Not relevant. 7: Not relevant. 8: Have a project life cycle plan, very loose, need a process that is well defined not well documented. 9: Not relevant. 10: Not relevant. 11: Not relevant. Question 19 - What software tools do you use when developing a Web engineering project, what tasks do you use them for, and at what stage in the development are they used? Answers 1: PROJECT DEFINITION: Outlook Express, Microsoft Project, Excel and Word. DESIGN: Photoshop, Freehand (for site Maps). IMPLEMENTATION: Cyberstudio and Dreamweaver for HTML creation, BBEdit or Notepad for HTML and PHP Scripts, Photoshop, FTP client, Flash, Director if there is a game to be produced, Internet Explorer and Netscape (normally just the most recent versions). TESTING: All testing is outsourced to a company who resides outside the United Kingdom. Testing is outsourced due to the lack of suitable testing suite in-house. MAINTENANCE: Same tools as implementation and CVS. A separate team to development handles all maintenance. 2: PROJECT DEFINITION: DESIGN: Primarily using UML for modelling middleware designs. IMPLEMENTATION: Forte for Java (Java IDE) because of its support for CVS. Primarily JavaBeans development to support thin client architectures. TESTING: Internally developed testing suite. MAINTENANCE: Same as implementation. 3: PROJECT DEFINITION & DESIGN: Microsoft Word, Excel, Project, Illustrator, Photoshop, Visio (for site maps), Flash, Dreamweaver. IMPLEMENTATION: Same as project definition and design, Internet Explorer and Communicator. TESTING: Internet Explorer and Communicator. MAINTENANCE: Same as implementation. 4: PROJECT DEFINITION: Internet Explorer, Netscape, Outlook Express, Office 2000, Microsoft Project. DESIGN: IMPLEMENTATION: TESTING: MAINTENANCE: 5: HTML Editor, Dreamweaver and Interdev, Photoshop, Outlook Express, Internet Explorer and Netscape, SQL Development client, Office 2000, Flash. 6: Photoshop 5.5, Illustrator 8.0, Freehand 9.0, Flash 4.0 & 5.0, Fireworks 3.0, Dreamweaver 3.0, Ultradev, BBEdit, Homesite, FTP Clients, Outlook Express, Office 98, Acrobat Distiller and Viewer and Director. © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 21

7: PROJECT DEFINITION: Microsoft Project, Office 2000. DESIGN: Photoshop, Illustrator, Zara, Quark Express. IMPLEMENTATION: ColdFusion, Homesite, SQL Server, Microsoft Office 2000, Flash 4.0 & other small freeware and shareware tools. TESTING: Word, Internet Explorer versions 4 and 5, Netscape versions 3, 4, and 5. MAINTENANCE: VNC. 8: PROJECT DEFINITION: Pine, Netscape, OFFICE 2000. DESIGN: Photoshop, BBEdit, Flash, Freehand, and pine, Netscape. IMPLEMENTATION: Sitepad Pro for VRML, Perl, Java, PHP, HTML.TESTING: Netscape and Internet Explorer. MAINTENANCE: 9: PROJECT DEFINITION: Pine, OFFICE 2000, Acrobat. DESIGN: Illustrator, Word and PDF. IMPLEMENTATION: Photoshop, Freehand, Illustrator, Fireworks, BBEdit, Sun’s JDK, PHP 4.0, Perl, JVM, Postgress, and Access. TESTING: Netscape, Internet Explorer on Mac OS, Win 32 and Linux. MAINTENANCE: 10: DESIGN & IMPLEMENTATION: Photoshop, illustrator, freehand, BBEdit and Flash. 11: Front Page or ColdFusion. Serano for volume Web testing. Question 20 - What types of conflict arise between different developers in a Web development project? Answers 1: We often find it hard to communicate technical issues within team. Also find difficulty in communicating project constraints to the client. 2: Conflicts in coding and development style. Happens more often when contractors are brought in. 3: Always a trade off between function and form. 4: Biggest problem is people from a design background not understanding business objectives. Also many technical developers who do not rate the importance of design. 5: No problems system works really well. 6: Conflicts between technical and creative developers and content providers. Also function vs. form arguments. 7: Not answered. 8: Don’t really have any! 9: Very few, good working relationship. 10: Arguments over tools. 11. Couldn’t think of any. Main disagreements come in terms of time and effort prediction. Question 21 - How are conflicting views resolved between different developers in a project? Answers 1: Resolve issues between ourselves. 2: No answer. 3: Trace conflict back to scoping document and resolve according to which solution best meets requirements. 4: Educate team by going back to scoping document, and resolve conflict according to which decision best meets project goals. 5: Not applicable. 6: Meet internally and find a compromise that works. 7: Resolve at management level, educating team members at the same time. 8: Not applicable.


Page 22

9: Not applicable. 10: Winner usually proves his point. 11: Team Manager and Client resolve all issues. Question 22 - What mechanisms to you employ to measure the success of a project? At what stage in the development process do you employ these mechanisms? Answers 1: Success is measured by client satisfaction with site, and is linked to client payment. 2: Two weeks before project launches the technical specification is tested and the client goes through site. 3: Achieving proposed time scale and budget for project in question. Note workload is measured in hours. 4: No formal project effectiveness mechanism in place. Use client feedback, profit margin, repeat business and referrals to measure success. 5: No set mechanisms, primarily consider a project successful if client is happy. 6: Projects success is measured by clients satisfaction with project, also a project is considered successful if it gains good PR for firm. 7: Repeat business, often customer goals are so vague that it is difficult to measure whether or not their needs have been addressed. There is quite a big conflict between educating the client as to what they need over what they want. 8: If site works quickly on low bandwidth connections, if it’s delivered on time and if the client’s happy. 9: Client feedback, longevity of deliverable and relationship with client. 10: If the end product works and is consistent across all platforms and browsers. 11: Make sure project is well-defined (personal commitment made by all project managers). Establish change control mechanisms. Use separate Quality and Testing teams.

2.3 Analysis Questions 1 and 2: Revealed a wide range of titles within the industry and within organisations. While many people were performing similar types of roles they often had different titles. For example Senior New Media Designer and Producer were very similar in their responsibilities. Question 3: In the contracting categorisation of Web development organisations the average number of developers in a Web development team including external contractors and members of the client organisation was observed to be 6. The results of this survey indicated the average size of Web development teams in organisations categorised as contractors to range from 2 to 9 developers. Developers in Company A, noted that it was nearly impossible to manage development teams larger than 9 people, this observation came from previous company experience where teams had been larger than 9 people. It has been noted in software engineering literature that small development teams don’t require a heavily documented development process due to the ease of communication within a small team. However documentation is essential for communication between different teams and for ensuring the quality and longevity of the software components through maintenance. Company A may have found it difficult to mange teams greater than 9 people due to the lack of a formal well documented development process. However there are a number of other factors that could also © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 23

have also contributed, these included: diversity of types of developer, short life cycle time, and poor communication with clients. It should also be noted that the answers from developers within the same organisation varied. Question 4: According to this data the average age of a Web developer in organisations who service outsourced Web development projects is 26 years. The average age of Web developers was observed to range from 24 to 30 years. It should also be noted that the answers from developers within the same organisation varied. Question 5: On average in contracting organisations 74% of Web developers are male, and 26% of Web developers are female. In the largest development organisations as much as one third of their Web developers are female. It should also be noted that the answers from developers within the same organisation varied. Question 6: According to the answers given to question 6, 31% of Web development teams are comprised of developers from a technical (software engineering) background. 31% of Web development teams are comprised of developers from a creative (creative design) background. 20% of Web development teams are comprised of managers. 9% of Web development teams are comprised of developers from a business background. 9% of Web development teams are comprised of domain experts. Note these figures are based on the average classification of developers, by each interviewee, in an average Web development team. In an average team of eight web developers approximately two developers would be there to provide technical skills, two would be there to provide creative design skills, two would be managing the group and there would be one business and one domain expert. This would comprise an average team of 8 Web developers. The figure of eight developers for the size of an average Web development team differs by two developers from the average number of developers in a Web development team given in question 3. This is due to interviewee 11 giving the average size of a web development team to be eight, but using an example of the breakdown of the types of developer involved from a current project with seventeen developers. Question 7: The average development life-cycle time for the development of a Web application is 3 months. Most Web engineering projects have a life cycle of 1 to 6 months. No Web engineering projects were reported to take longer than 6 months. Questions 8 and 10: Answers to questions 8 and 10 must be treated with some scepticism, I often felt that I was getting the company’s politically approved answers to these questions. This was confirmed by one interviewee who ‘off the record’ stated that their organisation had never achieved predicted budget costs or delivered on time for any Web-based project in their entire history, this contradicted the answers given to questions 8 and 10. In addition when alerting one interviewee that reasons for running over might be due to problems created by the client the interviewee in question changed their answer from 0% to 100% of the time. Also one interviewee refused to answer question 8 and two refused to answer question 10.


Page 24

In my opinion one of the prime reasons for going over budget on a project is due to a failure to deliver on time. Observing the answers given by the interviewees highlights a number of interesting anomalies. Interviewee 9 stated that approximately 62% of the projects that they have worked on run over predicted time efforts, interviewee 9 then stated that only 5% of projects run over budget. In addition interviewee 6 stated that 25% of projects run over budget but that 0% of projects run over time. Also interviewee 4 stated that 25% of projects ran over time, but that 0% of projects ran over budget. Question 9: All the interviewees who answered question 9 gave one of the reasons for Web engineering projects running over time, as the result of poor communication with, late changes by, or poor understanding of the process of building a Web application on the behalf of the client. In addition, two interviewees mentioned poor project management on behalf of the contractors, as being a reason for projects running over time. Note interviewee 1 never answered question 9, and the question was not applicable to interviewees 5 and 6 as their answers to question 8 stated that projects never ran over time. Question 11: Interviewees 3, 6, 7, 9 and 10 mentioned client changes or additions to the project scope, poor allocation of client resources and failure of clients to deliver content amongst their reasons for projects going over budget. Note interviewees 1 and 2 never answered question 11, and the question was not applicable to interviewees 4 and 5 as their answers to question 10 stated that projects never ran over budget, also interviewee 8 was not informed enough about project budgets to answer question 11. Question 12: 82% of interviewees in the contracting category acknowledged that their Web development teams often lacked critical expertise of one kind or another. 36% of interviewees acknowledged that their Web development teams often lacked business or domain expertise. 64% of interviewees stated that their Web development teams often lacked technical expertise. 18% of interviewees stated that their Web development teams often lacked creative design expertise. 18% also stated that their Web development teams often lacked suitable management expertise. In teams as small as eight developers with one manager for every three developers, it is a major cause for concern to hear of a lack of management expertise. In addition it was also stated that a lack of appreciation and respect for other types of developers, and poor management of skills across multiple concurrently developing projects was also a major problem. Question 13: According to the answers to question 13 approximately 70% of all Web engineering projects result in a deliverable. Like the answers to questions 8 and 10 the answers to question 13 should be treated with some cynicism, again I often felt I was being fed the company’s corporate line during answers to question 13. It should be noted that for undisclosed reasons four of the interviewees would not or claimed to be unable to answer this question. Question 14: Client’s lack of budget, poor initial requirements or changes to requirements by the client were seen as the main reason for Web-based projects not resulting in a deliverable. Also losing out at bid stage to other vendors was also mentioned as a reason for projects not resulting in a deliverable. Note five interviewees would not or claimed unable to answer this question.


Page 25

Question 15: Each interviewee was found to be using a bespoke in-house development process for building Web applications. The first stage in the building of any software system is the Analysis stage, which should help to define the problem(s) to be solved or the goals to be achieved. According to the all the interviewees bar one, no Analysis stage seems to be carried out by any contracting organisation that was surveyed. Then follows the Requirements Specification, where the solution to the specified problem in the Analysis stage is defined in addition to any constraints or special features that have to be taken into consideration in the design of the solution. If the Requirements Specification defines what the solution will do, the following stage the Design stage specifies how the solution will be realised. Usually the first stage in a Web engineering process used by contractors is to produce a Scoping document, that is a mixture of requirements and design information for the Web application to be developed. In the vast majority of cases the Scoping document is the only document that is produced by the contractors throughout the development of the Web application. In my opinion mixing the requirements and design information is not a good idea. Not only is there not a clear Analysis stage that informs everyone involved of the goals of the project, or the problems to be addressed, but before a solution is properly defined decisions are already being made about how that solution will be realised. The stage following Design is the Implementation stage, where the design is actually created. Then follows the Testing stage. Testing is a process where by the developers try to ensure quality and correctness, often this is done by verifying that the deliverables satisfy the project requirements. Thorough testing of a Web application must be severely complicated if not impossible without an explicit requirements document available. Note often the interviewees had no explicit testing phase but stated that testing is carried out during implementation. The vast majority of the Web development processes described and used by the interviewees comprise: the creation of a scoping document, followed by an implementation phase and on the rare occasion followed by a testing phase. In addition to having no clear verification stage no one mentioned validation or evaluation of the deliverable Web-based system. Evaluation is normally carried out to see if the deliverable achieves the goals or solves the problems highlighted in the Analysis section. So not only is there nearly no effort to check if the project has built the product right, but there is no mention of checking to see if the contractors have built the right product. In addition no one mentioned or supported maintenance of the project deliverables. Without at least minimal support for maintenance the quality of the delivered Web application will be poor and its longevity could be unnecessarily shortened. Two of the contracting organisations acknowledged that they were currently in the process of trying to develop a better Web development process, after failing to find a suitable existing software engineering process. Question 16: The answers to question 16 varied too much to draw any valid conclusions. Question 17: Seven of the eleven interviewees mentioned changes by clients, poor communication with clients or the management of clients as being the major features of their current approach to building Web-based systems that are problematic. I believe a lot of these problems are the result of shoddy Analysis and Requirements approaches to building Web


Page 26

applications. If competent methods of understanding the problems or goals of the project and specifying clear solutions to these problems were adopted, the clients would not be the major stumbling block that they appear to be to contractors at the moment. A number of answers to this question highlight the need for better documentation and a formal process as being some of the major problems with contractors current Web development methods. Question 18: All interviewees had adopted a bespoke approach to developing Web-based systems, a few interviewees had tried to adopt traditional software development life-cycles but had rejected them quickly as being unsuitable to the development of Web-based systems. As a result question 18 was redundant. Question 19: Question 19 was designed to highlight any tools that were being used consistently throughout the industry at specific stages in the development process or for specific Web development tasks. Unfortunately the question did not work very well with a number of the interviewees, who were unsure of my descriptions of the identified stages. However the answers showed some consistency among different organisations in terms of the tools used e.g. Photoshop was the most popular raster image manipulation tool and Outlook Express was the most popular email client across all major operating system platforms. Question 20: There were a number of conflicts observed to occur internally between different types of developer. Two interviewees observed conflicts between technical developers, two interviewees observed conflicts between technical developers and creative designers and one interviewee observed conflicts between management and technical developers. Note I consider a lot of the clients to be developers, who are split into domain and business developers during the development of a Web application. Conflicts were seen to arise between technical developers and business experts, between creative designers and business experts, between technical developers and domain experts and also between creative designers and domain experts. Question 21: Conflicts are usually resolved either internally between developers or by line managers, unfortunately only two interviewees mentioned looking at requirements to resolve conflicts by using the goals of the project to consider what is best for the success of the project! Question 22: According to the answers to question 22 project success is measured primarily by how happy clients are with the delivered Web application. Most interviewees never mentioned evaluation by end users or achieving measurable project goals. Unfortunately this is consistent with the lack of proper Testing and Evaluation stages in most of the contractors Web development process. I can only conclude that most of the deliverables are tailored to the client’s desires rather than the client’s needs.


Page 27

3. Outsourcers 3.1 Design Section 3 of this survey was conducted using two Web developers from one organisation who outsource a number of their Web engineering contracts, in addition to developing a minority of their projects internally. The organisation in question is based throughout the North of Britain, with over 400 employees in their main office, and a further 1200 in other offices throughout the region. Each interview was conducted following the questionnaire in Appendix B. The results of the interviews are presented anonymously in order to assist in gaining information that is of a commercially sensitive nature.

3.2 Results NOTE: The numbers 12 and 13 will refer to the interviewees in the order they were questioned. Question 1 - What is your current job title? Answers 12: IS Manager (software engineering background) 13: Project Manager (business management background) Question 2 – What are your responsibilities within your organisation? Answers 12: Manage the software application development for the organisation, including both in-house and external projects. 13: Lead business transformation within organisation. E-enable entire organisation by 2003. Question 3 – How many people within your organisation are involved in a typical Web engineering project? Answers 12: 20 full time developers, plus an additional 20 out with the IS department. 13: 120 in total, which will be split into smaller teams as e-enabling develops. Question 4 – What is the average age of people, within your organisation, who are involved in a typical Web engineering project? Answers 12: 25–30 years of age. 13: 30 years of age. Question 5 – What percentage of males and females make up the people who are involved in a typical Web engineering project?


Page 28

Answers 12: Male: 80% 13: Male: 40%

Female: 20% Female: 60%

Question 6 – Classify the types of people involved in a typical Web engineering project in your organisation! Answers 12: Software Engineer: 10 Creative Designer: 0 Team Manager: 4 Business Expert: 2 Domain Expert (content provider): 0 Other: 4 (1 database analyst, 1 technical author, 2 report writing experts) 13: Not applicable at this stage, too early in project to classify. Question 7 – What is the average length of a Web development project, from inception to first delivered working system? TICK AS APPOPRIATE! Answers 12: 1-4weeks _____ 13: 1-4weeks _____

1-3months__√_ 3-6months______ 1-3months__√_ 3-6months______

6+ months______ 6+ months______

Question 8 – How often do your Web development projects run over time? Answers 12: 0-24% of the time 13: 25-49% of the time. Question 9 – What are the reasons for projects running over in terms of time? Answers 12: Not being able to properly define requirements. 13: Not Answered. Question 10 – How often do Web development projects run over budget? TICK AS APPROPRIATE! Answers 12: 0% of the time. 13: 25% of the time. Question 11 – What are the reasons for projects running over budget? Answers 12: Not applicable. 13: Not properly managed, not tied into business process. Question 12 – What are the primary reasons for outsourcing a Web engineering project?


Page 29

Answers 12: One, don’t have the necessary technical skills in-house. Two, time-scale is too short for inhouse teams to cope with. 13: Skills not found in-house, not organisation’s core competence, when time is a critical factor (employ organisation with ten developers for 6 weeks) as opposed to 2 people for 7 and 1/2 months. Question 13 – What criteria do you look for in vendors tendering for a Web engineering project? Answers 12: Evidence of a quality approach to development (BSI, Certification, and Quality Manual), Similar technical framework, established track record in the field. 13: Evidence of necessary skills, proven track record, use of similar development technologies to in-house teams. Also good proposed project cost, and evidence of innovative approach in terms of solutions delivered. Question 14 – Describe the process of putting a Web engineering project out to tender? Answers 12: Due to the short time scales in Web engineering it takes too long to assess each vendor on a project by project basis. As a result every two years four vendors are selected to become preferred suppliers. Approved suppliers are given a requirements specification with a fixed budget. If they agree to take the project on, a testing and evaluation document is produced by the IS Team, while the project is in the design and implementation stage, this is given to the vendors only if requested! The organisation then use the testing and evaluation document as the metric the deliverable must pass in order for the project to be accepted. 13: Identify business problem, do business analysis, invite parties to tender, evaluate tenders, through presentations and written proposal which are evaluated by panel, refine list of vendors, interview and then commission project. Question 15 – Describe the process of selecting a vendor for a Web engineering project! Answers 12: See question 14. 13: Gain executive approval, create panel to evaluate vendors, short list, mark and score each short listed vendor’s proposal, then select supplier. Question 16 – Describe the process of accepting the final deliverable of an outsourced Web engineering project! Answers 12: See question 14. 13: Assess what is actually delivered compared to detailed brief. Question 17 – How many proposed projects result in a delivered system? Answers 12: 100% © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 30

13: Not enough! Question 18 – What are the prime reasons for a project not resulting in a delivered Web system? Answers 12: Not applicable. 13: Not a rigorous enough appraisal process, typically not enough business analysis done. Question 19 – Do you insist on a well-defined and documented development process or methodology for building Web sites? Answers 12: Yes. See question 14. 13: Yes and No. Our department would but organisation does not always do so. Question 20 – What features of your Web development and design process or methodology do you consider successful? Answers 12: Documentation. Note can’t always get the requirements and proposal well defined. 13: Understanding that it is about business processes, not technology, need to be very specific about functionality and business needs. Technology seduces too many people. Question 21 – What features of your Web development and design process or methodology do you consider problematic? Answers 12: It is not possible to alter the requirements after they have been signed off. 13: Business buy-in, ownership of project deliverable, lack of clear leadership, and poor ability to manage change in both business and IT. Question 22 – If you do not use a Web development process or methodology, why not? Answers 12: Not Applicable. 13: Not Applicable. Question 23 – What software tools do you use when you are involved in a Web engineering project, what tasks do you use them for, and at what stage in the development are they used? Answers 12: Outlook Express, IE5, Netscape, NT 4.0, OFFICE ’97, Rational Rose, Visual Interdev, Visual Studio, Visual Basic, JavaScript, Visio, XML, ASP, Site Server, Ingress, SQL Server, Oracle. 13: OFFICE, TEAM PHONE, Project, GIS System. Question 24 - What types of conflict arise in your organisation and with the vendor in a typical Web development project?


Page 31

Answers 12: Getting internal clients to sign off on projects. ‘Getting the requirements right is the hardest part of the process’ 13: Often technological solutions do not address business goals, need better defined business goals. Also conflicts arise when people want to change scope of project after requirements have been signed off. Question 25 – How are these conflicts resolved? Answers 12: Sign off projects without client approval, or educate them, finish this project and start another to deal with issues! 13: Use of Standards Procedure incorporating design and technical architecture, before project budget can be spent. Question 26 – What mechanisms do you employ to measure the success of a project? At what stage in the development process do you employ these mechanisms? Answers 12: If a project is delivered on time then it is seen to be successful. Note: use expert opinion and analogy for project effort prediction and estimation. 13: Cost Model, use base-case to measure benefits of deliverable. Need to track return y on investment x. New role being created to monitor this.

3.3 Analysis Questions 1 and 2: Both interviewees are in management positions within the organisation in question. According to both interviewees job titles this organisation has not adopted the new breed of e-titles that has currently swept the contracting market. Question 3: Unfortunately question 3 was poorly understood, with both interviewees giving the size of their entire departments rather than the size of an average Web development team. Although interviewee 13 did acknowledge that his team of 120 developers would be split into smaller teams as e-enablement develops. Question 4: The average age of developers involved in a Web engineering project within the organisation questioned is between 25 and 30 years of age. Note this figure is consistent with that of the average age of developers in a contracting organisation. It should also be noted that the answers from developers within the same organisation varied. Question 5: The answers to question 5 varied too much to draw any valid conclusions. Question 6: According to interviewee 12, 50% of people involved in a Web engineering project are from a software engineering background, 20% are from a management background, 10% are from a business background, and 20% are from other disciplines. Note interviewee 13 could not answer the question.


Page 32

Question 7: The average length of a Web development project is between 1 and 3 months, this figure is also consistent with the length of a Web development project serviced by contractors. It should be noted that larger projects are often broken into many smaller projects as result larger projects can take longer than 6 months to develop. Question 8: Approximately 25% of projects run over in terms of time. Again like questions 8 and 10 in the contracting section I felt the answers to this question were politically influenced. This is confirmed by the answers given by interviewee 12 who stated that projects hardly ever run over time, where as interviewee 13 stated that just under 50% of projects run over time. Question 9: Only interviewee 12 answered this question, interviewee 13 did not answer. As was found in most of the answers to question 9 in the contracting section the reason given for projects running over time was due to problems encountered during the requirements definition. Question 10: Like questions 8 and 10 in the contracting section I felt the answers to this question were politically influenced. Again both interviewees answers were different! Question 11: Interviewee 12 stated that projects never ran over budget and therefore did not answer this question. However interviewee 13 gave the main reason for projects running over budget as being due to a lack of project management and poor tie in with business processes. This answer when considered with the answer to question 11 by contracting developers seems to indicate problems in managing the development process and understanding and achieving project goals. In essence developers are finding it difficult to answer the question. Are we building the right product? If they are asking it at all! Question 12: The main reasons for projects being outsourced are due to a lack of in-house skills, the inaccessibility of in-house skills or because the time scale cannot be achieved in-house given current employment restrictions. For example, when the outsourcers want ten developers for six weeks as opposed to two developers for seven and a half months. Question 13: Both interviewees identified the main criteria that are sought in a vendor tendering for a Web engineering project to be: a quality approach to development, use of approved development technologies and tools and proven successful track record in building similar Web applications. In addition interviewee 13 also looked for good proposed project costs and an innovative approach to development, although interviewee 13 could offer no consistent mechanisms for evaluating both these criteria. Question 14: Three types of documentation can be produced during a Web engineering project: Business Analysis document, Requirements and Design document, and a Testing and Evaluation document. Unlike contractors the outsourcers go through an Analysis stage before starting the Requirements Specification and Design stages. The Analysis stage results in a Business Analysis document, however this is not made available to the contractors. In addition like the contractors Web development process the requirements and design stages are carried out as one phase and result in one deliverable, although it should be noted that design contributions are expected from the contractor during the selection process. It is my opinion that the contractors design contributions would be more useful if access to the Business Analysis document were made available to all vendors. Also designs resulting from two different groups of developers with


Page 33

access to different levels of information must surely cause problems when trying to deliver the right solution. Once the project has been assigned to a contractor, in-house developers create the Testing and Evaluation document, which the deliverable must satisfy before it is accepted. Unusually the Testing and Evaluation document is only made available to the contractors if specifically requested. The Web development process for the outsourcing organisation in question is more mature than that of those used by the contractors in this survey, with at least some emphasis on Analysis and Testing. However there are still a number of key factors missing from what I would consider to be a development process that at least tries to address all the required stages for a successful Web development process. Keeping all of the Business Analysis documentation hidden from the contractors is a bad idea. While commercially sensitive information may have to remain inhouse, the information that is in the Business Analysis document that can be made available to the contractors should be. This would allow contractors to better advise on the refinement of design and requirements specification given a clearer understanding of the goals or problems to be addressed, it would also allow contractors to improve their evaluation/validation process of the project deliverables. Again I feel the mixing of Requirements and Design information to be a bad idea (see 2.3 question 15 for a more detailed discussion). Also without access to the Business Analysis document the contractors design input is not based on the same information as is the outsourcers design decisions. If contractors do not have access to the Testing and Evaluation document they must needlessly find it more difficult to plan the project’s development. Like the contractors, no mention is made of maintenance in the development process, this can only lead to the conclusion that guarantees of quality and longevity of the deliverables are poorer than they need be. Question 15: Once executive approval is obtained for a particular project, the Business Analysis Stage is completed and the initial Requirements and Design documentation produced. Due to time constraints most projects are sent out to a closed list of vendors, who are approved approximately every two years. Selection is then carried out through written proposals and presentations to a project specific panel. The panel then confer and commission the project. Question 16: Both interviewees mentioned comparing the Requirements and Design Brief to the deliverables through the Testing and Evaluation document. Question 17: The answers to question 17 varied too much to draw any valid conclusions, with the exception that we suspect that one of the interviewees was not being completely truthful. Question 18: Question 18 was not applicable to interviewee 12 as he stated that all projects resulted in a deliverable. The main reason given by interviewee 13 for a Web engineering project failing is due to the lack of proper business analysis at the beginning. This is clear confirmation that poor Business Analysis procedures are one of the main reasons for projects not resulting in deliverables. Question 19: Both interviewees answered yes when asked if they insisted on a clearly defined


Page 34

and documented development process for the building of Web applications. See 3.3 questions 14, 15 and 16 for a more detailed discussion of this process. Question 20: Interviewee 12 mentioned the documentation that is produced to be the major success of the organisation’s Web development process. Interviewee 12 also took this opportunity to mention the difficulties encountered when trying to define the Requirements and Design Brief. Interviewee 13 stressed the importance of every member of the development team understanding that the project is to solve business goals and is not about technology. In particular interviewee 13 stressed the need to focus on functionality and business needs and to avoid being seduced by Web technologies. Question 21: When asked what features of their Web development process the interviewees considered problematic, interviewee 12 mentioned the inability to change requirements. Upon further investigation I discovered that once a projects scope had been signed off, when the project is commissioned, no alterations to the requirements are allowed. Interviewee 13 mentioned industrial inertia, ownership of deliverables and inability of the process to effectively manage business and IT successfully. Question 22: Since both interviewees use a Web development process this question was not applicable. Question 23: Question 23 was designed to highlight any tools that were being used consistently throughout the industry at specific stages in the development process or for specific Web development tasks. Unfortunately the question did not work very well with either of the interviewees, who were unsure of my descriptions of the identified stages. However the answers showed some consistent tool use with contracting organisations where Outlook Express was the email client used by both interviewees. Question 24: Technical, managerial and creative design developers seem to be in conflict with business and domain experts. Both interviewees acknowledged this conflict as being a major problem. Interviewee 12 mentioned the frustration business and domain experts encounter when they request changes to the Brief (Requirements and Design document). The Web development process does not allow any changes once the Brief has been signed off when the project is commissioned. The business and domain experts are then faced with two choices. They can either refuse to sign off the project deliverables, or sign off the project deliverables, then start a new project to make the requested changes. Refusal to sign off is not really a viable option as the IT Manager can sign off all projects without business or domain experts approval, and does so when business and domain expert’s refuse to do so. Thus the IT Manager guarantees 100% success rate and ensures a flawless record during the Quality Assurance assessment. Interviewee 13 also mentioned the inability to change the scope of the project once the project has been commissioned as being a major problem. Question 25: Interviewee 12 resolves the conflicts mentioned in question 24 by signing off on projects without client’s approval or by educating the client, and starting another project to properly achieve business goals. Interviewee 13 is trying to introduce technical and design architectures to help ease development conflicts.


Page 35

Question 26: According to interviewees 12 and 13 a project is seen to be successful if it delivered on time and to budget. Again there is no mention of verification or validation of the project deliverables to the Brief and Business Analysis stages respectively.


Page 36

4. In-house 4.1 Design Section 4 of this survey was conducted using two Web developers from a financial services sector company with interest’s worldwide. Both developers are based in Central Scotland with responsibilities on projects that target audiences throughout the European Union. Each interview was conducted following the questionnaire in Appendix C. The results of the interviews are presented anonymously in order to assist in gaining information that is of a commercially sensitive nature.

4.2 Results NOTE: The numbers 14 and 15 will refer to the interviewees in the order they were questioned. Question 1 - What is your current job title? Answers 14: Project Manager (software engineering background) 15: WebZone Manger (business management background) Question 2 – What are your responsibilities within your organisation? Answers 14: Responsible for the delivery of solutions on time, to budget, that meet project requirements. 15: Business and Line manager for WebZone. Organise virtual teams for each project. Note we use a matrix structure for populating teams. Question 3 – How many developers (inclusive of third party or external developers) make up your average Web development team? Answers 14: At the moment 20 people make up the WebZone team. Approximately 6 people per project (for example the Intranet project). 15: 25 people in total make up the WebZone team. 2-5 developers on an average team although this figure could rise to 30 on large projects. Question 4 – What is the average age of a member of your Web development team? Answers 14: 27 years. 15: 24 years. Question 5 – What percentage of males and females make up the people who are involved in a typical Web engineering project? Answers © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 37

14: Male: 50% 15: Male: 65%

Female: 50% Female: 35%

Question 6 – Classify the types of people involved in a typical Web engineering project in your organisation! Answers 14: Software Engineer: 4 Creative Designer: 2 Team Manager: 4 Business Expert: 0 Domain Expert (content provider): 0 Other: 2 people testing, 4 Research and Development technologies, 4 coders. 15: Software Engineer: 0.5 Creative Designer: 1 Team Manager: 0.2 Business Expert: 1 Domain Expert (content provider): 0.3 Other: 0 Question 7 – What is the average length of a Web development project, from inception to first delivered working system? TICK AS APPOPRIATE! Answers 14: 1-4weeks _____ 15: 1-4weeks _____

1-3months__√_ 3-6months______ 1-3months__√_ 3-6months______

6+ months______ 6+ months______

Question 8 – How often do your Web development projects run over time? Answers 14: 0-24% of the time. 15: 0-24% of the time. Question 9 – What are the reasons for projects running over in terms of time? Answers 14: Scope creep and business indecision. 15: It takes too long for approval and implementation. Question 10 – How often do Web development projects run over budget? TICK AS APPROPRIATE! Answers 14: 1-25% of the time. 15: 1-25% of the time. Question 11 – What are the reasons for projects running over budget? Answers 14: Unforeseen costs (e.g. having to buy additional encryption modules). Also existing infrastructure proves unsuitable. 15: Generally business requirements are not solid.


Page 38

Question 12 – Do you use any outside organisations to assist in the development of your Web projects? If so who are they and why do you use them? Answers 14: YES. TATA Consultancy Services for extra developers. Buchanan International for security audits and ‘ethical hacking’. 15: YES. TATA Consultancy Services for extra developers, prefer to manage developers in-house as opposed to outsourcing

Question 13 – What internal departments play a role in the development of your Web engineering projects? What do these departments contribute to projects? Answers 14: Sales & Services win the business and manage the customer relationship. Program Office record metrics for the projects. Testing & Integration verify the project deliverables and ensure that they operate smoothly with existing IT infrastructure. Service Delivery manages the Web-based systems. Planning & Consulting check architectures. 15: Electronics & Telephonics, Products & Processes, Marketing Services, Service Delivery, E-enablement (1 person), Payments. Question 14 – Do you ever find yourself short of development expertise in a Web development project? Answers 14: YES. Lack of people keeping their eye on new technologies, e.g. Java, XML and WAP. 15: YES. Technical skills generally pull people in from TATA. Question 15 – How many projects proposed result in a delivered system? Answers 14: 51-75% 15: 51-75% Question 16 - What are the prime reasons for projects not resulting in a delivered Web system? Answers 14: Lack of budget, unable to justify development costs! 15: Lack of Funding. Annual budget system restricts development. Question 17 – Do you have a well-defined and documented development process for building Web sites? How does this process begin and end? © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 39

Answers 14: YES. We use Dynamic Systems Development Method DSDM™ (http://www.dsdm.org) for all Web-based development projects. DSDM™ is a Rapid Application Development approach, unfortunately there are very few people within the organisation who are comfortable with it. 15: YES. Use DSDM™ after receiving operational vision. Stop when business clients are satisfied, and then do post implementation review. Difficult to do verification and validation as this takes 6 months on non web-based projects, and we often just have a couple of weeks. Question 18 – What features of your Web development and design process do you consider successful? Answers 14: DSDM™ allows close working with business people, bringing business on board. The iterative and incremental approach allows for market pressures enabling soft launches of Webbased systems. 15: Project Definition Workshop that takes the form of a structured meeting. Question 19 – What features of your Web development and design process do you consider problematic? Answers 14: Managing scope is still difficult under DSDM™, also business people rebel against existing techniques, such as exhaustive testing and integration procedures as this takes a great deal of time. 15: Lack of understanding of traditional software infrastructure, and poor end to end testing. Question 20 – If you do not use a Web development process, why not? Answers 14: Not applicable. 15: Not applicable. Question 21 – What software tools do you use when developing a Web project, what tasks do you use them for, and at what stage in the development are they used? Answers 14: PROJECT DEFINITION: Microsoft PowerPoint, as it forces business people to bullet point! DESIGN: Not aware of any design tools. IMPLEMENTATION: Domino, Dreamweaver, Flash, Apache, CORBA, SOAP, IIS, ASP and MQ as middleware solution to legacy infrastructures on IBM and Compaq Mainframes. TESTING: Endeavour, Test Director. MAINTENANCE: 15: Not answered. Question 22 – What types of conflict arise between different developers in a Web development project?


Page 40

Answers 14: Undercurrent of people who denigrate graphic design. Also major conflicts between traditional IT developers and Web developers. 15: Often traditional IT developers over engineer, do not know when to get off the development life cycle. In addition one of the developers is blind and complains regularly about poor support for disabled users. Question 23 – How are conflicting views resolved between different developers in a project? Answers 14: Get people working closer together. Educate people that Web development is different from tradition IT development. Also a Web team breakfast. 15: Conflicts are resolved by line managers. Question 24 - What mechanisms to you employ to measure the success of a project? At what stage in the development process do you employ these mechanisms? Answers 14: Program Office metrics, function point analysis. Measure efficiency of developers. Questionnaire for receiving customers, we did this with a sample of six customers for our new WAP site. 15: Post implementation testing.

4.3 Analysis Questions 1 and 2: Both interviewees are involved in managing Web-based teams. Interviewee 14 is heavily involved in managing individual projects on a day to day basis, where as interviewee 15 is more involved in managing the entire WebZone team. Question 3: Both interviewees stated that the average Web development team is less than 7 developers. Interviewee 15 observed that this figure could rise to over 30 developers depending on the characteristics of the project. These figures are consistent with the size of Web development teams in contracting and outsourcing organisations involved in this survey. Question 4: The average age of one of their Web developers is 26 years. This figure is also consistent with the average age of developers in a contracting and outsourcing organisation. It should also be noted that the answers from developers within the same organisation varied. Question 5: According to the answers to question 5 approximately 58% of Web developers are Male, 42% being female. Question 6: The answers to question 6 varied too much to draw any valid conclusions. Although it should be noted that interviewee 15’s breakdown of classification of developers showed domain and business experts to be almost twice the number of mangers and software engineers. Question 7: The average development phase of a Web-based project is between 1 and 3 months.


Page 41

Again this figure is consistent with contractors and outsourcing organisations. Question 8 and 10: Although both interviewees answered the same to questions 8 and 10, I suspect that both interviewees were giving politically approved answers and therefore these answers should be treated with some scepticism. However according to their answers 0-25% of projects run over in terms of time, and 0-25% of projects run over in terms of budget. Question 9: Poor requirements capture, changes in specification, inability to give project approval, inability to estimate development time and business indecision were the prime reasons for projects running over time. These problems seem to confirm the answers given by the contractors and outsourcers highlighting the problems associated with analysis and requirements capture in the Web development processes. Question 11: Lack of solid business goals, unforeseen costs and difficulties integrating Web systems with existing business systems were the main reasons for projects running over budget. Again the problems mentioned here seem to confirm the answers given by the contractors and outsourcers highlighting the problems associated with analysis and requirements capture in the Web development processes. Question 12: The organisation in question temporarily hires extra developers rather than outsource projects as they find it easier to control development. The only major outsourcing of contracts occurs in relation to security checks, e.g. ‘ethical hacking’. This is an interesting point. Could it be that rather than outsourcing, bringing in extra developers is a better way to manage Web development projects? Clearly further investigation is required to answer this question. Question 13: The interviewees mentioned nine other departments that are involved in Web development projects. Note each interviewee mentioned five departments, only one department was found in both interviewees’ lists. Clearly there are a large number of other departments involved in Web development outside the Web development team. In my opinion the large number of internal departments involved would suggest that a successful Web engineering process would have to provide mechanisms to communicate and influence business and domain models. Question 14: Both interviewees mentioned a technical skills shortage amongst their Web development teams. Question 15: According to both interviewees 51-75% of all proposed projects result in a delivered system. Like question 8 and 10 these answers should be treated with some scepticism. Question 16: Lack of funding and a culture of annual spending were seen as the prime reasons for projects not resulting in a delivered system. Question 17: The organisation in question uses DSDM™ to develop Web-based systems. DSDM™ is a high level RAD approach independent of any particular technique. Like a lot of prototyping approaches it suggests close integration with business clients. However this technique conflicts with other internal development methods and conflicts arise between other IT developers and Web application developers. Like a lot of prototyping approaches when practised in industry on projects with short time scales and strict budgets, DSDM™ has a tendency to © University of Glasgow 2001, Department of Computing Science Technical Report R-2001-79.

Page 42

favour delivering what the client wants rather than what the client needs. Thus ignoring end users as the mechanism for verification and validation of the project’s deliverables, and using the client instead. Interviewee 15 states that the development process stops when the client is satisfied. In addition interviewee 15 acknowledges the difficulty in doing verification and validation as this is normally done by Testing and Integration department and takes approximately 6 months on more traditional in-house software development projects. Question 18: Close integration with business clients and structured meetings with business clients were seen as the major advantage to using DSDM™. However close integration with business clients without focus on the end users can be very dangerous resulting in deliverables that satisfy the client’s desires rather than the client’s business goals. Neither interviewee mentioned end users as a mechanism for validating projects. Question 19: Managing Scope (Requirements and Design) and poor understanding of the difference between a prototype and fully functional working system were seen as the major problems with using DSDM™. These problems are symptomatic of an approach where verification and validation are measured buy client’s desires rather than client’s needs. Also nontechnical developers often have difficulty in distinguishing the difference between a prototype and a fully functional working system. Often once a client observes a prototype that has the desired user interface functionality modelled, they often exert extreme pressure on the technical developers to quickly deliver the system. Question 20: Both interviewees use a Web development process, therefore this question was redundant. Question 21: One of the most interesting answers throughout the whole survey was given by interviewee 14, who insists on using only PowerPoint as a tool during Project Definition and Design stages. This is because it forces clients to bullet point what there problems are and bullet point how they see them being solved using Web technologies. Question 22: Conflicts arise between Web engineers and Web Managers, between Web engineers and Business Experts, between Web engineers and Graphic Designers and between Web developers and traditional IT development staff. These were respectively observed to be a result of over engineering, lack of understanding of the need for design skills, and the lack of a structured and coherent approach to delivering Web-based systems. Question 23: Conflict resolution is seen to be the domain of line managers and in some cases between developers at a regular team breakfast. Question 24: Projects are considered a success if the client is happy with the deliverable, and also if it passes testing procedures. In only one occasion have actual users been polled to find out if the deliverable is successful. The sample totalled six users evaluating the system for 1 week.


Page 43

5. Aknowledgements We would like to take this opportunity to thank all the individuals and organisations that participated in this survey. In addition we would like to thank Jim Devine, Malcolm Atkinson and Matthew Chalmers for their comments.


Page 44

6. References [1] H. D. Benington, 1956, ‘Production of Large Computer Programs’, Proceedings Symposium on Advanced Programming Methods for Digital Computers, 28-29 June 1956, Republished in Annals of the History of Computing, Oct. 1983, pp. 350-361. [2] B. W. Boehm, 1988, ‘A Spiral Model of Software Development and Enhancement’, IEEE Computer, Volume 21 5, 1988, Page(s): 61-72. [3] F. Coda, C. Ghezzi, G. Vigna, & F. Garzotto 1998, ‘Towards a Software Engineering Approach to Web Site Development’, in Proceedings of the 9th International Workshop on Software Specification and Design, 20th International Conference on Software Engineering, Kyoto (Japan), April 1998, Page(s): 8-17. [4] J. Conallen, 1999, ‘Building Web Applications with UML’, Addison-Wesley, 1999. [5] Dynamic Systems Development Method (DSDM) Limited, DSDM homepage, http:// www.dsdm.org/ [6] ‘Electronic Commerce Site Deployment, Based on INTERSHOP enfinity™, Sample Project Plan and Costs’, INTERSHOP Communications, 2000, http://www.intershop.com/ [7] M. Gaedke, H-W. Gellersen, A. Schmidt, U. Stegemüller and W. Kurr, 1999, ‘Object-oriented Web Engineering for Large-scale Web Service Management’, Proceedings of the 32nd Hawaii International Conference on System Sciences, 1999, Page(s) 1-9. [8] S. Hansen, S. Y. Deshpande and S. Murugesan, ‘A Skills Hierarchy for Web Information System Development’, Proceedings of the First ICSE Workshop on Web Engineering, 16-22 May 1999, http://fistserv.macarthur.uws.edu.au/san/icse99-webe/ICSE99-WebE-Proc/default.htm [9] S. Hansen, S. Y. Deshpande and S. Murugesan, ‘Web Engineering: Beyond CS, IS and SE An Evolutionary and Non-Engineering View’, Proceedings of the First ICSE Workshop on Web Engineering, 16-22 May 1999, http://fistserv.macarthur.uws.edu.au/san/icse99-webe/ICSE99WebE-Proc/default.htm [10] I. Jacobson, G. Booch, J. Rumbaugh, 1999, ‘The Unified Software Development Process’, Addison-Wesley, 1999. [11] E. Mendes 2000, ‘Investigating Metrics for a Development Effort Prediction Model of Web Applications’, Proceedings of the 2000 Australian Software Engineering Conference, 28-30 April 2000, Page(s): 31-41. [12] E. Mendes & E. Counsell, 2000, ‘Web Development Effort Estimation using Analogy’, Proceedings of the 2000 Australian Software Engineering Conference, 28-30 April 2000, Page(s): 203-212.


Page 45

[13] S. Murugesan, Y. Deshpande, S. Hansen and A. Ginige, 1999, ‘Web Engineering: A New Discipline for Development of Web-based Systems’, Proceedings of the First ICSE Workshop on Web Engineering, 16-22 May 1999, http://fistserv.macarthur.uws.edu.au/san/icse99-webe/ ICSE99-WebE-Proc/default.htm [14] R. S. Pressman, 1998, ‘Can Internet-Based Applications Be Engineered?’, IEEE Software, Volume: 15 5, September/October 1998, Page(s): 104-110. [15] R. S. Pressman, 2000, ‘What a Tangled Web We Weave’, IEEE Software, Volume: 17 1, January/February 2000, Page(s): 18-21. [16] M. Ramsay and J. Nielsen, 2000, ‘WAP Usability Déjà Vu: 1994 All Over Again, Report from a Field Study in London, Fall 2000’, Nielsen Norman Group, December 2000, Page(s): 4. http://www.nngroup.com/reports/wap [17] D. J. Reifer, 2000, ‘Web Development: Estimating Quick-to-Market Software’, IEEE Software, Volume: 17 6, November/December 2000, Page(s): 57-63. [18] W. W. Royce, 1970, ‘Managing The Development of Large Software Systems: Concepts and Techniques’, In WESCON Technical Papers, v. 14, pages A/1-1-A/1-9, Los Angeles, August 1970. WESCON. Reprinted in Proceedings of the Ninth International Conference on Software Engineering, 1987, pp. 328-338. [19] S. Tilley, 2001, ‘Web Site Evolution’, Scott Tilley’s Homepage, http://mulford.cs.ucr.edu/ stilley/research/wse/index.htm [20] S. Ward, P. Kroll, 1999, ‘Building Web Solutions with the Rational Unified Process: Unifying the Creative Design Process and the Software Engineering Process’, Rational Software Corporation, July 1999. http://www.rational.com/ [21] WWW7, 1998, Proceedings of the First International Workshop on Web Engineering, WWW7 Conference, Brisbane, April 1998. http://btwebsh.macarthur.uws.edu.au/san/WebE98


Page 46

Appendix A - Contractor Questionnaire Note: The questions in bold were specific to the contractor questionnaire, the questions in italics were common to the contractor, outsourcer and in-house questionnaires. Question 1 - What is your current job title? Question 2 - What are your responsibilities within your Web development team? Question 3 - How many developers (inclusive of third party or external developers) make up your average Web development team? Question 4 - What is the average age of a member of your Web development team? Question 5 - What percentage of males and females make up your average Web development team? Question 6 - Break down the number of developers (inclusive of third party or external developers) in an average Web development team into the following roles! Software Engineer:__ Team Manager:__ Domain Expert:__

Creative Designer:__ Business Expert:__ Other:__

Question 7 - What is the average length of a Web development project, from inception to first delivered working system? 1-4weeks__

1-3months__

3-6months__

6+ months__

Question 8 - How often do your Web development projects run over time? Question 9 - What are the reasons for projects running over time? Question 10 - How often do your Web development projects run over budget? Question 11 - What are the reasons for Web engineering projects running over budget? Question 12 - Do you ever find yourself short of development expertise in a Web development project? Question 13 - How many Web projects proposed result in a delivered system? Question 14 - What are the prime reasons for projects not resulting in a delivered Web system? Question 15 - Do you have a well-defined and documented development process for building Web engineering projects?


Page 47

Question 16 - What features of your Web development and design process do you consider successful? Question 17 - What features of your Web development and design process do you consider problematic? Question 18 - If you do not use a Web development process, why not? Question 19 - What software tools do you use when developing a Web engineering project, what tasks do you use them for, and at what stage in the development are they used? Question 20 - What types of conflict arise between different developers in a Web development project? Question 21 - How are conflicting views resolved between different developers in a project? Question 22 - What mechanisms to you employ to measure the success of a project? At what stage in the development process do you employ these mechanisms?


Page 48

Appendix B - Outsourcer Questionnaire Note: The questions in bold were specific to the outsourcer questionnaire, the questions in italics were common to the contractor, outsourcer and in-house questionnaires. Question 1 - What is your current job title? Question 2 – What are your responsibilities within your organisation? Question 3 – How many people within your organisation are involved in a typical Web engineering project? Question 4 – What is the average age of people, within your organisation, who are involved in a typical Web engineering project? Question 5 – What percentage of males and females make up the people who are involved in a typical Web engineering project? Question 6 – Classify the types of people involved in a typical Web engineering project in your organisation! Software Engineer:__ Team Manager:__ Domain Expert:__


Question 7 – What is the average length of a Web development project, from inception to first delivered working system? 1-4weeks__

1-3months__

3-6months__

6+ months__

Question 8 – How often do your Web development projects run over time? Question 9 – What are the reasons for projects running over in terms of time? Question 10 – How often do Web development projects run over budget? Question 11 – What are the reasons for projects running over budget? Question 12 – What are the primary reasons for outsourcing a Web engineering project? Question 13 – What criteria do you look for in vendors tendering for a Web engineering project? Question 14 – Describe the process of putting a Web engineering project out to tender? Question 15 – Describe the process of selecting a vendor for a Web engineering project!


Page 49

Question 16 – Describe the process of accepting the final deliverable of an outsourced Web engineering project! Question 17 – How many proposed projects result in a delivered system? Question 18 – What are the prime reasons for a project not resulting in a delivered Web system? Question 19 – Do you insist on a well-defined and documented development process or methodology for building Web sites? Question 20 – What features of your Web development and design process or methodology do you consider successful? Question 21 – What features of your Web development and design process or methodology do you consider problematic? Question 22 – If you do not use a Web development process or methodology, why not? Question 23 – What software tools do you use when you are involved in a Web engineering project, what tasks do you use them for, and at what stage in the development are they used? Question 24 - What types of conflict arise in your organisation and with the vendor in a typical Web development project? Question 25 – How are these conflicts resolved? Question 26 – What mechanisms do you employ to measure the success of a project? At what stage in the development process do you employ these mechanisms?


Page 50

Appendix C - In-house Questionnaire Note: The questions in bold were specific to the in-house questionnaire, the questions in italics were common to the contractor, outsourcer and in-house questionnaires. Question 1 - What is your current job title? Question 2 – What are your responsibilities within your organisation? Question 3 – How many developers (inclusive of third party or external developers) make up your average Web development team? Question 4 – What is the average age of a member of your Web development team? Question 5 – What percentage of males and females make up the people who are involved in a typical Web engineering project? Question 6 – Classify the types of people involved in a typical Web engineering project in your organisation! Software Engineer:__ Team Manager:__ Domain Expert:__


Question 7 – What is the average length of a Web development project, from inception to first delivered working system? 1-4weeks__

1-3months__

3-6months__

6+ months__

Question 8 – How often do your Web development projects run over time? Question 9 – What are the reasons for projects running over in terms of time? Question 10 – How often do Web development projects run over budget? Question 11 – What are the reasons for projects running over budget? Question 12 – Do you use any outside organisations to assist in the development of your Web projects? If so who are they and why do you use them? Question 13 – What internal departments play a role in the development of your Web engineering projects? What do these departments contribute to projects? Question 14 – Do you ever find yourself short of development expertise in a Web development project? Question 15 – How many projects proposed result in a delivered system?


Page 51

Question 16 - What are the prime reasons for projects not resulting in a delivered Web system? Question 17 – Do you have a well-defined and documented development process for building Web sites? How does this process begin and end? Question 18 – What features of your Web development and design process do you consider successful? Question 19 – What features of your Web development and design process do you consider problematic? Question 20 – If you do not use a Web development process, why not? Question 21 – What software tools do you use when developing a Web project, what tasks do you use them for, and at what stage in the development are they used? Question 22 – What types of conflict arise between different developers in a Web development project? Question 23 – How are conflicting views resolved between different developers in a project? Question 24 - What mechanisms to you employ to measure the success of a project? At what stage in the development process do you employ these mechanisms?


Page 52