Agile Methods handling Offshore Software Development ... - IEEE Xplore

Agile Methods handling Offshore Software Development Issues Muhammad Faisal Nisat’ , and Tahir Hameed’ Technology Wisdom, Pakistan 2 Higher Education Commission, Pakistan [email protected], [email protected]

Abstract Agik Methods ~ L W Eemerged cis a new set of scftware devcEopment processes and praclices, that cater veqirirements change by delivering rhe safiwnve in mirltiple small reeleases, increasing the client coulidence and enables himiher to commuiiicate the bmirirss reqrriremenfs change at higher pace. 011the other hand, Technoiogy World is switching la ojfihorz sqfiware d e w l o p e n t rapidly due lo globir! wonoiitic wc~sons. In !his yoper, I introduce the Agile Melhods. Oflshore Development achiiies and issues. ‘4fi.r that I elaborate each qf the qlfihore issiies .finding out how the hest practices proposed by Agile Methods oddress the issue. The recornmenddons aye bused upon experiences gathered ,from several small to medium sized projects qf 'distributed and web iippiiwfiot7.r ’ domain. execuied by ‘Technology Wisdom’ J1 ci/ and a number af veseavch papers.

1. !ntroduction Traditionally software development process is tcrmcd as an enginecred, planned and rcpeatabte activity. But in recent years gbbalization and emergence of e-business, has changed thz traditional processes. In past, software development projects targeted primarily the implementation of well-known business processes. On completion, onIy those projects were considered suc.cessM which implemented all the given requirements correctly, within time and budget. The emphasis was heavily placed on the processes and documzntation. This scenario puts the real player of thc projcct behind focus, which is the client.

0-7803-8680-9/04/S20.00 02004 IEEE.

Tlic development efforts circle around the planned design and processes, whereas these should be building client satisfaction and improving hisihcr business, These methods can also be termed as plandrivcn methods (like waterfall, spiral, etc.) applied to software projccts for the last 3 decades. Yet the software projects failurc rate i s too high, according to TT Cortex Conclusion [XI, based on 5 survey reports, stating “7 out of I O IT projects ‘fail’ in some respect”. So, thc implementation of processes and plan doesn’t guarantee the software projcct succcss. This is mainly due to the rapid change in busincsses needs to stay ahead and along with the market competition and forces, which in tu& raises the need to improve the software development methods. From mid of 90s several individual effurts in different organizations were made to fulfill the nced of improvement in the traditional plan-driven methods. This resulted in new flexible development mcthods. In February 2001 these leading experts From the industry formed the ‘Agile Alliance’ [ I ] and presentcd a manifesto for software development [2]. The agile manifesto gives 4 values and 12 principles to follow. On the other hand, the world is moving fast towards a global economy as weli as businesses are taking advantage of cheaper facilities and talented work force. Software Industry is practicing offshore software development, from about half decade, using the same plan and processes driven approach. In recent years the offshore trend has increased and more companies are switching to offshoie development, Forrester Rescarch forecasts that 3.3 million IT related jobs will move out of USA by 2015 [73. Based on the experience of Agile Method’s practices for small to medium size projects, at Technology Wisdom, Pakistan 1161. Here 1 address these two emerging trends, idcntifying issues and propose solutions, OR how Offshore Development can take advantage of Agile Methods, for maximizing the benefits,

INMTC 2004

Table 1: Typical offshore activities division

1.1. Agile Methods? -

The definition says: ‘Agife methods are the mix of iterative dcvetopmcnt, with set of best practices to cope software changes and increase client satisfaction.’ The Agile manifesto [2] places greater value on ’working software’ over comprehensive documentation, and emphasizes simplicity: ‘maximizing the amount of work not done‘. Its inventors define, Agile Methods as: Iterative, (deliver full system at beginning and then change the functionality of each subsystem with several new releases.) Incremental, (system is portioned into small subsystems by hnctionafity; new functionality added with new releases.) Self-organizing, (team has the autonomy to organize itself to best complete the work items.) Emergent, (Technology and requirements are allowed to emerge through the product development cycle) [3]. All ‘Agile Methods’ value: 1. Individuals and interactions over processes and tools. 2. Working software over comprehensive documentation. 3. Customer collaboration over contract negotiation and, 4. Responding to change over following a plan. and follow 12 principles of Agile Manifesto [Z]. ‘Agile Methods’ has many different custom implementations, of which the best-known methods are: Extreme Programming (XP) [ 5 ] , Scrum, Feature Driven Development (FDD), Crystal, Avaion E3, and Agile Modeling. 1.2. Offshore Software Development? To understand offshore software development, lets divide the software project activities into two parts, front office activities and back office activities as listed in Table 1. Please note that activities division mentioned in Table 1 is a typical case, and any activity can be transferred from front to back office or vice versa. The. important point in offshore model is that, the back office activities are carried out on a distant location, to achieve cost and quality benefit. The back office either owned by the same front office company or by a partnered outsourcing company.

418

.

Front Office Activities - Project Inception Requirements Specs - Requirements Change Management - Analysis - Integration Testing - Deployment

- Implementation - Testing - Maintenance -

Change lmplementation

- Project Monitoring 2. Current Industry State Let us consider first the current state of the industry. Agile methods are new in thc industry but are now becoming very popular, because of the promising software delivery and higher rates of client satisfaction. Similarly the offshore inodel has become the most widely used modcl to reduce cost and increase quality. A BCS forecast rcport [7] states, by year 2006, 100% of the European companies will be tising offshore outsourcing for their IT projects in some respect. In industry, currently there are two different opinions on this topic. 1. Agile Methods is a mismatch with offshore development. [IO] 2. Agile Methods is a best fit for offshore if modified carefully [9,13,14,15]. The given major reasons for the first opinion that negates the possibility of using agile methods for offshore development are: a. Agile methods demand high communication with end client throughout the project and b. Offshore project already involves high risks of security, decreased project visibility, configuration management and integration. Thus it is believed that it requires better planning and execution with increased documentation. This i s contradictory to agile methods. Through my studies and experience, I have found that the first opinion is based on two factors: 1. The theory that thc usc of agile methods can slow down the offshore shift in USA and other developed countries, which decreased the job opportunities in these countries. 2. Apparently Agile Methods are not felt its tit for offshore development due to some of its principles. Let me analyze here, somc of the major issues of offshore software development and discuss how the practices of Agile Methods heIp in solving them.

Table 2, shows the major challenges faced by offshore development.

1 Communication gap Transfer of business domain 3 Decreased projcct visibility 4 Configuration Managcment 5 Disconnection 011 prqject estimations 6 Client business security 7 Documents maintenancc and synchronization 8 Cultural differences 2

3. Challenges, Issues & Solutions 3.1. Communication Gap According to Matt Simons [9], “Due to time zonc differences between westem business centers (like US, UK) and many of the major offshore development sites in India and China (and now Pakistan), there are very few hours in the day when project participants (onshore and offshore) are in both office locations at thc same time. This factor, as well as the current cost and quality of telecommunications, serves to significantly decrease the volume of communication between offshore and onshore teams.” In short, the distance, the time zone differcnces and communication cost increases this gap; In Agile Methods the major focus is on regular face-to-facc communication and rest of all the principles revolve around it. Hence increasing the demand for communication. We can see that the coinmunication gap seems to be a mismatch with Agile Methods high demand of communication, but in practice I look at it as a solution like: “Offshore Development requires more communication” “Agilc Methods provide more communication”

Ifwe Iook at it this way, it appears to be the best fit. Now the question is how to mod$ the Agile Methods, to reduce the gap. Practically it is not possible to provide face-to-face communication in offshore, so that we have to devise other methods; Following are some of the best practices to handle this issue: 1. Having onlinc voice or video chat session at least once in a week.

Having a daily 15 to 30 minutes progress discussion with onshore personnel over any instant messenger. 3. ‘Just Right Documentation’ [9]; Do the documentation only what is required and when it is required. Small documents do not take much time to develop and disseminate. 4. Use regular ‘builds’ (almost daily) to get feedbacks. Do not deliver working version daily, but at least a version that is integrated and show some progress. 5. lncreased use of tools like Wiki, CVS, IM, Discussion Boards [6,9,14,15] 6. For large projects: Increase the initial time spent onshore by an analyst to define initial scope for the architecture and develop sufficient understanding with the client 1121. 7. Send ambassador from offshore to onshore and vice versa for some time period, to understand environment on both sites [6]. This is workable only for large budgeted projects. The short releases and feedback cveles along with focus on ‘working software’ itself decrease the communication gap. 2.

3.2. Transfer of business domain The major issue with all the offshore projects is the transfer of ‘business domain’ to the implementation team. Development of an accounting system, sates, inventory or banking system is trivial as their knowledge is commonly shared and known. The issue arises with the special customized software products that need to be impIemented according to the uncommon business domain. It was assumed that offshore development couldn’t handle specialized and critical projects. But now several companies are practicing to shift their critical projects to offshore, some SUCCGSS stories are; General Motors outsourced the development of its wcb-based ‘New Owner Center’ project. Similarly Caterpillar Financial Services outsourced their core financing system [137. These examples and some other prove that offshore development can handle critical and specialized projects. Some good practices found in Agile Methods implementation to handIe this issue are: 1. Using a proxy client: Proxy client is a business analyst who might not be businessperson himself but has interest, can understand and communicate the business domain and is well versed with technology. In our project for a German client, an offshore proxy client who has a good understanding of the business process in

41 9

Germany (but not about the business itself) helps us a lot in resolving several issues efficiently. 2. For large project the method of sending ambassador from one site to another also helps very much in acquiring of business domain [6]. Please remember this is a costly practice. 3. Using low but standard documentation practices, like UML [15]: A simple one-page activity diagram explains more then 10 pages of documentation

3.3. Decreased project visibility Even in on-site projects, it is often difficult for project managers and investors to get an accurate sense of project progrcss. Tn many o f the occasions, during later stagcs of the projcct, managers and investors are surprised by the high and unpleasant differences in respect of “%complete” reports and actual project progress. So measuring project progress is a problem, when the project team is collocatcd. We can imagine that it becomes worse in the case of offshore, where project team is far apart from the managers and investors. .4gile Methods focuses on delivering the ‘working software’ in shorter iterations, which is the best milesto~ieto measure the progress aid tasks left. Shoitening the project iterations increases the delivery of the project-working model to the client and project managers, giving them the best picture, which no other document or communication channel can provide. Thus all the mistakes in project estimations can be corrected very soon, and project inanagcrs can get the time to take any necessary corrective actions earlier.

3.4. Configuration Management ‘4s discussed by Matt Simons [9]: “Bringing it all togethcr, for implementation in the production environment is one of the most difficult parts of any software project. Many teams that have built components offshore have been beset by problems when the time came to integrate the offshore and onshore pieces into a working system”. I have experienced the headaches of integrating large and mid sized software, and found that integration always takes extra time in a projcct release, if it is not handled during the dcvelopment. In the projccts where the integration of different tested modules is done at the end, we always had problem meeting the schedule due to last minute integration bugs. Since we are using the ‘Continuous Integration’ practice of

420

the Agile Methods, we are saving lots of our last minute efforts. Agile focuses on “Continuous Integration”, a best practice that is appreciated by majority of the experts [9,6,15,14]. Matt Simons states: “If practiced with discipline, contimous integration-the process by which developers integrate their code and build the entire system whenever they have made changes-should reduce or even eliminate configuration-management issues.” The industry also provides several high-tech tools to support continuous integration, some of the most popular tools in use are. Maven, Chcck Style, Ant, Cruise C.ontro1, Jira, CVS, Subversion, Perforce

[L1,14,15].

3.5. Disconnection on project estimations All of us, who are involved in software development, know that points of view of a software project, by programmer, managers and investors are different. Each of them estimates the project based on the information provided by the other, including some fbdge factor. In offshore model, we knaw that the development team is never going to meet the managers and invcstors face-to-face. So the deviation between planned and actual project cstirnates can vary largely. Hence assessing the accuracy and reliability of estimates in offshore also becomes difficult. Most of the agile methods emphasize the importance of having the people “The one who will perform the work shall estimate the work”. Ideally, agilc project plans are built by rolling up the

aggregate estimates on a set of features, where the development team has provided the estimates. Agile methods enablc the trust in development team, and emphasize on developers feedback to make project time estimations [9,11]. This also increases the commitment level of the team and makes them responsible to follow the self-defined timeline. I have found this a very helpful practice to keep the commitment of the project team and providing them a sense of project ownership and responsibility, which is vital for project success. The Agile Methods focus on frequent ‘working software’ releases also helps in realizing the project progress and estimations at an early project stage.

3.6. Client business security Many clients of agile projects are rightly concerned about confidential data and intellectual property o f their businesses. An approach used by

FastnLoose [ 123 use broadband routers that have a VPN feature to create a secure, private network shared between the dispersed development team and their clients. As the network infrastructures are iinproving day-by-day, this issue is getting smaller from this aspect. But client ais0 needs a sense of security, which he can get by looking at the ‘working software’ according to hisiher requirements, and using Agile Methods the client gets the ‘working software’ maximum after every two weeks. In practice, 1 have found that out of every 5 offshore clients, 3 are highly concern about the sccurity of thcir data, and not willing to provide all the data and information, unless you prove to them, that you can deliver the requirements accurately. Agile Mcthods plays a very supportive role here, by devcloping the sofnvarc in small steps, and delivering frcquent ‘working software’ model, that increases the client’s confidence and make him release the confidential information to the offshore team during the course of the project.

3.7. Documents synchronization

maintenance

and

Developing new document is easy if coinpared to maintaining it. The developers are usually moving at very fast pace and new business demands are also coming in at high speed, handling both of these is not an easy task. New document get out-dated very quickly by only one simple changc request. Agile Methods focuses to speedup the development process to cope the business requirements change by decreasing the extra overheads and focus more on what is required, rather then what is written to be requircd. Off course we cannot rcmovc the docunicntation part completely. Agile Methods focuses more on “developing the document which are required and when they are required”. Best practices found through experiencing Agile Mcthods and its studies are: 1. Document only when necessary [6, 91 2. Document the features in a release planning that are queued up for the upcoming iterative release only. 3. Forget the hope of keeping up to date documents bul focus on developing new documents, each tiinc with only the changes or modified functions documented. 4. Find a ‘Just Enough Point’ in documentation on startup, which can vary from project to project. The essence is that you do not always need all the documents all the time for a project with the sanic level of detail.

3.8. Cultural differences The cultural differences are a major force that must not be underestimated when doing an offshore development project. In most of the developing countries (which are the major offshore platforms) the management is more tenned as the command and control process. To make Agile Methods work you need much more decision making by the implementers. Thus these cultural distances make it difficult to implement Agile Methods in offshore, and this can only be reduced with time and continuous practice. Thought Works [6] and ABN-AMRO [I31 mention thcse differences play significant role, and prove i n practice that these can be handled with care and not by demoting the offshore team’s cultural demands. I have found in practice that in Pakistan we do not realize thc ‘Eastcr’, and ‘Thanks Giving’ holidays. Similarly the US clients do not understand the festivals of ‘Eid’ and ‘Ramazan’. This issue can not only be handled by Agile Methods, But if you ilre following Agile Methods, you already have a good cominunication developed with your client, and usually you already have given short ‘working software’ releases to the client? which makes client more supportive when it comes for having vacations festivals like ‘Eid’.

4. Agile methods practiced at ‘Technology

Wisdom’ Technology Wisdom [I61 is a software development facility in Pakistan with its projects outsourced from clients in UK, USA, UAE and Germany. The Agile and Extreme Programming (XP) [5] principles followed and benefited by Technotogy Wisdom are: I. Client Satisfaction should be the top most priority in all aspects of project development 2. Always welcome the change, although we incorporate the change usually in next iteration 3. Frequent development iterations; maximum 2 weeks cycle time, this helps us developing continuous mature reIeases in project 4. ‘Working Software’ is the primary measure of progress. And this also increases the client confidence in the team, especially when most of our client have not met the development team or even managers in some cases 5 . Frequent communication with offshore clients. A t tninimuin a status chat once in two days. This is a very good practice and through this a client or an offshore partner always keeps in touch and

42 1

this also helps in communicating and handling the project delays Continuous attention to the technical excellence. We use to keep on reviewing the code and use the coding best practices Usc of pair programming for critical project modules and sections, enabies us to transfer the technical domain and train our team by sharing each other experiences. Iteration planning. The focus of development team is on the iteration on which it is currently working. This helps us in breaking a larger problerdproject in to smaller pieces that can be handled more wisely each at a time (Divide & Conquer). Practicing these principles have proven to work for Technology Wisdom within very limited resources, and provide a 100% rate of client satisfaction.

[6] M. Fowler, “Using an Agile Software Process with Offshore Development”,April 2004.

5. Conclusion

[I31 L. G . Hayes, “Everything You Know About Offshorc Outsourcinx is Wrong“, Datamation Magazine, Feb 28, 2003,

[7] E. Sparrow, “British Computcr Socicty- Review 2004, IT outsourcing: current trends and future possibilities”

[8] Failure Ratc SkdtlStiCS: http://www.itcortex.com/Stat-Failure-Rate.htm

191 M. Simons, “Internationally Agile”, Informit.com. March 2002. [io] R. Hitchcns, “Can Agile Slay the Offshore Dragon”, O’Reilly Network, June 2003.

[ I I ] D. Rees, “Distributed Agile Development”, ITWales.com,April 19, 2004.

~

Thc critical most point is that, the devetopment methods are developed far us to use them according to project requirements and our needs, so we must use them to maximize our productivity. And this is the core reason and thought, which has given birth to Agile Methods. After this study and experiences for the comparison and use of Agilc Methods and Principles in Offshorc Development model, I can state that: ‘Agile Methods plays a better role in Offshore Development, comparative to plan driven methods’. Agile Methods and Principles support the Offshore Development, although we need to define carehlly how we implement these principles in practice when handling offshore projects.

6. ‘References [I]

http://www.agileaIliance.com

[2] http://www.agilemanifesto.org

131 Empirical

Findings in

Agilc Methods, 1

Fraunhofer Center f u r Experimental Softwarc

Engineering, Maryland, 2 North Carolina State University- 3 University of Southcrn California Center for Softwarc Enginecring, 4 University of Maryland Empirical Software Engineering Group [4] P. Kutschera and S. Schafer, “Applying Agile

Methods in Rapidly Changing Environments”. [5j http://www.xprogramming.com

422

[ 121 A. Wills, “Programming in Bed”, Fastn’Loosc, Nov 5,2003.

1141 Technical Talk by Vincent Massol on ‘Agile Offshore Methods’ ThcServerSide.com,Jan 6, 2004 to Feb 4,2004. [ 151 Interview - Vinecent Massol, “Agile Offshore

Developmcnt”, Scrvcrsidc.com [ I61 http://www.technologywisdom.com