Survey on Software Maintenance Profile and Knowledge Requirement in Public Higher Learning Institutions Mohd Zali Mohd Nor Faculty of Computer Science and Information Technology Universiti Putra Malaysia, Serdang, Selangor
[email protected] Abstract Software maintenance (SM) environment is highly complex, knowledge-driven and collaborative. Therefore, Knowledge management (KM) is critical to provide an environment for sharing and sustaining knowledge. Issues such as inadequate knowledge and lack on information sharing are still regarded as major challenges in SM. This paper presents the results of a survey on KM of SM process in selected higher learning institutions (HLIs) in Malaysia. Based on the survey, several deficiencies are identified as common to HLIs. Domain knowledge is important but these are seldom stored in KMS or other electronic means. Maintainers also spent considerable efforts collaborating with other parties to obtain information. Therefore, resolving these issues should be given high priority.
1.
Introduction
In recent years, many SM organizations consider knowledge management (KM) to be strategically important to their business [9][10][18]. With spending estimated at USD12 billion in 2012, KM is envisaged to contribute to the organizations in the following manners [8]: • • • •
Bring synergies among different teams, units or departments Accelerate innovation and boosting revenues for market development. Improve quality in operational and functional processes Reduce costs and exposure to business risks
With the above ‘promises’, KM are also enticing to software development and maintenance organizations, especially in managing software engineering activities.
Rusli Hj. Abdullah Faculty of Computer Science and Information Technology Universiti Putra Malaysia, Serdang, Selangor
[email protected] Within software engineering activity cycle, software maintenance (SM) has yet to receive proper attention [6]. It is a costly process, where previous works [4][12][14][20] estimated SM costs of between 60% to 90% of total software life cycle costs. The motivation in applying KM in SM in driven by the fact that the activities are knowledge-intensive [17], and depend largely on expertise of the maintainers. Most of the time, maintainers depend on experience and “hunches” when making decisions. Some of these expertise are documented as implicit knowledge, but more are hidden as tacit knowledge due to scarcity of documentation[23]. However, as SM organizations grow, become distributed and members need to collaborate and share ideas, these individual knowledge need to be shared. Consider the following example: when a superuser reported a bug in an application via email, the helpdesk operator shall log the report and check against the known-bug list. If this is a new bug, helpdesk shall assign it to maintenance team lead, who will call or email the superusers to ask for more details. Many times, the team lead is unsure on the source of the problem and therefore will involve other parties such as systems analyst to assist in investigation, analyze the impacts and prepare change design or proposal. Once done, maintenance manager shall decide on priorities and schedule, before real changes to the programs take place. In all of the above activities, efficient collaborations between parties are critical to ensure smooth maintenance of the system.
1.1 Software Maintenance Environment Software maintenance(SM) environment is defined by the collection of tools, processes and resources used to maintain software systems. Many SM organizations have their own best-practice processes and activities to suit the organizational and business practices. Detail
activities may differ among in-house maintenance, vendor or outsourced maintenance, or of-the-shelf application maintenance. The differences are also visible for different types of software or applications being maintained, and the people or teams involved in the process [16]. Maintenance activities are knowledge-driven [17], and depend on largely expertise of the maintainers. Some of these expertise are documented as implicit knowledge, but more are hidden as tacit knowledge due scarcity of documentation [21] This is why, Nonaka and Takeuchi [13] suggested that KM requires a mechanism or ‘place’ to convert internalized tacit knowledge into explicit knowledge (information), and also allow individuals to internalize meaningful information. In order to do these, maintainers have to collaborate with colleagues and other parties to obtain various information to enable them to carry out their software maintenance tasks. As an overview, software maintenance is defined as “The totality of activities required to provide costeffective support to software system. Activities are performed during the pre-delivery stage as well as the post-delivery stage.” [7]. In term of differences in types of maintenance, The four widely known are depicted in Figure 1 below[7][11]: • Corrective – to fix discovered program faults. • Adaptive – to cater for changes in business requirements and external environment. • Perfective – to improve performance and maintainability. • Preventive – to detect and correct latency faults before it becomes faulty.
Figure 1: Maintenance Types Among the above maintenance types, studies [4][12][14] indicates that 60% - 80% of efforts are spent for perfective maintenance, and only 17% are spent on corrective maintenance. However, a more recent study [20] has taken into account new technologies, new processes and procedures to design and develop software, new types of applications and significant reliance on reuse. The study concluded that
the corrective maintenance nowadays may consumes up to 50%-68% of all maintenance types. Software maintenance processes and activities have been largely standardized. Standard organizations such as ISO, IEEE, and CMMI have detailed the activities to be carried-out by software maintainers [1][7]. At a very minimum, the activities include process implementation, problem and modification analysis, modification implementation, maintenance review and acceptance, migration and software retirements, as depicted below in Figure 2:
Figure 2: ISO/IEC 14764 Maintenance Process Many software maintenance organizations may have their own best-practice processes and activities to suit the organizational and business practices. Detail activities may vary depending on the following setups: • Types of maintenance organizations – such as inhouse maintenance, vendor or outsourced maintenance, or commercial-of-the-shelf (COTS) application maintenance. • Team setup – such as similar or separate development and maintenance team. • Types of software or applications being maintained [16] • Maintenance approach or model – Those using Waterfall approach may differ from those using Agile approach.
1.2 Knowledge Required in SM The knowledge needed in SM can be summarized as follows [5][17][18]: •
Organizational knowledge, such as roles and resources. The parties involved in software maintenance activities consist of various application users and software maintainers. The list may include end-user, superuser, maintenance manager, business analyst, systems analyst, project
•
•
• •
manager, QA personnel, build manager, implementation personnel and trainer. Attached to these roles are the area of expertise. Managerial knowledge - such as resource management, task and project tracking and management. Technical knowledge – such as requirement analysis, system analysis, development tools , testing and implementation. Critical to this is also the knowledge on supporting groupware and CASE tools such as Software Configuration Management (SCM), helpdesk and testing tools Domain knowledge – knowledge of the products and business processes. Knowledge on source of knowledge – where the knowledge resides, such as source codes, documentation, supporting CASE tools and more importantly, the where the experts are.
Still, it is often heard complaints that the process from bug report to getting the corrected system is slow due to complexity and time taken collaborating to obtain information, sharing information, and internalizing information onto knowledge. There are various issues associated with the above knowledge, which makes organizing, storing, sharing and disseminating knowledge difficult. Among the problems are: •
•
Documentation are most of the time not up-todate. As mentioned earlier, Earlier studies indicates around 50% of efforts are spent on this activity and rely more on source code than any other source of information [4] Domain knowledge are becoming more important to software maintainers [19]. However, this knowledge are often not available within the software maintenance CoP, especially in vendor and distributed environment.
•
Human experts hold most of the tacit knowledge that are not readily available to others. are the major source of knowledge. However, there are still reservation toward knowledge sharing. ‘'If getting promotion, or holding your job, or finding a new one is based on the knowledge you possess what incentive is there to reveal that knowledge and share it?' [24] . The issues of inadequate knowledge available to maintainers are common. Maintainers, especially novice, spends a lot of time searching, discovering and understanding software knowledge. Earlier studies indicates around 50% of efforts are spent on this activity and rely more on source code than any other source of information [9][12]. The ‘containers’ could be in different electronic forms, which sometimes need to be mined and manually extracted. Or worse, in paper form which require more effort to place it in KMS. For example, email may be good for communication, but very hard to manage as a identifiable group of information for sharing use. Software maintenance, by itself, is complex and is a constantly changing process, with different phases, activities and projects. Organizations often have problems identifying resources and use of knowledge [17
To assess how maintainers in their daily battles, it is important to access their activities, and how knowledge are acquited, shared and used. This paper shall be structured as follows: Section 2 reviews the related works on the subject issue. Then, section 3 briefly discuss the research design and section 4 discusses the survey results and recommendations.
Table 1: Survey on SM organization Yeh - Taiwan A profile of a typical software system under maintenance Software maintenance organization and management The distribution of maintenance activities Major maintenance problems Measures employed to improve maintenance
X
Yip - Hong Kong X
X
X
X X
Poo and Chung - Singapore
X X X
X
where the information to resolve the issues are obtained. 3. Find out the knowledge required to plan, manage requests for changes and releases. 4. Determine how explicit knowledge in MRs are combined and internalized by analysts/developers/maintainers. 5. Determine the extent of previous “lesson learned” stored as explicit knowledge that can be used for future problem/issues
2. RELATED WORKS Surveys on SM organization profiles and use of tools to support SM activities have been carried out in Taiwan, Singapore and Hongkong [25][15][26]. In the SM area, the following issues surveyed are listed in Table 1. Earlier studies on major maintenance problems were studied by Lientz and Swanson [22] and Dekleva [3]. Dekleva study ranked the perceived problems in SM as follows:: • Managing fast-changing priorities • Missing or incomplete software documentation • Slow in adapting to rapid business changes • A large number of awaiting user requests • Lack of methodology, standards, procedures and tools specific for SM. • Difficulty in meeting user expectation • Loss of expertise when maintainers leave. The above studies, however, do not include the important aspect of how SM activities are conducted and how knowledge is shared and used.
4.0 DISCUSSIONS AND FINDINGS In summary, the following can be concluded based on the survey of SM environments in selected HLI:
4.1 Part 1 - Profile of the Software Maintenance organization setup, standard and methods • •
3.0 RESEARCH DESIGN This study is a part of a longitudal study, with the aim to develop the framework, system architecture and tools to support KM in collaborative SM. To identify the organization profile, processes and knowledge required to perform SM activities, a survey was carried-out in SM organizations in HLI. Questionnaire were sent to the computer center directors in 7 major public universities in Malaysia. Since the questionnaire are quite comprehensive, and covers the helpdesk, analysis, coding and release activities, prequestionnaire discussions were held to explain the questions to participants. With this, variations in understandings on the questions were minimized. From the 7 questionnaires sent out, 5 were completed. The questionnaire is divided into five parts, with the following objectives: 1. Profile of the SM organization setup, standard and methodology used. This includes the types of applications maintained, distribution of activities and types of maintenance being performed. 2. Find out the knowledge required for 1st level support activities. i.e., How helpdesk handles user complaints and requests and what and
•
•
100% of the surveyed SM organizations use the same team for development and maintenance. KMS is not widely used to store knowledge acquired in all activities. Out of five HLI surveyed, only one organization has KMS deployed, with only information on location of experts available. In term of types of tasks carried out, almost 72% are for operational support, followed by minor enhancement (9%) and emergency datafixes (9%) (see figure 3). Waterfall process approach remains the favourite approach, for major bug-fixes and enhancements.
4.3 Part 2 – Knowledge Required for Helpdesk Activities •
•
80% of respondents uses helpdesk applications to record complaints and issues. However, helpdesk still rely on telephone (100%), email (100%) and paper-based form (60%) to report bugs and complaints to Helpdesk personnel (see figure 4). This could be inefficient in tracking the issues. During issue investigation, helpdesk personnel relies more on past experience/common sense (80%), references from experts (80%) and earlier reported errors (60%). Reliance on best-practice guidelines (20%) and documented known issues (40%) are less utilized (see figure 5).
Figure 4: Problem and Request Reporting Methods
Figure 3: Requests per Maintenance Type
Figure 5: Source of Knowledge for Helpdesk
Figure 7: Rank of Perceived Problems in Helpdesk
Figure 6: Other Knowledge Required for Helpdesk •
•
•
Domain (business) knowledge is considered important to aid investigation (60%) (see figure 6). However, it is noted that this knowledge are not stored electronically. Participants ranked Quality of documentation, followed by lack of users interest and understanding, changes made to the system, and lack of domain knowledge as the main perceived problems in helpdesk operation (see figure 7). Majority of the time is spent collaborating with maintenance team (41%). 20% of the time is also spent communicating with users to obtain information (see figure 8).
4.3 Part 3 - Knowledge Required For Planning and Analysis Activities •
•
•
Majority of respondents agree that domain knowledge are important for systems analysts to resolve maintenance requests ((MRs) (75%), and knowledge on experts and their locations (50%) (see figure 9). Most of the time spent by analyst are to discuss issues with users (22%), followed by collaborating with colleagues on requirements (20%), and collaborating with developers to brief the requirements (20%) (see figure 10). In term of perceived problems in planning and analysis, participants ranked quality of documentation as the main problem. This is followed by lack of users interest and
Figure 9: Knowledge Required For Planning & Analysis
Figure 8: Activity time for Helpdesk
Figure 10: Perceived Problems in Planning & Analysis understanding, quality of original programming, and unrealistic user expectations (see figure 10).
Figure 11: MR Planning & Analysis Activities •
4.4 Part 4 - Knowledge Required and Shared During Coding and Testing •
•
•
To assist programmers and vendors to understand the requirements better, most respondents use informal briefing sessions (100%) as the preferred approaches (see figure 12). In term of the perceived problems, inadequate system to support change and testing process, quality of documentation, unrealistic requirements, and lack of interest and understanding among maintainers are ranked higher (see figure 13). Majority of respondents agree that domain knowledge are important (80%) to assist programmes during coding (see figure 14). However, these knowledge are not captured electronically.
In term of estimated activities, 22% are spent coding followed by 18% studying and understanding specification,. Another 15% are spent collaborating with SAs to clarify the requirements (see figure 15).
4.5 Part 5 - Are "Lessons Learned" Stored •
Most of the solutions to previous problems are either not stored (60%), or stored only in MR system (40%) (see figure 16), which limits the knowledge retrieval for future reference.
5. Conclusion and Future Work Software maintenance is complex, knowledgeintensive and highly collaborative. Managing knowledge in this area is therefore critical to ensure that maintainers can perform software maintenance activities properly and timely, by sharing and obtaining vital knowledge.
Figure 12: How Knowledge Are Shared During Coding/Testing
Figure 13: Perceived Problems in Coding & Testing Figure 14: Knowledge Required For Coding/Testing
Figure 15: Changes and Testing Activities Figure 16: Where are Lessons Learned Stored This paper presented the results of a survey on managing knowledge of SM process in HLI. The major issues common to the HLIs are identified are as follows: •
•
80% of the surveyed SM organization do not use KMS to store knowledge acquired during the maintenance activities. Hence, this could contribute to problems and lateness in getting the information from other experts. In various aspects of SM activities (helpdesk, planning and analysis and coding and testing), between 60% to 80% of respondents consider domain knowledge important to assist them in daily SM activities. However, they admit that the knowledge generated from these activities are not stored in KMS or other electronic means, thus making them harder to extract, shared and turned explicit.
•
•
Substantial efforts are spent collaborating with users, experts, SAs, and vendors to ascertain the problems and requirements. In the survey, 41% and 20% of helpdesk time are used to collaborate with maintenance team and users, respectively. Meanwhile, in the planning and analysis, 22% of the time is spent discussing issues with users, 20% with colleagues and another 20% with developers. Without a systematic approach to these information acquisition and sharing, these efforts shall remain a major issue. Overall, in term of the perceived problems, quality of application documentation and inadequate system to support remains as major issues.
This paper represents a part of a longitudal study, with the aim to develop the framework, system architecture and tools to support KM in collaborative
SM, which shall assist maintainers to solve one or more of the above issues. Our next step is to investigate how we can integrate the domain knowledge and SM process knowledge, to assist helpdesk, analysis, planning and maintainer personnel their daily activities. A tool to capture, retain and share the above integrated knowledge shall also be proposed.
References [1] April A. et al., Software Maintenance Maturity Model (SMmm): The Software Maintenance Process Model, Journal of Software Maintenance and Evolution: Research and Practice, Vol. 17 No. 3, 2005. [2] Das, S., Lutters, W., Seaman, C., Understanding Documentation Value in Software Maintenance, Proceedings of the 2007 Symposium on Computer human interaction for the management of information technology, 2007.
[13] Nonaka, I. & Takeuchi, H., The Knowledge-Creating Company. New York: Oxford University Press, Inc., 1995. [14] Pigoski T.M., Practical Software Maintenance:Best Practices for Managing your Software Investment, John Wiley & Sons, 1997. [15] Poo D. and Chung M.K.., CASE and software maintenance practices in Singapore,Journal of Systems and Software,Volume 44, Issue 2, December 1998 [16] Pressman R.S., Software Engineering: A Practical Approach, 6th Edition, McGraw Hill, 2005. [17] Rodriquez O.M., et al., Understanding and Supporting Knowledge Flows in a Community of Software Developers, Lecture Notes in Computer Science, Vol.2198, 2004. [18] Rus, I., and Lindvall, M., Knowledge Management in Software Engineering, IEEE Software, Vol.19 Issue 3,2001.
[3] Dekleva S.M., Software maintenance: status, Journal of Software Maintenance, 1992.
[19] Santos G. et al., Knowledge Management in a Software Development Environment to Suport Software Process Deployment, Lecture Notes in Computer Science, Vol.3782, 2005.
[4] Fjeldstad R. and Hamlen. W., Application Program Maintenance Study: Report to Our Respondents, 1979, in S.L. Pfleeger, Software Engineering- Theory and Practices, Prentice Hall, 1998.
[20] Schach S. R., Jin B.O., Yu L., Heller G.Z. and Offutt J., Determining the Distribution of Maintenance Categories: Survey versus Measurement, Journal of Empirical Software Engineering, Vol 8, 2003.
[5] Ghali N., Managing Software Development Knowledge: A Conceptually-Oriented Software Engineering Environment, PhD. Thesis, University of Ottawa, Canada, 1993.
[21] Sengupta B., Chandra S., Sinha V., A Research Agenda for Distributed Software Development, Communications of the ACM, 2006.
[6] Guide to the Software Engineering Body of Knowledge (SWEBOK), (2004), The Institute of Electrical and Electronics Engineers, Inc.:New York, NY. [7] IEEE 14764-2006, Standard for Software Engineering Software Life Cycle Process – Maintenance. The Institute of Electrical and Electronics Engineers, Inc.:New York, NY. [8] KPMG European KM Survey 2003, retrieved from www.knowledgeboard.com. [9] KPMG Knowledge Management Research Report 2000, retrieved from www.knowledgeboard.com. [10] Lawton G., "Knowledge Management: Ready for Prime Time?," IEEE Computer, Vol.34, No. 2, 2001 [11] Lientz B.P. and Swanson E., “Problems in application software maintenance”. Communications of the ACM, Vol. 24 No.11, 1981. [12] Lientz, B. P., Swanson, E. B., and Tompkins, G. E., Characteristics of Application Software Maintenance. Communications of the ACM, 1978.
[22] Swanson, E.B. and Beath, C.M., “Departmentalization in Software Development and Maintenance”, Communications of the ACM, Volume 33, no. 6, 1990. [23] Viscaino A. et al., Supporting Software Maintenance in Web Repository through a Multi-Agent System, Lecture Notes in Computer Science , Vol. 2663, 2003. [24] Wilson, T.D., The nonsense of knowledge management, Journal of Information Research, Vol.8 No.1, paper no. 144, 2002. [25] D., Whu C.K. and Jeng J.W., Software Maintenance in Taiwan and a Comparative Analysis With Other Countries, Proceedings of the International Computer Symposium, 2006. [26] Yip S.W., Software maintenance in Hong Kong, 11th International Conference on Software Maintenance (ICSM), 1995.
