The Other Skills of the Software Architect Brian Berenbach Siemens Corporate Research, Inc.
[email protected] taught at university. Finally, I discuss a rarely mentioned topic, what I believe are the personality attributes of the successful architect.
ABSTRACT The skills that are taught to the software architect in academic programs often leave him or her completely unprepared to assume the role of solution architect on a project with more than four or five developers. Furthermore, the globalization of software development has created a need for architects that can effectively manage multiple teams at multiple locations. Additionally, the lack of qualified architects has resulted in serious difficulties for industry. This position paper provides insight into the “real” role of the architect on a large project, and examines the gap between what is taught at the university and what skills are needed for success.
2. MISPERCEPTIONS ABOUT THE ROLE OF THE SOLUTION ARCHITECT Several years ago I was on a large MIS project to develop a commercial payroll system. The appointed architect seemed to spend all his time programming. When I asked to see the architecture for the product, he stated that it was “all in his head” and that we would “just have to trust him”. I asked the department head why that particular individual had been made the architect and the response was that he was the “best C++ programmer in the department.”
Categories and Subject Descriptors K.6.1 [Project and People Management]: Life cycle, D.2.9 [Management]: Life cycle, Productivity, Programming teams
Individuals promoted from team lead or developer to the role of architect are often completely unprepared for the dynamics of the position. Similarly, employees who are recent graduates of universities with advanced degrees are often pushed into technical management without operational experience as an engineering manager.
General Terms Management
Keywords Leadership, Management, Architect Role
Computer Science programs (and often the software engineering programs) may give students the wrong impression of life after school. First, at the undergraduate level, skills are learned “bottom up”. That is, students start by learning computer languages. They sit at a terminal and program, and as their programming and computer science skills get better their programs become more sophisticated. When students do this for three to four years as part of a degree program, seat-of-the-pants software development becomes ingrained; old habits are hard to break. Second, when students do team projects, their goals are quite different than the goals in industry. These goals are described in table 1.
1. INTRODUCTION According to many experts, a crisis is looming on the horizon because of a lack of properly trained software engineers [1]. The crisis may be caused, in part, by the increasing complexity of software projects, the difficulty of managing distributed work, and compressed development cycles. The focus on software architecture at both the undergraduate and graduate level tends to be technical in nature, e.g. tools and techniques, architectural patterns, etc. Recently, there have been efforts to focus on other skills that the architect needs [1]; however the descriptions have tended to be broad without focusing on the operational details necessary to succeed. For example, a guideline might require documentation of the architecture, but exactly how to do that is left to the architect. Unfortunately, without experience or a prior apprenticeship on a successful project, many architects “crash and burn” on their first big assignment without knowing exactly why they failed in their duties. In the following sections of this position paper, I first discuss common misperceptions about the duties of the software architect. I then describe what I feel are important leadership and interpersonal skills needed to succeed as a corporate architect that are rarely
Table 1. Academic vs. Industrial Software Goals
Goal Time frame Software Quality Design Maintenance Customer satisfaction
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. LMSA’08, May 11, 2008, Leipzig, Germany. Copyright 2008 ACM 978-1-60558-027-9/08/05...$5.00.
Student Very short term As little as possible Ad hoc Not an issue Get a passing grade
Professional Project duration Dictated by standards, professionalism and regulation Planned Planned Mandatory
3. AGILE DEVELOPMENT AS A PANACEA There is a spectrum of software development from agile to planned [2]. Unfortunately, as Barry Boehm points out, many of the professionals involved in software development are inadequately prepared for their roles. Computer science (CS) graduates will often
7
The architect and his staff need to address such questions up front before they become crises.
use what seems comfortable in a crisis or when under pressure, e.g. the ad-hoc, anarchic approach used at university. The malleability of software tends to become the quicksand of project development; there is the unfortunate perception on the part of software and nonsoftware professionals that informal, ad-hoc, lightweight processes can work on the development of large, complex software products, or complex products where software is a major component. No one would expect a skyscraper, locomotive or passenger airplane to be designed and built “iteratively”, yet software management might expect a complex system to be put together with lightweight designs, minimal coding standards, and laissez-faire management. Alternatively, there might be strong management, but with a focus on the “backlog”, with minimal attention paid to long term product viability, extensibility and maintenance.
One often overlooked issue on large projects is staff turnover. The initial staff gets together, walks through the requirements and proposed architecture, there is give and take, standards and procedures are reviewed, and the project kicks off. However, when there is staff turnover, none of the people coming on board receive the same level of indoctrination as the initial core team. How will that be handled (See 4.5)?
4.2 Creation of Standards, Templates and Samples When working as a senior architect, staff would come into my office and say “I do not know how to do that”. I learned very quickly that telling someone to do something was quite different from getting it done. Yet I see many projects today where, for example, samples are not available or prepared for the staff. One example is in the annotation of code. If we ask a developer for 25% of his lines to be annotation, we might wind up with something like this:
4. NON-TECHNICAL SKILLS OF THE ARCHITECT The technical skills necessary to succeed as an architect have been well documented [3]. In this section I will discuss some of the needed skills of the solution architect. In some cases, these skills can be learned on the job. Some of the skills can only be learned through prior experience in various project positions and as an apprentice architect.
// i = j + 1; i = j + 1;
4.1 Ability to Define an End-to-End Lifecycle Process and Tool Set
By planning ahead and having standards, peer reviews, “perfect” samples for illustration and automatic generation of documentation from code, such problems can be avoided. In my situation, whenever I wanted my staff to create a document, annotate code, or create plans of any kind, I always made sure that a “near perfect” example of what I wanted was available for use as a template. Providing samples became easier as time went on. On my projects I learned to be initially very demanding of staff, the quality of the initial work products was, consequently, very high; as new members were added to teams they had a large number of high quality samples to use, and a culture of excellence had already been established.
In the classroom and, unfortunately, in many project scenarios, the focus is on the initial design of a software architecture. A skill that is at least as important as design for the senior staff is to be able to visualize an end-to-end process, e.g. to be able to see the “big picture”. When my son was young and I was putting him to bed, I would ask him to brush his teeth, and his response would be “and then what?” We would play this little word game frequently. I do not see many architects at the senior level playing this game, yet it is vital to the success of a project. Walking through the lifecycle processes, team members can better understand the back end consequences of front end decisions and omissions. For example, a forgotten design standard will become apparent when defining the part of the lifecycle dealing with design reviews (e.g. review the design against what standard?). By conducting open, frank, round table discussions with staff at all phases of a project, problems come to the surface early and are addressed early. For example, given a non-functional requirement for maximum capacity transactions per second, during early meetings staff members might ask questions such as: • • • •
4.3 Understanding of Test and Maintenance Needs I believe that the job of the architect does not end until a software product is in maintenance, and the maintenance process is well established, efficient and effective. Unfortunately, that belief is not shared by the entire software engineering community. As a result, in many cases there is a “throw the design over the wall” mentality. We do not see such scenarios in other industries. In aviation, transportation, civil engineering, etc., the architect is there from beginning to end, e.g. from ground breaking thru final occupancy. Yet if one asks software architects when their job ends, we hear all too often the response “when the design is finished”.
“Can we use simulation to determine that a proposed architecture will meet the performance needs prior to investing in development?” “How do we measure current transactions per second in the running system?” “What measure can we use to demonstrate to the client that we have met the contractual requirements?” “How do we gather statistics on performance when the system goes into maintenance?”
4.4 Understand and Resolve Stakeholder Conflicts When capturing non-functional requirements with stakeholders it is important to understand whether the stakeholder is speaking for himself or for the entire solution space.
8
4.7 Ability to Manage Non-Functional Requirements The software project architect is the most likely role to manage nonfunctional requirements as his or her duties start with contract negotiation (for external1 projects and end with product maintenance (for in house projects). The management of nonfunctional requirements is a task that is analogous to an iceberg; only a small number of the necessary activities are immediately visible to project management. For example:
Interpersonal
•
Skill set needed by the senior architect
Business Domain Knowledge Expertise
• •
Software Architecture
• •
Software Engineering
How are the non-functional requirements placed in the context of the functional requirements? How are they quantified [5]? What tracing mechanisms are necessary and/or will be in place? How will they be contractually accepted? What about external regulation?
5. PERSONALITY TRAITS OF THE SUCCESSFUL ARCHITECTS
Figure 1 Skill Set Needed by the Senior Architect
In the U.S. military, the percent of generals in the officer corps was less than 0.4 percent in 1996 [6]. Part of the reason is regulation, and the other part is that the higher up the rank, the more interpersonal skills tend to overshadow pure technical skills and fewer people are available with the convergence of technical and personal skills necessary to be a military leader. It is the same in the corporate world, being a senior level software architect requires a convergence of interpersonal and technical skills that can be hard to find. Couple this with the need for domain expertise, and the ideal candidate for a senior architect position can be very hard to find (Figure 1).
There may be several organizations or customers planning to use a product and they may have conflicting views of functionality and quality. The solution architect has to successfully resolve those conflicting needs without alienating clients.
4.5 Ability to Define a Viable Training Process Based on personal experience, I know the pain of investing a great deal of time in training a core team, only to have some of them rotate out of the project at key points in time. The departures can be because of promotion, transfer, moving on, or personal problems. It happens.
5.1 Ability to Listen The ability to listen must be learned. It is not just the ability to hear, understand and act on (if necessary) what other people are saying. It is also the ability to make staff feel comfortable in reporting bad news as well as good, or expressing opinions that are contrary to that of other staff.
Since it happens, we need to expect and plan for turnover. This implies some kind of ongoing training or orientation process. Large projects can be very, very complex, but it is really important for every team member to have a general understanding of the system (imagine someone working on the avionics for an airplane without knowing how an airplane works – scary!). Furthermore, when standards and procedures are “on the shelf”, and no one tells the new team member about them, he will not know about them! Even better if the necessary standards and procedures are active or preemptive such as embedding standards, procedures and templates into a rule based workflow environment so that staff are prompted for what they need to do next and mistakes are caught as early as possible.
5.2 Ability to Motivate and Coach Motivation of staff can be difficult, especially when there are delivery pressures or a project has dragged on longer than staff would like. Yet, without motivated staff, quality and productivity will suffer.
5.3 Attention to Detail Attention to detail is an important but often overlooked characteristic of the successful architect. There is no place that the saying “the devil is in the details” [7] is truer than in software engineering. In the extreme, one line requirements in contracts can potentially wipe out a company.
4.6 Understanding of “Complete” as Applied to Project Artifacts When people are under pressure, they may react accordingly. When a developer finishes a component, he or she may check it in and declare completion without effective testing. When this culture permeates an organization, problems will cascade and their effects will magnify. Consequently, it is important to have a viable definition, for each artifact, and process, as to what the meaning of “done” and “completion” is. This will improve both the development culture and team morale, as completion is well defined as opposed to nebulous.
1
9
By external I mean a project where the development is done for a customer in a different organization or company; internal is where the supplier and customer are in the same organization.
To be a visionary requires that the professional keeps up with trends and techniques. All professionals should seek to constantly broaden their knowledge, and architects are no exception. Continuing education, participation in professional associations, and attendance at conferences are ways to keep skills sharpened and up to date.
5.4 Paranoia Trusting staff with a proven track record is one thing, but assuming that work will be done properly without controls and supervision is folly. Clearly, I am not advocating that mental illness is a benefit, but on a scale from “trust everybody”, to “trust nobody”, a healthy dose of skepticism is usually required.
6. RECOMMENDATIONS I have two sets of recommendations, one for augmenting the undergraduate CS and software engineering curriculum, and another for industry.
5.5 Up and Down Communication Solution architects work in the murky world halfway between business and engineering. Consequently they need to understand business goals and stakeholder needs and able to communicate ideas both up and down the organization chain of command. Furthermore, an understanding of marketing and selling techniques can be valuable. For example, take the scenario where an architect is given the assignment of creating a software platform for a product line, without a well defined set of business goals or requirements. Marketing literature describing the platform as a salable product can be a great aid to the crystallization of a requirement set.
6.1 Suggestions for Undergraduate Topics It is important to introduce computer science and software engineering students as early as possible to processes and interpersonal techniques that are effective in the development of complex systems. This includes concepts in the design of complex systems, engineering management, project management, logistics, team dynamics and the development of interpersonal skills through leadership training, organizational and social or organizational skills training. These courses already exist at the undergraduate level, and they are taught as part of industrial engineering, systems engineering and industrial psychology programs. In order to make room for new topics, courses such as compiler theory may have to be compressed into introductory or survey courses, and be taught in detail at the graduate level. Very few graduating CS or software engineering students today will ever have to work on a compiler, but sooner or later they will all have to work in a team environment and accept some technical management responsibility. It is unfortunate that many managers in industry today are not cognizant of the difference between a computer science and software engineering graduate.
5.6 Decision Making The inability or unwillingness by senior staff to resolve conflicts through compromise or make decisions can paralyze a project. The solution architect is sometimes in a position where not making a decision promptly can result in project delays, missing important dates, and even the loss of milestone payments. Meetings are a chore when there is a lot of work to do. When the same topics are brought up meeting after meeting without a resolution, then there is a management rather than technical issue, and the issue needs to be addressed quickly.
5.7 Open Mind With software, there is almost always more than one solution to a problem. Occasionally a “pet” technique or the solution best known to the architect might be the one chosen even though it might not be the best solution. For example, an architect might be an expert in Java, and therefore choose the Java language to implement an embedded system, even though other staff on the project has compelling arguments for the use of C++.
6.2 Suggestions for Industry Selecting a senior architect for a project can be a bit like buying a lottery ticket. Management may not discover the strengths or weaknesses of the chosen individual until the project is completed successfully or in crisis. This is especially true if the solution architect position is outsourced. There are, however, some steps that can be taken to improve the skill set of candidates for solution architect; the suggestions below are intended for internal staff that is being groomed for taking on more responsibility. I suggest:
I have seen, on rare occasion, a project architect select a methodology or solution for the sole reason that it was the topic of his graduate thesis. But, was it the best solution, given the organization needs?
•
Optional, recommended training in selected industrial engineering, organizational psychology, and systems engineering material. It is usually possible to find universities to partner with; many schools already have the necessary courses and qualified faculty, and might be willing to craft a custom industrial program.
•
Certification of staff that have the skills to take on senior positions on large, mission critical projects. Siemens Corporate Technology is currently piloting such a program.
•
Encouraging staff to create a track record by taking on increasing responsibility over time on larger projects.
•
Interviewing former colleagues and subordinates of candidates selected for important architect positions.
•
Providing apprentice opportunities for promising candidates under successful architects; some skills for senior positions have to be honed on the job.
5.8 Visionary An architect may have to be a bit of a visionary when designing a product that will take a significant time to reach the market, or is expected to be on the market for a considerable period of time. It is important to design for the future, not for today’s technology. Computer systems and technology change so rapidly that it is nearly impossible to score a bull’s-eye with a system design, however, it is important to be as on target as possible. For example, designing a software product that uses 100% of computer processing power today means that in two or three years that same software product will only use 30-50% percent of processing power or less [8]. Sometimes it is necessary to guess, sometimes we guess wrong. But the one thing that is guaranteed is that if a product is designed for today’s systems and technology, it will be obsolete the day it is released.
10
•
Collectively tracking successes and failures and carefully documenting them as case studies to gain understanding as to why there was success or failure; encouraging management and staff to learn from the collective corporate history.
•
Keeping software teams together where possible makes good business sense. Teams that stay together grow and improve. A project team given a similar assignment will always do better the second time around, in most cases a lot better.
qualified senior candidates. The program was first announced by Mr. Reinhold Achatz, the president of Siemens Corporate Technology, at his 2006 ICSE keynote speech [9]. Also, the Software Engineering Institute offers a certification program in software architecture. As part of this program, they now offer courses that include training in social and leadership skills [10].
8. REFERENCES [1] Dr. Lyle N. Long, “The Critical Need for Software Engineering Education”, Crosstalk Magazine, January 2008
7. CONCLUSIONS The solution architect is often the linchpin of critical projects. Furthermore, he or she may be the only one in a position to perform important functions such as managing tracing, predicting technological changes, motivating staff and providing leadership and training.
[2] http://www.sei.cmu.edu/architecture/arch_duties.html [3] Barry Boehm, “Agile and Plan-Driven Methods - Oil and Water?”, presented at the Agile Universe, Chicago IL Aug 2002 [4] L. Bass, P. Clements, R. Kazman, Software Architecture in Practice, Second Edition. 2003 Addison Wesley, Reading.
Unfortunately, the non-technical methods learned in a computer science or software engineering curriculum often foster techniques that are counter to the skills that will be needed to perform effectively as a senior or solution architect.
[5] Martin Glinz, “A Risk-Based, Value-Oriented Approach to Quality Requirements”, IEEE Software, April 2008. [6] The Congressional Budget Office, http://www.cbo.gov
While technical knowledge is important, for senior architects it represents only a fraction of the skills that are needed to succeed in industry. Most universities to date have not focused on the leadership and management aspects of architecture that are needed for tomorrow’s technical leaders. Finally, industry and academia have begun to address the task of improving the non-technical skills of senior staff. For example, Siemens Corporation has initiated a pilot certification program in software architecture for highly
[7] Gustave Flaubert, 1821-1880. [8] G. Moore, “Cramming more components onto integrated circuits”, Electronics, Volume 38, Number 8, April 19, 1965. [9] www.isr.uci.edu/icse-06/program/keynotes/achatz.html [10] http://www.sei.cmu.edu/products/courses/aft.html
11
