Growing software applications by incremental development of Heterogeneous Micro-Applications using cellular regeneration concepts Brian Carter, Chippewa Software Technology
Abstract— In this paper, the concepts of growing software applications by incremental development of heterogeneous microapplications using cellular regeneration concepts are explored. The proposal is the move from homogeneous monolithic architectures to heterogeneous micro-architectures, to allow for application evolution. The focus is on growing legacy applications that will evolve to remain healthy. Cellular regeneration concepts are used to show that applications should always be regenerating, like cells living in an organism, to become resilient to events that cause disturbance and damage. Understanding regeneration can give insights into growing computational systems that are more resilient. A definition for micro-architectures and comparisons on how such an architecture differs from typical monolithic architectures are given. Concepts to allow applications to adapt to new and changing business and scientific computational needs are designed. Development and operation complexity is addressed to handle the increase in activity and tasks. We conclude with future states for micro-applications.
Index Terms—micro-application, micro-architecture, monolith, microservices, services, service oriented architecture, SOA, heterogeneous, regeneration, cellular
I. INTRODUCTION The word “legacy” expresses the concept of affection when referring to the work of artists, educators, authors, building architects, and many other professions where one’s actions made a difference for others in the long run, therefore yielding a positive balance of one’s existence. We often say “to leave a legacy”. In computing, legacy often refers to old methods, technology, computer systems, or applications. Often a pejorative term, referencing an application as “legacy” implies that it is out of date, poorly written, or should be replaced. One may ask, why there is an expression of contempt or disapproval for the term legacy in computing. The story is assumed to be a tragedy, full of failure that has its share of warts and dark corners [1]. For software applications, the life expectancy is just a few years due to missed requirements, outdated features, and failures. It has been shown that good software takes ten-years to evolve [2]. There are a few This work was supported by Chippewa Software Technology.
companies that have successfully grown legacy applications (Lotus Notes, Oracle RDBMS, and Microsoft Excel). Failure to understand this ten-year rule leads to crucial business mistakes and failure [2]. The ten-year rule doesn’t mean the application is released after ten-years; the definition states the application is continuously released and evolves while growing more and more value over a ten-year period. How do you create legacy applications that survive beyond your ten-year? In this paper, that question is explored. Much like cellular regeneration, applications must be developed at the cellular level, tiny or micro components that can regenerate. Regeneration is the process of renewal, restoration, and growth that makes genomes, cells, organisms, and ecosystems resilient to events that cause disturbance and damage. This can be applied to computational systems to allow for parts of the system to be removed, replaced, or restored to keep the environment resilient to business changes; thus allowing the application to remain in a healthy state for a longer span of time. II. CELLULAR REGENERATION Animals can regenerate complex body parts with full functionality after amputation or an injury. Among vertebrates (animals with a spinal cord), fish can regenerate parts of the brain, eye, kidney, heart and fins. Frogs can regenerate the limb, tail, brain and eye tissue as tadpoles but not as adults. And salamanders can regenerate the limb, heart, tail, brain, eye tissues, kidney, brain and spinal cord throughout life. Why explore organisms and cellular regeneration when discussion computational systems? Understanding how regeneration works can give insights into making software applications more resilient. Recent research in regenerating has shown that there are various strategies used for regenerating body parts [3]. It has been found that this varies by animal. A single stem cell can regenerate the entire body for some Brian Carter is with Chippewa Software Technology, La Grange, KY 40031 USA (e-mail:
[email protected]).
2
animals; other animals require different stem cells to generate tissue types. Fig. 1 outlines the possibilities.
centralized database; exchanging messages with other systems; and returning a HTML/JSON/XML responses. Presentation components, business logic, database access, and integration are logical components in functional areas of the application. The monolithic application is referred to as a single entity and deployed as a single process. Fig. 2.A shows the typical architecture for a monolith.
Fig. 1. Possibilities for regenerating a limb [3].
In both possibilities, the animal often survives the event and lives on by generate cells. As in nature, an understanding of the animal, in our case application, is required. An application that is simplistic, may benefit from a monolithic approach. As the components get more complex, the need to separate the areas of concerns into smaller, specialized service types position the application to grow more capabilities. The next section applies the cellular regeneration concepts to applications. Understanding of such concepts may lead to innovative ways to grow software that regenerates to survive tragic events and changing environments. III. MICRO-APPLICATION APPROACH To understand the micro-application approach, we will compare micro-architectures to monolith architectures. Micro and monolith software applications concepts and techniques have been discussed for many years [4]. Like in cell regeneration, the struggle for multipotent stem cells or different stem cells in the growing of an animal will always exists. It’s not an A or B decision, both may co-exists, but rather which is better for a given problem. In some situations a monolith application is better due to its simplicity and self-containment. In our case, the micro-application approach will be explored for the problem of growing an application to survive the ten-year legacy. In monolithic architectures, a server-side enterprise application is developed and maintained as a single entity [5]. Such applications have a domain (mydomain.com) with several areas of the company getting served by the same application (mydomain.com/products, mydomain.com/services …). The areas are coupled to a single application entity. The application supports a variety of different clients including desktop browsers, mobile browsers, and native mobile applications. The application might also expose an API for 3rd parties to consume. Typically it integrates with other systems via web services, service brokers, and batch files. The application handles requests by executing business logic; accessing a
Fig. 2. Monolith application
Micro-Architecture application
A Micro-Architecture is shown in Fig. 2.B. It supports the requirements of the monolith (consumers, APIs, application logic, and data storage), yet has a different approach. Similar to the different stem cells design (Fig. 1.B), the use of smaller independent applications are used to meet the given requirements. The Micro-Architectural implementation differs greatly from the monolith. Simply, it is an approach to developing an application as a suite of small applications, or microapplications. The micro-applications will have functionality and data stores that closely align with the need of the function or capability; avoiding the monolith coupled libraries and large centralized data stores. The term “applications” is meant to be all encompassing to represent web applications, mobile applications, web services, microservices, or whatever new concepts that are created in the future ( environment, platform, entity, software, solution, design, approach, robotics, bioinspired, …). The goal is to have application formation through the growth of micro-applications. This will provide the foundation and development of applications based on need and ingenuity. Applications are complex and the modeling of the mutual crossinteroperability between features can’t be preconceived at the start of product development. The micro-applications can be deciphered as functional mechanisms of the encompassing application as features and are discovered and understood. The development, proliferation, differentiation, and migration of the micro-applications should keep the application in steady state “homeostasis” even in stress induced environments. An understanding of the mutual cross-interoperability between micro-applications and their dynamic behavior must be
3
understood to allow regeneration and ability to handle items that move into a quiescent non motile mode, while preserving their developmental and operational potential. These dynamic interactions include monitoring, metrics, and focus on remodeling niche regulation of application components showing abnormal growth. The analogy and comparison to cell regeneration can be shown in implementation abstractions. As shown in Fig. 3, works from immunology is abstracted and compared to software application development. Fig. 3.A and 3.B, show the Stem Cell behaviors [6]. Fig. 3.C, 3.D show how an application can evolve by the regeneration of its underlying microapplications.
IV. REGENERATION As applications age, many things change: requirements change to meet need, hardware advances, new environments are available, software languages are introduced, improvements in development and operations, partners change, and many other areas of application development are impacted. For a system to reach the legacy stage, such changes need to be absorbed and handled. With microapplications, the natural decoupling allows for regeneration of the changed “cells” or features without destroying the entire application. As shown in Fig. 4, the original F3 can be replaced or regenerated with a new F3 (green boxes show impacted areas). The reason for regeneration varies vastly as noted above. Regeneration can be infused by rewriting the capability with a new version, using a new vendor, or the removal of the capability all together.
Fig. 3. Stem cell behaviors (A, B) vs micro-application (C,D)
The micro-framework architecture style is an approach to developing applications as a suite of smaller applications that specializes on a given feature. By decoupling the application, the ability to replace an aging component is comprehensible within today’s resource constraints (scope, schedule, costs). New areas of the application can be added without disrupting the entire ecosystem. This modular design allows for regeneration of system features as needed. Microservices are one interpretation, narrower in scope, than the concepts presented in this papers. A microservice is an implementation following the micro-framework. The microframework can be applied to services, applications, or complete ecosystems. Micro-frameworks design micro-applications that are self-contained and independent entities running in their own process; they can be microservices or grown into an application that is mobilized with a UI to provide a higher level of interoperability. Additional information for microservices can be found in the works of Martin Fowler and James Lewis [7].
Fig. 4. Regenerating part of the system. Green box shows impacted areas.
The removal is a critical aspect that must be handled by the ecosystem. The host must be prepared to handle failures (lost availability, failing dependencies…), of any micro-application. Similar to an animal losing one of its limbs, the animal must survive until the limb is regenerated. The loss of one part of the application can’t jeopardize the entire system. Depending on the micro-application, the loss can be handle based on the context of the application; where a microservice may return an
4
HTTP error code with a JSON message, an HTML application may show that part of the page as not available or try another micro-application providing the same capability, or a Raspberry Pi device may take itself out of the cluster. The disposing of micro-applications is another critical aspect. Each functional capability is small, like in cellular generation, and can be thrown way. Positioning parts of the application to be disposable. Advantage here, removing the fear of change. Accomplished by having all parts of the system small enough to be understood within a few hours. There will be turnover during the ten-year of an application, it must survive the introduction of new staff. Staff must be positioned to understand given micro-applications without the preunderstanding in its entirety; understanding of the ecosystem will come about with more exposure. Change will also introduce new languages. Being disposable, using new languages and platforms will keep the application current and exciting to work on. Avoid the staleness that comes with nondisposable systems [8].
running in their own process naturally creates boundaries that support modularity. Micro-Architectures provide the flexibility to use heterogeneous platforms. Allowing developers to use the tools that fit the problem. Constraining solutions to a given platform, environment, language, or data store introduces complexity when trying to fit tools to solutions. Complex applications require the use of many different tools to meet requirements. VI. DEVELOPMENT AND OPERATIONS
When growing an application, it should be a rule that it will be heterogeneous. Heterogeneous by default should be supported and should be the desired development practice. The computational fields are growing at a rapid rate and heterogeneous aspects must be embraced and supported. The mechanism may follow a given standard, but the inner implementation may vary greatly. The rule for growing an application is the fact that it will be composed of applications on different platforms, written in different languages, using different stores, and running in different processes.
With advancements in application environments, the realization of micro-applications can be supported. As discussed in earlier sections, the concept is to decouple applications into micro-applications that can be reused, regenerated, and replaced. The deployment for monolith applications is vastly different from micro-applications. In monolith environments, the entire application is deployed to large resourced servers. To scale, the entire application is deployed to another large resourced server (Fig 5.A), often taking months to complete the entire process. Whereas micro-applications are deployed at the micro level – spinning up smaller servers to support a given microapplications in their own process (Fig 5.B). Data storage is also impacted. The same pattern is followed, where monolithic has centralized data stores whereas the micro-application pattern decentralized the data stores and places them closer to the application logic. There are some challenges that must be meet to give such micro-applications an environment to grow. The provisioning of new servers must be rapid and reliable. In a microapplications ecosystem, nodes will die and be regenerated
The micro-applications must have module boundaries. This will preserve modularity by all communication through network interfaces. Avoid the common monolith inter-module dependencies that share mutual state. Using applications
rapidly. The deployment of servers must be in minutes, not days or months. The addition of new applications on new servers must occur immediate and should be automated. This deployment pattern allows for each application to maintain an
V. HETEROGENEOUS
5
individual existence. Monitoring is another challenge and is critical. It is essential to have the basics metrics in monitoring; they need to be available real time and have historic values available. Teams will need to know when micro-applications becoming nonresponsive, when they are going down, and when important interactions are experiencing difficulties. The ability to view the health of an application in its entirety with the ability to drill down to the detailed metrics of each micro-application is essential. A culture that supports the rapid deployment of servers and applications is and will be a change for most organization. Netflix takes this to a new level by the introduction of a Chaos Monkey, where the monkey will shut down instances of applications and virtual machines randomly [9]. This allows for simulation of failures in their ecosystems to ensure all items continue to function, operate, and recover during failures. VII. CONCLUSION In this paper, a micro-application approach for growing legacy applications was proposed. The micro-application was compared with cell regeneration, showing how this field may advance by studying such behaviors. The micro-architecture was compared to monolithic architectures, where the advantages of micro-applications include simplicity, consistency, inter-module design, modularity, regeneration, partial development, availability, and the use of heterogeneous platforms. The challenges for development and operations were discussed and items to include to ensure a healthy ecosystem were given. We are still trying to understand how to grow applications that will survive a ten-year legacy. With the computational field advancing at a rapid pace, it is essential to grow applications that may survive, evolve, and adapt to new and changing environments. The unique roles of microapplications, which regulate both normal and application migration and development, are currently under investigation. VIII. REFERENCES [1]
[2]
[3]
[4]
[5] [6]
[7] [8]
Leaving a Legacy…System. Retrieved from http://chadfowler.com/blog/2011/03/17/leaving-a-legacy-system/, October 1, 2015. Good Software Takes Ten Years. Get Used To it. Retrieved from http://www.joelonsoftware.com/articles/fog0000000017.html, October 1, 2015. Regeneration: what does it mean and how does it work?. Retrieved from http://www.eurostemcell.org/factsheet/regeneration-what-does-it-meanand-how-does-it-work, October 1, 2015. WEnbin Li. Towards a resilient service oriented computing bases on adhoc web service compositions in dynamic environments. Engineering Sciences. Institud d’Optique Graduate School, 2014. Microservice architecture. Retrieved from http://microservices.io/patterns/monolithic.html, October 1, 2015. Itkin T. et al. (2012) FGF-2 Expands Murine Hematopoietic Stem and Progenitor Cells via Proliferation of Stromal Cells, c-Kit Activation and CXCL12 Downregulation. Blood. 120(9):1843-55. Microservices. Retrieved from http://martinfowler.com/articles/microservices.html, October 1, 2015. From Homogeneous Monolith to Radically Heterogeneous Microservices: The Wunderlist 3 Story. Berlin International Software Development Conference 2014, Track: Monoliths & Microservices /
[9]
Time: Friday 10:20 - 11:10 / Location: Hall 10. Retrieved from http://martinfowler.com/articles/microservices.html, October 1, 2015. Chaos Monkey. Retrieved from https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey, October 1, 2015.
