Single Points of Failure Within Systems-of-Systems

Single Points of Failure Within Systems-of-Systems Kirsty E Lever, Madjid Merabti and Kashif Kifayat School of Computing and Mathematical Sciences, Liverpool John Moores University, Liverpool, L3 3AF, UK [email protected], {M.Merabti, K.Kifayat}@ljmu.ac.uk

Abstract— Computer technology has become highly complex, widely available and thanks to growth and popularity of the Internet, society and organisations are becoming heavily reliant on distinct computer technology to meet objectives and fulfil daily demands. Unfortunately distinct systems cannot always fulfil the complex requirements demanded; hence distinct systems are now being integrated to form larger collaborating systems to meet intended requirements. These systems are defined as “Systems-of-Systems”, and are emerging in areas such as critical infrastructure, space exploration, and the military. This paper highlights some of the challenges faced when integrating distinct systems to form these large complex heterogeneous Systems-of-Systems, we identify Systems-ofSystems that have failed and the disastrous consequences. Our research also discusses how single points of failure can heavily impact collaborating systems abilities to fulfil objectives, which can result in them failing with disastrous consequences. Keywords - Systems-of-Systems; SoS; single points of failure; integrated systems; collaborating systems

I.

INTRODUCTION

“Systems-of-Systems” (SoS) are not new developments. Organisations within areas such as the military, manufacturing, transportation, critical infrastructure, and aerospace, have been researching and developing these large scale heterogeneous networks for several decades, after realising that a distinct system could no longer be developed to meet all required objectives. Thus distinct systems capable of being operated and managed independently are being integrated to form larger extended systems, to function on objectives that distinct systems could not fulfil on their own. These SoS have become highly popular as they not only allow complex objectives to be fulfilled, they can reduce financial cost of operation and maintenance, increase system robustness, performance, and reliability, and allow interoperability between new technology and legacy systems. The scale and size of these SoS over the last few decades has surprised many industry practitioners, particularly how rapidly they have been developed and deployed with many critical systems for example now being reliant upon them. Alarmingly systems controlling critical infrastructure for example have been forced to implement upgrades without taking systems off-line, due to society’s dependence upon them. Meaning as new requirements were identified, systems were merged on top of existing infrastructure like building blocks [1]. Our research has showed that SoS are still being heavily researched and developed yet complex challenges still need to

ISBN: 978-1-902560-27-4 © 2013 PGNet

be faced, and while SoS are formed by integrating physical components, SoS are only truly capable of collaborating upon objectives via the exchange of data, should data be restricted in any form, data has the potential within SoS to become a single point of failure. The remainder of this paper is organised as follows. In section II we provide background on Systems-ofSystems. Section III describes the challenges associated with Systems-of-Systems; and within section IV we discuss existing approaches that attempt to overcome these challenges. For section V we examine the impact single points of failure can have upon Systems-of-Systems, and in section VI we summarise and detail our areas for future work. II.

SYSTEMS-OF-SYSTEMS BACKGROUND

Academics and industry practitioners have remained divided upon a single definition to best describe all Systems-ofSystems. However Maier [2] defined three separate types of SoS (directed, virtual and collaborative) basing each upon the actual relationships perceived between the distinct systems which formed the SoS. While Dahmann and Baldwin [3] expanded upon this work and gave recognition to a fourth type of SoS (acknowledged) due to development and growth within the US Department of Defence systems. • Directed SoS are centrally managed to fulfil specific objectives, however their distinct systems operate independently, thus remaining subordinate to complete their managed objectives or new objectives defined by managers [2, 3]. An example is the Royal Naval collaboration between the ARTISAN 3D radar and Seawolf Mid-Life Update program, which delivers a SoS in an attempt to increase ship survivability. Both the radar and missile system can function independently, and potentially can simultaneously collaborate with other distinct systems to achieve objectives outside the scope of this SoS [4]. • Virtual SoS are neither centrally managed nor have a centrally agreed objective. Within these types of SoS largescale behaviours potentially can emerge, forcing the SoS to heavily rely upon invisible components to meet objectives [2, 3]. An example would be the World Wide Web, as it is physically and managerially distributed, and since its establishment no single organization has directly controlled it [2]. • Collaborative SoS do not have the ability to command systems directly to fulfil objectives from the central management, instead the distinct systems must volunteer to work in partnership to fulfil any agreed central requirement.

An example would be the Internet; this began life as a directed SoS however is now defined as a collaborative SoS due to its continuous evolvement. This is because organisations like the Internet Engineering Task Force developed standards for the Internet, however currently have no authority to enforce them [2, 3]. • Acknowledged SoS have defined management, resources, and established functions, though the collaborating systems each maintain their distinct functions, ownership, funding, and development. Should distinct systems evolve or require adaption, both the SoS and the distinct systems must collaborate. An example is the Department of Defence when SoS managers have no direct control over the collaborating distinct systems, instead they only have the ability to advise and influence other managers as the SoS evolves and new objectives are identified [3]. It makes no difference if the SoS has emerged in place, is pre-planned, has been long established, or created quickly in response to a critical incident, society is heavily reliant upon these SoS. They have been so integrated within our daily lives we don’t really notice they are there until one fails, and while we continue to invest in these large complex networks Systems-of-Systems are still failing. Such examples include the UK governments FiReControl program, which was hounded with challenges and incompetency which included spiralling costs, technical delays, poor management, and the project was too complex and would cost an estimated £860 million to provide the necessary resources to complete [5]. As well as the disastrous automation project for dispatching London Ambulances, this project was developed badly and was only cancelled after it had been operational for 48 hours in which time 20 people lost their lives [6]. These projects failed directly due to the challenges that still require research within SoS, which we will discuss in the following section. III.

SYSTEM-OF-SYSTEMS CHALLENGES

When combining distinct systems major challenges must be faced, if merged incorrectly then integrated systems could be combined with disastrous results, systems could be insecure and vulnerable to security attacks. Systems could become unstable, system wide crashing could occur, systems performance could diminish, and systems might fail to meet required objectives [7]. For this paper we categorise these challenges and the potential vulnerabilities they cause under the characteristics defined by Maier (operational independence, managerial independence, evolutionary development, emergent behaviour, and geographical dispersion) [8, 9]. Operational independence challenges include distinct systems maintain their independent operation, systems are established with unique polices, systems are compiled using varying components and security, and each system can be obliged to simultaneously function on objectives outside of the SoS. These challenges if not addressed can cause incompatibilities to develop causing conflicts within system security, operations, and their abilities to meet objectives, consideration must also be given to security as the system with the weakest can expose the entire SoS to potential attacks [9]. The Mars Climate Orbiter is an example of a SoS failing due to

operational independence; it was lost in the atmosphere of Mars due to the sheer complexity of the integrated systems, specifically because systems were independently developed. Collaborative testing never identified it measured in English units which directly resulted in its loss [10]. Fortunately distinct systems are not affected by operational independence unless they have been integrated within a SoS [9]. Managerial independence challenges include each system can be independently managed, systems prioritise their own objectives, systems can be altered via the management of other systems, and management can add, remove, or update systems without consultation. These challenges if not addressed can result in altered systems being unable to fulfil desired functions and objectives, if security has been altered then systems become vulnerable to attacks, any alterations can cause conflicts to arise thus impact system operations, and it could result in no organisation having the ability to control the SoS. Responding to these vulnerabilities and detecting them would be highly challenging, mainly due to the large number of collaborating systems which form each SoS [9]. The open day of Heathrow Terminal T5 is an example of a SoS failing due to managerial independence, baggage handling systems failed and 68 flights had to be cancelled, as one collaborating organisation delayed construction of the terminal and provided equipment late, thus no staff could test or train on the SoS. Collaborating organisations also failed to establish crisis management, consequently as the SoS failed it was difficult to respond and identify who was responsible, the systems involved and who should implement any solutions [10]. Evolutionary development challenges include the SoS can be deployed without being fully formed, the SoS continually evolves as new requirements are identified and objectives have been met, and functions and components can be phased in and out while the SoS is operational. These challenges if not addressed can cause an increase in emergent behaviour. Systems are more vulnerable to unknown and unpredicted security attacks, and during operation changes could be made that impact distinct systems objectives and their system components ability to function [9, 11]. The US Coast Guards acquisition program Deepwater is an example of a SoS being impacted due to evolutionary development. The tragic events of 9/11 forced Deepwater to re-evaluate its objectives and execute new requirements, forced partly due to new government legislation. Requirements included increasing security, incorporating chemical, radiological and biological defences and increasing communication with external agencies. Because evolutionary development was thrust upon the project, Deepwater struggled with rising costs and delays, eventually the Coast Guard took control of the program to salvage projects that still showed promise [11, 12]. Emergent behaviour can occur once a SoS has been deployed; this behaviour is considered to be outside of the distinct systems normal behaviour. Identifying the systems responsible, who should respond, who should find and implement solutions can be highly problematic, as well as ensuring mechanisms are robust to monitor the entire SoS in near real-time to ensure security breaches, system misbehaviour, and emergent behaviour is identified and reported effectively. If emergent behaviour develops within the

SoS then systems can become unpredictable, fail and repeatedly crash, severe disruptions can impact systems performance and their ability to fulfil objectives both inside and outside of the SoS, severe security vulnerabilities could also arise [9]. The chaos caused by NatWest Bank, Ulster Bank and RBS is an example of emergent behaviour causing a SoS to fail. In the summer of 2012 these three banks rolled out a software upgrade resulting in unpredictable abnormal behaviour developing, when the SoS failed customers were denied access to the entire banks resources, with many accounts inaccessible for several days [13]. While emergent behaviour is a major challenge to be faced, positive emergent behaviour can also occur within SoS. It can allow SoS to become more robust with systems altering their operations to ensure objectives are met should other systems become incapacitated. Research continues in this area to see if this behaviour can be identified and guided to harness its capabilities but is outside of the scope of this paper [14, 15]. Geographic behaviour is challenging as SoS can be geographically dispersed with collaborating systems being located in different jurisdictions and countries, meaning local laws and policies must be considered and applied. What is permitted in one country could be considered a crime in another, hence system location can directly impact how components function, security is applied and upheld, and can affect a systems ability to meet objectives [9]. Collaborating globally means different languages, time zones and jurisdictions can delay and hamper collaborating efforts between managers, delays can prevent objectives from being fulfilled, reduce system performance and cause weaknesses in security. Language can also impact the ability of managers to learn collaborators systems, as often the complex manuals describing the systems are developed in their native language and use local dialect and slang terms [10]. Research identified that a major challenge associated with geographical distribution is all distinct systems heavily rely upon networking capabilities, more accurately distinct systems only share data to collaborate and fulfil objectives. If data cannot be transmitted between components then the SoS could fail in its objectives. Hence ensuring data flow is vital, or data flow is at risk of becoming a single point of failure. The disastrous rescue attempt of AirFlorida Flight 90 in 1982 is an example of a SoS failing due to geographical dispersion. Federal, State and local agencies struggled to communicate, as systems had been developed independently, also contacting and integrating systems between agencies within other jurisdictions proved difficult. The delays severely hampered the rescue attempt resulting in only 5 people surviving the crash [10]. IV.

EXISTING APPROACHES

It has been identified that 72% of Europe is now wired and reliant upon digital infrastructure [23], and 95% of organisations have been either directly targeted or suffered with systems being compromised within Networks [21]. With SoS still facing problematic challenges research and development continues by both academics and industry practitioners. Systems-of-Systems face the same challenges as those of traditional networks, and while monitoring small networks can be difficult, the size and complexity of SoS

makes monitoring more taxing. While there is extensive research to combat the challenges associated with SoS often they rely upon high computational capabilities or cannot be scaled due to the sheer number of collaborating systems. We have researched SoS broadly looking at the challenges to be faced, and areas that we have reviewed with interest include SoS security, risk analysis, attack analysis and failure analysis [20, 25-31, 34]. Due to the sheer complexity of these large heterogeneous SoS and because of society’s dependence upon the infrastructure, a wide majority of research is purely theoretical. This is because academics and industry practitioners do not have the required resources available to them to test and implement their procedures. As recognized by the US army which is currently using Network Integration Events to test emerging technologies in a controlled lab based military environment. This is to strengthen the development and deployment process, as network managers are able to detect and diagnose risks before systems are deployed to troops. They have defined a Laboratory Based Risk Reduction method, allowing for network integration, design and allows automated analysis, this method allows issues to be detected, systems to be debugged, detect miss configuration, and allows upfront analysis of performance for the network. While this method provides numerous benefits, it is highly time consuming and expensive. However it is more effective than using modelling and simulation, as often these methods hide issues that later manifest upon deployment. Unfortunately small scales of networks tend to be inaccurate. Network loading, performance, and dynamics tend to not scale, and issues resulting from interoperability and loading tend to manifest after deployment [25]. The method they propose does not increase the resilience of any SoS they develop, as abnormal behaviour will often manifest long after a SoS has been operational for numerous years, and security vulnerabilities can be unknown and not imaginable at the moment a SoS is deployed. This approach truly only strengthens the development and initial deployment stages, highlighting the importance of continued research. To assure data security micro firewalls are also under development, while they are capable of being configured to recognise only relevant traffic, they require high computational power that the SoS might not be able to support. Intrusion detection and prevention systems can also monitor data being transmitted between integrated systems, however they are heavily reliant upon the signatures of known attacks, and require an in depth knowledge of the vulnerabilities associated with the protocols which are to be monitored. Intrusion detection systems also struggle to monitor some protocols, for example protocols such as DNP3 (distributed Network Protocol Version 3) and Modbus which can be relied upon within smart grids and critical infrastructure using Supervisory Control and Data Acquisition (SCADA) controls [17, 18]. Also Cryptography and key management continues to be heavily researched [31, 33], as it can assist in overcoming many of the challenges associated with ensuring data integrity. However while many SoS use open standards such as IPSec (Internet Protocol Security) and SSL (Secure Socket Layer) which are compatible with encryption, there are still many protocols and legacy systems currently in use that do not support such

features. For example DNP3 does not support authentication or encryption [17]. There has been considerable research conducted into attack analysis, specifically concentrating within intrusion detection and prevention. Unfortunately many of these methods only identify existing known attacks, and while other methods have been proposed they can be costly, time consuming, and require high processing capabilities. Furthermore anomaly based detection systems tend to identify a high number of false positives. Kim et al [26] proposed a method to visualise and analyse network attacks, with the assistance of parallel coordination. While visualisation can potentially identify previously unknown emerging attack patterns, it does not use mathematical formulas or any statistical algorithms, instead relies upon visualised patterns [26]. Whereas industry practitioners using attack profiles have conducted research into risk analysis using economic models, specifically developing profiles of attackers based upon evidence left behind after an attack. Evidence collected is then used to predict the type of the attacker, allowing for risk management policies to then be formulated for network security. Dantu et al. [20] propose a methodology for risk management based upon the behaviour of attackers. They define a five step model combing behaviour based attack graphs and Bayesian methodology to calculate risk levels for critical resources. Furthermore this method indicates it will rely upon the human element to implement solutions to identified vulnerabilities. V.

SINGLE POINTS OF FAILURE WITHIN SYSTEMS-OFSYSTEMS

Throughout our research under all the categories defined by Maier, a common element that frequently showed vulnerabilities was that of data. For example our research clearly identifies that data as a whole has the potential to be a single point of failure. If data cannot be created, stored or transmitted within the SoS then simply the SoS has the potential to fail in its entirety. It could be described that data has a life cycle, it is created, stored, used, shared, archived and then destroyed [16]. Yet data does not instinctively flow between components forming the SoS, many challenges have to be overcome to allow systems to be integrated and collaborate upon objectives, many of which were described above in section III. Academics and industry practitioners have invested heavily in research and development within SoS, to ensure collaboration and to guarantee data. However they tend to focus upon a specific area such as data security, they concentrate on a specific attack such as Distributed Denial of Services (DDOS) [32] or jamming attacks, or they examine risk analysis in regards to data flow. There has also been work undertaken in an attempt to identify issues once a SoS has been deployed, relying upon detection and analysis tools to finds anomaly’s occurring in near real-time, implementing solutions long after the event has occurred. Unfortunately while research continues, SoS are still being developed and deployed every year that fail. These failures cause organisations considerable financial loss, and can end with disastrous consequences.

With the advancement of communications and the Internet SoS are no longer isolated, organisations have increased their connectivity to become dynamic. Systems and networks are now connected via multiple access points (e.g. intranets, phone lines or Internet). Attackers with little knowledge or skill, and using freely available tools now take advantage of these connections by launching attacks and eavesdropping upon data as it is transmitted between components. Information such as authentication credentials, credit card transactions, email content, and control commands, can easily be exposed by listening to transmissions. Once an attacker has access to data transmissions they can launch simple attacks directly against data such as altering, corrupting, destroying or injecting false data, or they could simply impede the flow of data. Any alteration to the transmitted data packets could impede a SoS from fulfilling its objectives, thus creates a single point of failure [17, 18]. Attacks can include access control, injection and execution of malicious software or data, object reusability, masquerading attacks, sniffing, snooping, and DDOS attacks. While managers can implement several solutions in conjunction in an attempt to secure systems and ensure data flow, integrity, security and availability, there is currently no single solution that guarantees total security. Incorporating security features within a SoS does not ensure that data communications will be secure. Features currently employed include intrusion detection and prevention systems, firewalls, virtual private networks, content filtering, antivirus solutions, access control, and cryptography and key management. Implementing such features can also be time consuming as not all security solutions are automated, instead hosts must be correctly configured, polices must be periodically updated, and every integrated system must be secure or has the potential to be a point of attack [17, 19-21]. While it could be perceived that data is at its most vulnerable state during its transmission across the SoS, we look at data in its entirety and recognise there are many weaknesses that can truly affect data within SoS. Data is not just at risk from malicious attacks by outsiders, data can be at risk from legitimate user error, components within the SoS, the physical structure of the network and Internet, as well as natural disasters [22]. For example data can become corrupt via system components during its creation and processing, also malicious attackers from within the SoS can alter, corrupt or delete data just as easily as a legitimate users unintentional actions [23]. Data stored in components that are deployed in remote areas, unguarded and accessible by attackers, potentially can suffer power loss, be damaged by natural occurrences, have difficulty connecting to SoS due to loss of connectivity, and attackers can remove components or destroy them. This can affect the flow and availability of data, malicious attackers can gain physically access to stored data within the components along with jeopardising its integrity, and also it gives attackers the opportunity to inject false data into the SoS [17, 18]. Attackers are also exploiting highly publicised vulnerabilities associated with open standards, and off the shelf hardware and software, they are using freely available tools to launch directed attacks, eavesdrop upon network traffic [17], and in the future could exploit gateways that were previously isolated

and do not support security features such as authentication, confidentiality, integrity, and data privacy [24]. These are just a few of the challenges we have identified during our research that impact data directly, and have the potential to heavily impact the resilience of SoS. We also clearly identified that the challenges discussed in section III associated with operational independence, managerial independence, evolutionary development, emergent behaviour, and geographical dispersion can also cause single points of failure within SoS and have the ability to impact data directly. Any failure or vulnerability that impacts a component within the SoS which prevents data being created, stored, or transmitted can be considered a single point of failure, as it has the potential to prevent a SoS from fulfilling its complex objectives. To overcome some of the challenges mentioned in this section many organisations are turning to third parties (e.g. migrating to the cloud), in an attempt to circumvent some of the vulnerabilities and to assure data integrity, availability and security. While third parties offer services as a low cost solution they unfortunately just provide a new platform for malicious attackers to exploit. More concerning is the fact that organisations once they have outsourced their services rely heavily upon data flow to function, and these operations become a weak link thus create a new single point of failure that can impact the organisation and its ability to function [17, 18, 33].

span of the SoS. Hence solutions can be established and applied before the issues develop into devastating failures, increasing the SoS resilience. The objectives for the project will be achieved by examining and evaluating current risk and failure analysis approaches, investigating closely their methods, and identifying the areas in which they fail. This will allow us to develop our own methodology, which will assist us in developing a suitable strategy technique or approach that is required to identify single points of failure within SoS. This will include identifying and quantifying single points of failure during the development and deployment of the SoS. Also the technique would be required to continuously monitor the entire SoS once deployed and in an active state to ensure resilience, by predicting or identifying emergent behaviour that has the potential to develop in to a single point of failure. Unlike current approaches we will not focus upon a single type of SoS, particular challenge, or vulnerability. Instead we aim to develop a suitable technique that can be applied to various SoS types, and assess risks without the requirement of a complete view of the SoS. This will be beneficial due to the fact SoS can be deployed without fully knowing their end objectives. Our technique will also be required to constantly monitor the SoS in near real-time, identifying risks while objectives are met and new requirements are identified. REFERENCES

VI.

SUMMARY

Our research surveyed the associated challenges academics and industry practitioner’s face when developing and deploying Systems-of-Systems. These challenges are highly problematic and when we compare SoS that have been unsuccessfully deployed, we can accurately associate their failings in comparison to these vulnerabilities. Through the examination of the current research, we have found there are considerable challenges to overcome when integrating distinct systems to form these large complex heterogeneous SoS, many of which were described in section III and V. Numerous approaches have been implemented, yet still SoS fail and are left exposed and vulnerable, resulting in objectives not being meet and occurring huge financial losses. Many approaches tend to concentrate on one small specific area or vulnerability, they are highly theoretical, and they impact heavily upon system resources, or are not scalable due to the sheer number of collaborating components. Our research clearly identifies that these methods are insufficient and continued research and development is required for these large complex networks. Despite our project being in its early stages we recognise the impact that singe points of failure have upon the SoS ability to fulfil requirements. Our future work will focus upon the idea that everything within a SoS has the potential under the right circumstances to be a single point of failure. We hope to increase the resilience of the SoS by quantifying and identifying the risks of single points of failure during the SoS development, deployment, and operation. We acknowledge that not all risks can be prevented; however our aim is to develop and apply our strategy technique, thus have the ability to anticipate and identify the potential risks for the entire life

[1]

DeLaurentis, D.A, “A taxonomy-based perspective for systems of systems design methods”, 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 1, pp. 86- 91, 10-12 Oct. 2005. [2] Maier, M, “Architecting Principles for Systems-of-Systems”, Systems Engineering, vol. 1, pp. 267-284, 1998. [3] Dahmann, J. S, Baldwin, K.J, “Understanding the Current State of US Defense Systems of Systems and the Implications for Systems Engineering”, 2008 2nd Annual IEEE Systems Conference, pp. 1-7, 7-10 April 2008. [4] BAE Systems, “ARTISAN 3D Medium Range Radar Type 997” [online] http://www.baesystems.com/product/BAES_021203/artisan-3dmedium-range-radar-type-997 . [accessed 20th April, 2013]. [5] Ncube, C, "Causal factors behind the failed firecontrol project: A largescale system-of-systems”, 2012 7th International Conference on System of Systems Engineering (SoSE), pp. 247-252, 16-19 July 2012. [6] Bar-Yam, Y, "When systems engineering fails-toward complex systems engineering”, IEEE International Conference on Systems, Man and Cybernetics, 2003. vol. 2, pp. 2021-2028, 5-8 Oct. 2003. [7] Ncube, C, "On the engineering of systems of systems: Key challenges for the Requirements Engineering community", 2011 Workshop on Requirements Engineering for Systems, Services and Systems-ofSystems (RESS), pp. 70-73, 30-30 Aug. 2011. [8] Karcanias, N, Hessami, A.G, “System of Systems and Emergence Part 1: Principles and Framework”, 2011 4th International Conference on Emerging Trends in Engineering & Technology, pp. 27-32, 18-20 Nov. 2011. [9] Waller, A, Craddock, R, “Managing runtime re-engineering of a Systemof-Systems for cyber security”, 2011 6th International Conference on System of Systems Engineering (SoSE), pp. 13-18, 27-30 June 2011. [10] Naqvi, A, Chitchyan, R, Zschaler, S, Rashid, A, Südholt, M, (2010) “Cross-Document Dependency Analysis for System-of-System Integration”, C Choppy & O Sokolsky (eds), in: Foundations of Computer Software. Future Trends and Techniques for Development: 15th Monterey Workshop 2008, Budapest, Hungary, September 24-26, 2008, Revised Selected Papers. Lecture Notes in Computer Science, vol.

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21] [22]

[23]

[24]

6028, Springer, Berlin, pp. 201-226, Monterey Workshop 2008, 1 January. Barrett, T.J, "Deepwater methods to address system of system risks”, 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 1, pp. 92- 96, 10-12 Oct. 2005. Sarah Chacko, (2012) “Coast Guard acquisition chief: Deepwater dead” [online]http://www.navytimes.com/news/2012/01/coast-guarddeepwater-dead-says-acquisition-chief-010512w/. [accessed 1st April, 2013]. David Wilkes, (2013) “Chaos at cash machines: Computer fault halts RBS withdrawals… While the boss picks up £700,000 bonus”, [online] http://www.dailymail.co.uk/news/article-2289358/Computer-fault-haltsRBS-withdrawals--While-boss-picks-700k-bonus.html. [accessed 1st April, 2013]. DeLaurentis, D.A, "A taxonomy-based perspective for systems of systems design methods", 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 1, pp. 86-91, 10-12 Oct. 2005. Ke-wei Yang, Ying-Wu Chen, Yan-jing Lu, Qing-song Zhao, "The study of guided emergent behavior in system of systems requirement analysis", 2010 5th International Conference on System of Systems Engineering (SoSE), pp. 1-5, 22-24 June 2010. Cloud Security Alliance: Security Guidance for Critical Areas of Focus in Cloud Computing V2.1 [online] https://cloudsecurityalliance.org/csaguide.pdf. [accessed 26th April 2013]. Igure, V.M, Laughter, S.A, Williams,R.D, “Security issues in SCADA networks”, Computers & Security, Volume 25, Issue 7, pp. 498-506, October 2006. Zhuo Lu, Xiang Lu, Wenye Wang, Wang, C, "Review and evaluation of security threats on the communication networks in the smart grid", MILITARY COMMUNICATIONS CONFERENCE, 2010 MILCOM 2010, pp. 1830-1835, Oct. 31 2010-Nov. 3 2010. Kumar, P.A, Selvakumar, S, “Distributed Denial-of-Service (DDoS) Threat in Collaborative Environment - A Survey on DDoS Attack Tools and Traceback Mechanisms”, 2009 IEEE International Advance Computing Conference IACC 2009, pp. 1275-1280, 6-7 March 2009. Dantu, R, Loper, K, Kolan, P, "Risk management using behavior based attack graphs”, International Conference on Information Technology: Coding and Computing, 2004. Proceedings. ITCC 2004, vol. 1, pp. 445449, 5-7 April 2004. FireEye, (2013), “Next-Generation Threats” [online] http://www.fireeye.com/threat-protection/. [accessed 3rd April, 2013]. Yue Yu, Fry, M, Schaeffer-Filho, A, Smith, P, Hutchison, D, "An adaptive approach to network resilience: Evolving challenge detection and mitigation”, 2011 8th International Workshop on the Design of Reliable Communication Networks (DRCN), pp. 172-179, 10-12 Oct. 2011. Verizon, (2013), “2013 Data Breach Investigations Report” [online] http://www.verizonenterprise.com/DBIR/2013/ . [accessed 23rd April 2013]. Abdallah, A, Feron, E.M, Hellestrand, G, Koopman, P, Wolf, M, "Hardware/Software Codesign of Aerospace and Automotive

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

Systems", Proceedings of the IEEE, vol. 98, no. 4, pp. 584-602, April 2010. Badger, M, Bushmitch, D, Agnish, V, Cozby, R, Fikus, J, Halloran, F, Chang, K, McCabe, P, Erramilli, S, "Laboratory-based end-to-end network System of Systems Integration, design and risk reduction: Critical activity for System of Systems Integration Directorate and the Army”, MILITARY COMMUNICATIONS CONFERENCE, 2012 MILCOM 2012, pp. 1-6, Oct. 29 2012-Nov. 1 2012. Hoin Kim, Inyong Lee, Jaeik Cho, Jongsub Moon, "Visualization of network components for attack analysis”, IEEE Symposium on Computational Intelligence in Cyber Security, 2009. CICS '09, pp. 18, March 30 2009-April 2 2009. Despotou, G, Alexander, R, Kelly, T, "Addressing challenges of hazard analysis in systems of systems", 2009 3rd Annual IEEE Systems Conference, pp. 167-172, 23-26 March 2009. Bagheri, E,Ghorbani, A.A, "Risk Analysis in Critical Infrastructure Systems based on the Astrolabe Methodology", CNSR '07. Fifth Annual Conference on Communication Networks and Services Research, 2007. pp. 335-344, 14-17 May 2007. Priya S, Lambert, J.H, "Risk-based model for tracking complexity in system vulnerability analysis", Proceedings of the 2004 IEEE Systems and Information Engineering Design Symposium, 2004. pp. 65-70, 1616 April 2004. Jayadevappa, B, Soh, B, "A new risk analysis method for data backup strategy", 2009 IEEE Region 10 Conference (TENCON), pp. 1-6, 23-26 Jan. 2009. Cao, H, Zhu, P, Lu, X, Gurtov, A, "A layered encryption mechanism for networked critical infrastructures", IEEE Network, vol. 27, no. 1, pp. 12-18, January-February 2013. Aroua, M.K., Zouari, B, "A Distributed and Coordinated Massive DDOS Attack Detection and Response Approach", 2012 IEEE 36th Annual Computer Software and Applications Conference Workshops (COMPSACW), pp. 230-235, 16-20 July 2012. Kumar, A, Lee, B.G, Lee, H, Kumari, A, "Secure storage and access of data in cloud computing”, 2012 International Conference on ICT Convergence (ICTC), pp. 336-339, 15-17 Oct. 2012. Van Dyk, R.N, Pariseau, D.H, Dodson, R.E, Martin, B.T, Radcliffe, A.T, Austin, E.A, Haimes, Y.Y, Andrijcic, E, Zhenyu Guo, Werner, J.H, "Systems integration of Unmanned Aircraft into the National Airspace: Part of the Federal Aviation Administration Next Generation Air Transportation System”, 2012 IEEE Systems and Information Design Symposium (SIEDS), pp. 156-161, 27-27 April 2012.