Middleware and Programming Support Bridging the ...

Middleware and Programming Support Bridging the Cyber-Physical-Social Divide Olga Murdoch, B.Sc

The thesis is submitted to University College Dublin in fulfilment of the requirements for the degree of Doctor of Philosophy in the School of Computer Science and Informatics

Head of School: Prof. Padraig Cunningham Principal Supervisor: Dr. Rem Collier Second Supervisor: Prof. Gregory O’Hare

June 2015

Acknowledgements TLDR1 : I am lucky enough to have so many people to thank that you might not want to read all of this. I have often joked about my history of supervisors ‘abandoning’ me but, truth be told, I am lucky to have had the experience of working with so many talented academics. My sincere thanks to all of my PhD supervisors (past and present), Prof. Simon Dobson, Prof. Paddy Nixon, Dr. Rem Collier, and Prof Gregory O’Hare. I am extremely grateful for the support and guidance you have each given me throughout my PhD (and Simon and the rest of the SRG gang for all your work with me from long before that). I would also like to thank my PhD chair and examiners, Prof. Tahar Kechadi, Dr. David Lillis and Prof. Maurice Mulvenna, for their time, effort, and most importantly, ticking the ’pass’ box. I am particularly grateful to Dr. David Lillis for the thorough feedback he gave me towards my final thesis. It would be unfair if I didn’t also thank Dr. Debbie in that regard. The support and guidance I have received throughout my studies extends beyond those I have already mentioned. Poor Dr. Cormac Doherty has had to listen to me rant about my PhD topic for many years now - and he totally gets it! One of the most intelligent people I have the pleasure of knowing and a person that will most definitely impact our world - in a good way of course. Thank you for everything, Cormac. Henry McLoughlin, my dear friend and an outstanding lecturer in UCD Computer Science, thank you for all the years of listening, cheering up and advising. You are a rock to so many people and your enthusiasm for your work is contagious. A fantastic influence on any budding academic. Prof. Joe Carthy, my current boss and the Head of School for many years of my studies in UCD Computer Science. Thank you for always offering sound adivce, for being so supportive and for all of the opportunities you have given me. John Dunnion, another former Head of School, thank you for the support and kindness you have shown me over the years. There really is more people in Computer Science than I could list, each of whom have taken an interest in me as a PhD student, shown me support and offered me kind words throughout this journey. You know who you are, thank you! Throughout my PhD I had the fortune to work with many people, most recently being involved in a group project with some fantastic individuals from whom I have learned so much. Dominic Carr, Sean Russel, Barnard Croon, Levent Gorgu, Abey Campbell, David Lillis, Tadhg O’Sullivan, Jessie Wan, and Lina Xu - it has been a pleasure to work with each of you and thank you for all the support, proof reading, software testing and help meeting deadlines! I hope we get the opportunity to work together again. TOM has been a rock for me throughout the final stages of my thesis. Maeve O’Connell has been beyond supportive, always saying the right things and making me laugh, yet 1

http://lmgtfy.com/?q=tldr

i

always being strict and giving out if I don’t stay focussed. There have been days that I might not have remained quite as calm and collected without Maeve by my side. Even thesis submission day when she and Tadhg helped me scrape past the deadline. Thank you Maeve, for your unwaivering support, for believing in me when I didn’t so much, and for pushing me over the edge when it seemed a little too scary. Special thanks for all your hard work searching for typos before both of my submissions. Thank you also to Gerald Mills for your support and regular thesis checkups. I don’t think I ever told you but it was quite motivating knowing you were keeping an eye on things. My friends - I wouldn’t dare try to list you all - some of you I have known since my school days and others I have met over the past decade in or through people in UCD. Even when I have lost touch with you for months at a time I always knew you were there for me and that I could pick up the phone or turn up at your door any time. Thank you for that. Many of you have spent hours listening to me as I chatted away about things you didn’t understand or care about. Thanks for humouring me! Life hasn’t been easy for some of you over the years and I know I haven’t been there as much as I would have liked. I hope you know that I always have and always will hold you close to my heart. Your loyalty to me and faith in me has given me great strength over the years and I am so grateful to have each and every one of you in my life. My wonderful boyfriend, Pat. You have really put up with it from me haven’t you. Thanks for being so patient, understanding, supportive and helpful while I have been finishing up my PhD. It can’t have been easy to put up with my stress at times but, luckily, that hasn’t seemed to put you off. You’ve done so much for me and Jack over the past year or so and we are both so grateful, happy and lucky to have you in our lives. My family - I have a great family. You have always been there for me through everything, including my PhD. Without your support I wouldn’t have made it through my undergraduate degree, never mind writing this thesis. Thank you for all your support, financial, emotional and otherwise. Thanks for giving up so much of your time to mind Jack, so that I could work - and play. I love you all dearly. The most important person I want to acknowledge is my son. Jack - Thank you for putting up with me always being so busy, always being in a rush and always using my PhD as an excuse for being tired or unable to do fun things. You will be happy to know that I don’t have that excuse anymore. Thank you for all your hugs, love and laughter every day. Whatever I have acheived in my life so far doesn’t come close to your acheivements in your 12 years so far. I couldn’t be more proud of the wonderful boy you are now and the amazing man I can see you growing into. You are an inspiration to me and I know you will inspire many others throughout your lifetime. As this journey ends for me, a new one begins for you and I know it will be amazing. I love you more than words could ever say - and then some.

ii

Statement of Original Authorship I hereby certify that the submitted work is my own work, was completed while registered as a candidate for the degree stated on the title page and I have not obtained a degree elsewhere on the basis of the research presented in this submitted work. The thesis work was conducted from September 2008 - September 2014 under the supervision of Professor Simon Dobson (2008 - 2009), Professor Paddy Nixon (2009 2010) , Doctor Rem Collier (2010 - 2014) and Professor Gregory O’Hare (2008 - 2014) at University College Dublin. The research was conducted with several collaborators. The key collaboration was with Dominic Carr who was responsible for the design and implementation of the core SIXTH Sensor Middleware and its management services. The design of the core middleware was influenced by the contributions of this thesis from the outset.

iii

Contents 1 Introduction 1.1

1

Context and Context-Aware Computing . . . . . . . . . . . . . . . . . . . .

3

1.1.1

Rethinking Context . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

Ubiquitous Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

1.2.1

Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . .

5

1.2.2

Middleware and Programming Methodologies . . . . . . . . . . . . .

6

1.3

Evolution of the World Wide Web . . . . . . . . . . . . . . . . . . . . . . .

6

1.4

Motivation: Bridging the Cyber-Physical-Social Divide . . . . . . . . . . . .

7

1.5

Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

1.6

Core Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

1.7

Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.2

2 Cyber-Physical-Social Computing Emerging 2.1

2.2

2.3

2.4

12

What is Cyber-Physical-Social Computing? . . . . . . . . . . . . . . . . . . 14 2.1.1

Existing Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.1.2

CPS Computing for Context-Awareness . . . . . . . . . . . . . . . . 17

The World Wide web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.1

Social Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.2.2

Web of Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.2.3

The Semantic Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.2.4

The Future web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Ubiquitous Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.3.1

Ubiquitous Technologies and Applications . . . . . . . . . . . . . . . 29

2.3.2

Internet/Web of Things . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.3.3

Sensor Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Discussion: Moving Forward . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.4.1

Challenges and Benefits of CPS Computing . . . . . . . . . . . . . . 35

2.4.2

CPS Computing Infrastructure . . . . . . . . . . . . . . . . . . . . . 37

2.4.3

Role of a Supporting Framework for CPS Computing

iv

. . . . . . . . 40

3 Realising Cyber-Physical-Social Computing

42

3.1

SOA Requirements for CPS Context-Awareness . . . . . . . . . . . . . . . . 43

3.2

Approaches to Bridging the CPS Divide . . . . . . . . . . . . . . . . . . . . 44

3.3

3.2.1

WWW Accessible Devices and Networks . . . . . . . . . . . . . . . . 45

3.2.2

Utilising and Extending Social Networks . . . . . . . . . . . . . . . . 46

3.2.3

Extension of the Sensor Middleware Paradigm . . . . . . . . . . . . 47

3.2.4

Evaluation of Existing Approaches . . . . . . . . . . . . . . . . . . . 48

Towards Programming Support for CPS Context-Acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.4

3.3.1

Web Service Mashup Development . . . . . . . . . . . . . . . . . . . 51

3.3.2

WSN Programming Abstractions . . . . . . . . . . . . . . . . . . . . 55

Discussion: Informing Design Decisions

. . . . . . . . . . . . . . . . . . . . 60

4 Design of a Supporting Framework for Cyber-Physical-Social Computing

62

4.1

Basic WWW Building Blocks for CPS Middleware . . . . . . . . . . . . . . 63 4.1.1

Sources of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4.1.2

Data Processing Services

. . . . . . . . . . . . . . . . . . . . . . . . 69

4.2

CPS Sensing: Leveraging the Cyber Sensor Paradigm . . . . . . . . . . . . 70

4.3

Cyber Sensor Pipelines: Integrating the Web of Services . . . . . . . . . . . 74

4.4

Programming Abstractions for CPS Sensor Networks . . . . . . . . . . . . . 75

4.5

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

5 Detailed Design and Implementation 5.1

SIXTH: Sensor Web Middleware . . . . . . . . . . . . . . . . . . . . . . . . 82 5.1.1

5.2

5.3

5.4

81

SIXTH Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Key SIXTH Components

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

5.2.1

IAdaptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

5.2.2

SIXTH Sensor Adaptor . . . . . . . . . . . . . . . . . . . . . . . . . 87

5.2.3

Virtual Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

5.2.4

SIXTH Sensor Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Extending SIXTH for Cyber Sensor Pipelines . . . . . . . . . . . . . . . . . 91 5.3.1

Cyber Sensor Adaptor and Cyber Sensors . . . . . . . . . . . . . . . 93

5.3.2

Network and Infrastructure Programming Abstractions . . . . . . . 98

5.3.3

SIXTH Pipe Adaptor . . . . . . . . . . . . . . . . . . . . . . . . . . 101

5.3.4

Developer Interactions with SIXTH . . . . . . . . . . . . . . . . . . 104

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

v

6 Evaluation Part 1: Case Studies

111

6.1

Developing an Adaptor to monitor a Web Environment . . . . . . . . . . . 113

6.2

Ubiquitous Mapping Application . . . . . . . . . . . . . . . . . . . . . . . . 118

6.3

Monitoring Real World Disasters: Tasmania 2013 . . . . . . . . . . . . . . . 120

6.4

Monitoring and Analysing Medical Epidemics . . . . . . . . . . . . . . . . . 122

6.5

Autonomous Community Curation . . . . . . . . . . . . . . . . . . . . . . . 129

6.6

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

7 Evaluation Part 2: Usability Studies

135

7.1

ISO 25010 Usability Characteristic . . . . . . . . . . . . . . . . . . . . . . . 137

7.2

Usability Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

7.3

Usability Study Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

7.4

Usability Study Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

7.5

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

8 Conclusions and Future Work 8.1

149

Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

Appendices A

153

Programming Abstraction Properties . . . . . . . . . . . . . . . . . . . . . . 153

A.1 Network Abstraction Properties . . . . . . . . . . . . . . . . . . . . . . . . . 154 A.2 Infrastructure Abstraction Properties . . . . . . . . . . . . . . . . . . . . . . 154 B

Pipe Types Available in SIXTH . . . . . . . . . . . . . . . . . . . . . . . . . 158

C

SIXTH XML Tasking Messages . . . . . . . . . . . . . . . . . . . . . . . . . 161

D

SIXTH client API Javadocs . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

E

Evaluation Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

vi

List of Figures 1.1

Simplified view of data to situation inference process . . . . . . . . . . . . .

4

2.1

Prototype Architecture of Cyber-Physical Systems . . . . . . . . . . . . . . 14

2.2

Physical and cyber world interactions . . . . . . . . . . . . . . . . . . . . . 15

2.3

Physical -Cyber-Social Computing . . . . . . . . . . . . . . . . . . . . . . . 16

2.4

Cyber-Physical-Social Computing . . . . . . . . . . . . . . . . . . . . . . . . 17

2.5

Semantic Web 3.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.6

Overview of IoT Visions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.7

Global Sensor Web Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.8

Sensor Web layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.9

Open Web 3.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.10 Middleware Enablement for CPS web > 3.0 . . . . . . . . . . . . . . . . . . 40 4.1

Taxonomy of WWW-based Sources of Context . . . . . . . . . . . . . . . . 64

4.2

Node Level Abstraction for Cyber Sensor Pipelines . . . . . . . . . . . . . . 76

4.3

Network Level Abstraction for Cyber Sensor Pipelines . . . . . . . . . . . . 77

4.4

Infrastructure Level Abstraction for Cyber Sensor Pipelines . . . . . . . . . 78

5.1

SIXTH Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

5.2

UML Class diagram illustrating the IAdaptor, ISensorAdaptor and IPipeAdaptor Interfaces in SIXTH . . . . . . . . . . . . . . . . . . . . . . . . . 88

5.3

Abstract Sensor Adaptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

5.4

Adaptor Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

5.5

ISensorNode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

5.6

Sensor Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

5.7

CPS extensions to SIXTH core . . . . . . . . . . . . . . . . . . . . . . . . . 92

5.8

AbstractCyberSensorAdaptor . . . . . . . . . . . . . . . . . . . . . . . . . . 93

5.9

Cyber Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

5.10 Cyber Sensor Data and Data Model . . . . . . . . . . . . . . . . . . . . . . 96 5.11 CyberSensorFactory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.12 AbstractAdaptorStream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

vii

5.13 Network Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.14 Infrastructure Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.15 Pipe Adaptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.16 IPipeService . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 5.17 SIXTH DiscoveryReceiver . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.18 SIXTH client API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.19 AbstractApplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.20 Building Blocks for developers . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.21 Creation of a Cyber Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.1

Ubiquitous Mapping Application . . . . . . . . . . . . . . . . . . . . . . . . 118

6.2

Tasmania Bush Fire Case Study . . . . . . . . . . . . . . . . . . . . . . . . 123

6.3

Medical Epidemic Mapping Application . . . . . . . . . . . . . . . . . . . . 127

6.4

Medical Epidemic Analysis Application . . . . . . . . . . . . . . . . . . . . 128

6.5

Epidemic Monitoring Indonesia . . . . . . . . . . . . . . . . . . . . . . . . . 129

7.1

SUS Grade Rankings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

7.2

Network-Level Abstraction Survey . . . . . . . . . . . . . . . . . . . . . . . 142

7.3

SUS Score Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

7.4

Individual SUS Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

7.5

Participant Expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

viii

List of Tables 3.1

Middleware characteristics with respect to CPS sensor middleware requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.1

Classification of WWW-based Context Sources . . . . . . . . . . . . . . . . 65

4.2

Cyber Sensor Programming Abstractions . . . . . . . . . . . . . . . . . . . 76

5.1

SIXTH evaluation as a CPS Middleware . . . . . . . . . . . . . . . . . . . . 85

6.1

Functionality of thesis contributions in case study demonstrations . . . . . . 112

ix

x

Abstract Context-Aware Computing, enabled by ubiquitous sensing technologies, involves building highly responsive, self-adaptive and personalised systems. Over two decades since Mark Weiser’s seminal paper, The computer for the 21st century, outlining the Ubiquitous Computing vision, the World Wide Web (WWW) has evolved into a modern source of personal and informative data, applications and services. Bridging the divide between the WWW and the sensor networks that traditionally underpin Ubiquitous Computing will enable Cyber-Physical-Social (CPS) context-awareness. Context-Aware Computing requires a unifying framework to support the developement of systems driven by CPS context. Enhanced by access to more comprehensive context, CPS context-aware systems will benefit from optimised performance and greater usability. This thesis reflects on the evolution of WWW and Ubiquitous Computing research with an aim to defining this problem space for developers of future context-aware applications. Leveraging techniques from both broad fields, middleware and programming support for unified CPS context-awareness is designed which adopts a Cyber Sensor Pipeline approach to context acquisition. Wireless Sensor Network (WSN) programming abstractions are applied to the Cyber Sensor concept, enabling the rapid development of CPS context acquisition applications. Usability and case study evaluations validate the effectiveness of the approach as a tool for intuitive and rapid CPS context-acquisition.

i

Chapter 1

Introduction In a modern data-saturated world, knowledge- and inference-driven systems integrate intelligence into real-time services and devices allowing the provision of diverse, self adaptive, autonomous, personalised and intelligent computing applications and systems. This is the vision of Ubiquitous and Context-Aware Computing research, where the aim is to adapt the environment of a user or entity according to the current state of the environment, based on context information relating to the target user or entity and its inferred needs. Ambient Intelligence in particular demands unobtrusive deployments of technology that enable autonomous environments through the provision of in-situ intelligence and decision-making techniques. While traditional Wireless Sensor Network (WSN) and middleware research focuses on enabling this goal, the World Wide Web (WWW) has become a modern day source of personal and informative, real-time and archive data. Through user participation in social networks and use of personalised Web Services, the Social Web has become an extension of our everyday lives. Users self-report through social updates, share their location through ‘check-in’ services and build communities by connecting with other users of the Social Web. Social and non-social websites and services provide real-time reports of topical information, breaking news and environmental information such as weather reports and regional disaster alerts. Due to privacy and security concerns, future personal social networks are likely to be removed from ‘the cloud’ and operate on a peer-to-peer basis. Further, virtual and augmented reality research is beginning to enable real interactions with environments that are very different to the physical world we live in. Acknowledging these advances in technology requires us to extend our understanding of environments and what that means for Ubiquitous and Context-Aware Computing. An environment can be considered to be the geographical location and immediate surroundings within which a user or entity currently resides, or equally the software environment that a user or entity is engaged with or immersed in. A truly context-aware environment will be maintained by an abundance of ubiquitous technologies that support 1

multiple simultaneous applications in delivering services that meet the needs of all of its users. Sensor networks now need to provide real-time insights into both our physical and our software environments. Such sensor networks will bridge the Cyber-Physical-Social (CPS) divide, providing a more complete contextual foundation upon which intelligent decision-making techniques will perform. However, achieving this goal poses a very difficult challenge given the wide breadth and diversity of environments, types and behaviours of entities and users, and sources of data within these environments. Modern middleware and programming frameworks must accommodate application developers by managing diverse environments and data types while providing an intuitive programming interface that does not impose limitations on potential applications. It must also be forward thinking, accommodating of changes in technology trends such as the inevitable move from Cloud Computing to distributed peer-to-peer systems, and place emphasis on both ease-of-use and data control for end-users. Existing approaches towards bridging the CPS divide can be found in Web of Things (WoT) and Sensor Web research literature. Both paradigms envision connectivity of entities and users in web-based environments through adoption of internet enabled devices accessible through standardised interfaces. While the goals and approaches of WoT and Sensor Web research agendas are now converging, WoT approaches typically focus on connectivity of uniquely addressable devices while the Sensor Web paradigm is concerned with the adoption and connectivity of large scale heterogeneous sensor-actuator networks. Such networks support intelligent sensing and actuating of entities while treating all entities as equal citizens regardless of their origin, form or nature. While internet-enabled devices, sensor-actuator networks and the WWW form the underlying infrastructure for WoT and Sensor Web research, middleware and programming frameworks, incorporating data access and representation standards, provide a supporting framework for the development and deployment of applications enabled by this infrastructure. Supporting frameworks are required to ease the programming and deployment of sensor networks and enable the rapid development of applications through the provision of a programming model that removes the complexities inherent in programming with large volumes of data from diverse sources. There is a wealth of middleware research in the literature categorised under the broad and overlapping fields of Internet of Things (IoT), WoT, Sensor Web, WSN, Ubiquitous Computing, Pervasive Computing, Context-Aware Computing research. These offerings largely solve distinct problems or for specific domains, or only support the use of physical sensors, homogeneous in type. A generalised middleware approach has been argued for and repeatedly dismissed as unachievable without compromising application requirements. It has been argued that specific problems are best solved by custom and specialised techniques rather than generalised approaches that may provide a functional technique but is not optimised or guaranteed to perform well for all scenarios. More recently, Service

2

Oriented Middleware has been acknowledged as a solution to the heterogeneity problem, enabling interoperability of loosely-coupled diverse services. Despite advances towards service oriented frameworks for WoT and Sensor Web middleware, a single end-to-end support framework for CPS Computing does not yet exist. An introduction to Context-Aware computing (Section 1.1), Ubiquitous Computing (Section 1.2) and the WWW (Section 1.3) follows, informing the motivation (Section 1.4) and objectives (Section 1.5) of this thesis.

1.1

Context and Context-Aware Computing

Context-Aware Computing involves building highly responsive self-adaptive systems that aim to increase the integration of technology into the fabric of everyday living [176]. Practical applications include smart spaces and environmental monitoring. Smart spaces aim to provide ubiquitous (hidden) services such as ambient personalised displays, mood lighting, and pervasive services by embedding intelligence in devices within a utilised space such as a home or office. Environmental monitoring systems gather widespread data relating to the conditions of an environment, enabling the development of intelligent ecosystems that react to diverse phenomena such as pollution, wildlife migration, natural disaster or intrusion. “Context” is defined as any information that can be used to characterise the situation of any person, place or object of relevance to a context-aware application [6, 46]. This is now the widely-accepted definition of context, previous definitions having been too specific or constrained [16]. This is also the case in terms of CPS Computing, which requires a broadened understanding of situations, which this definition satisfies. Situational context can be defined in terms of location, identity, time, and activity, which are considered primary context types that not only characterise a situation but also act as indices to further information. While the first three context types can have concrete, known, values; activities are, like situations, more abstract and usually need to be inferred from their characterising context as a situation does. The context of an activity is the information that characterises that activity and no other information. For example, a user sending an email from a smart phone is an activity for which the user, the phone, the application and the user interactions form the context. Assuming this user is in a room with other people, the information about those people and the environment is considered peripheral context; which characterises the wider situation within which the activity is taking place. For example, given the identity of the people in this example, the relationships between those people can be identified. These relationships are then considered secondary context, i.e. information that can be acquired on the basis of having primary context information [6].

3

Figure 1.1: Simplified view of data to situation inference process Context-aware applications are those that respond to context either by self-adapting or autonomously adapting the services it provides to its users. The data collection and processing tasks may be performed by a context-aware system itself, or more commonly, by a supporting framework that feeds information to the application. Context-aware applications can be driven by context information or more abstract information such as inferred activities and/or situations. The process of transforming raw data to a situation is illustrated in a simplified form in Figure 1.1. Data in its raw format acquired by a sensing device is most often a numerical value in isolation. The first step is to transform that data into contextual information by associating metadata with the value. For example, the value ‘20’ which is raw data from a thermometer could be transformed to ‘temperature=20; units=Celsius; time=000001’, which is contextual information. Depending on the device, the location may also be available, enabling more complete context to be provided. The contextual information is then stored, perhaps as part of a larger contextual knowledge base or ontological model [16]. The second step involves software that will reason about the contextual information to infer what activity is taking place. There are many different approaches to activity recognition, each of which are better suited to different types of data and activities. An activity will be inferred, typically with an associated level of confidence. For example ‘activity=sending email; certainty=0.8’. As described in [133], activities can be described as cascading in that one activity subsumes another. In this

4

case, for step 3, a composite activity ‘activity=using smartphone; certainty=0.9’ could be inferred based on the previously inferred activity and the remaining peripheral context. The contextual knowledge base, identified activities and composite activities then form the basis for algorithmic inference of the overall situation. It is important to note that context and the more abstract activities and situations that can be inferred from context should not be considered a state, but rather part of a process. Context-aware applications are required to perform appropriately during the entire process in which a user is involved, rather than just upon identification of individual activities or situations [39]. A key challenge in Context-Aware Computing is in acquiring enough data to form a complete contextual representation of the environment and the interactions of its constituent users and/or entities, as the context changes over time.

1.1.1

Rethinking Context

Defining and understanding context enables application developers to decide what context is of importance to their application(s) and network engineers to develop infrastructures that will enable access to data that will enable detection of context as it changes in realtime. Due to the widespread use of Social Web and Web 2.0 technologies, researchers have begun to explore alternative definitions of context that accommodate these new sources of user-centred information. The term Pervasive Social Context is used by Schuster et al [147] to capture context that comes from both pervasive computing and social networks and is defined as follows: “[The] Pervasive Social Context of an individual is the set of information that arises out of direct or indirect interaction with people carrying sensorequipped pervasive devices connected to the same Social Network Service”. Situated Social Context was then introduced for a more focussed subset of social context for pervasive social computing [51]: “[The] Situated Social Context of an individual is the set of people that share some common spatio-temporal relationship with the individual, which turn them into potential peers for information sharing or interaction in a specific situation”.

1.2

Ubiquitous Computing

Enabling context-aware applications requires the availability and use of heterogeneous sensing devices that make it possible for data to be captured in real-time. Ubiquitous Computing by nature requires that sensors and services be unobtrusive to a user, meaning that it is essential to provide sensors that can be embedded or hidden in the environment.

1.2.1

Wireless Sensor Networks

A WSN can be defined as a set of wireless connected devices that have the ability to sense phenomena in their immediate environment. A wireless sensor network in a smart space, 5

for example, may be composed of heterogeneous devices that sense phenomena such as light and actuate objects such as household equipment. Environmental monitoring WSNs are typically much larger in terms of the number of sensors. Systems such as those for environmental monitoring require wireless sensors that can be left in the environment unattended for long periods of time, meaning the provision of sensing devices capable of intelligence is required. A sensor that is intelligent can monitor its own state and that of connected sensors. It can also decide when it is appropriate to report a phenomenon, thus reducing communication cost which will have a substantial impact on the life of a battery run sensor. Utilising a WSN requires each node in the network to be programmed and deployed in the environment. Considering the scale WSNs will reach [180, 106, 101] this is a daunting and very difficult task making a framework to support the deployment and use of WSNs essential.

1.2.2

Middleware and Programming Methodologies

Middleware and programming frameworks for WSNs aim to ease the process of deploying large scale WSNs, enabling developers to build pervasive and ubiquitous services without the learning curve involved with WSN deployment and programming. While these terms are often used interchangeably within the research literature, here a distinction is provided. A middleware system or framework is one that abstracts the low-level issues inherent when managing a WSN, e.g., data acquisition and network management. A programming framework in Ubiquitous Computing is one that provides a new language or set of abstractions that makes controlling a sensor network and/or acquiring applicationrelevant context easier. A complete solution will support the development process from start to finish, i.e., from sensor programming through deployment and application development through deployment.

1.3

Evolution of the World Wide Web

Since the birth of the WWW in 1990 it has been transformed from a web of documents to a web of interconnected information, users and services. This has become known as Web 2.0. The emergence and wide-spread adoption of social networks in particular has embedded users into the web by providing a mechanism through which they can connect and share insights. By providing programmatic access to user data (public or authenticated), social networks facilitate the development of personalised services that are driven by Web based user data, further enhancing a users web-based experience. The current WWW (Web 2.0) can be viewed as a web-based environment that is monitor-able in a similar fashion to the real-world environments previously described. A user interacting with a web page such as that of a social network can be considered 6

engaged with this environment. Web Services already enhance a users experience based on knowledge extracted from their profile and their immediate web-based environment. The emergence of Cloud Computing technologies has eased the development and provision of diverse, real-time, Web Services including IoT platforms that provide a means for publishing WSN data to the WWW. While Web 2.0 and Cloud Computing bring us closer to bridging the CPS divide, these approaches impose serious vulnerabilities for users in terms of privacy, security and ownership of data. There is no doubt that the WWW will continue evolving and future iterations are expected to be focussed around distributed peer-to-peer services where the user is their own data broker [22, 150, 98]. Regardless of the environment and entities involved, providing real-time data-driven services is complicated and justifies support from a framework. As described in relation to Ubiquitous Computing, middleware and programming frameworks aim to ease this process. The same can be said for web-based environments where middleware and application programming interfaces (APIs) are utilised for the development and deployment of datadriven Web Services.

1.4

Motivation: Bridging the Cyber-Physical-Social Divide

Over two decades after Mark Weiser’s seminal paper ([176]) outlining his vision for Ubiquitous Computing, it is clear that modern technological advances require us to re-evaluate the goals of Ubiquitous Computing, review what has been achieved to date and assess the next steps in achieving our newly-defined goals. Motivated by human ability to exploit their environments, Rogers [138] describes an alternative goal to Weiser’s ‘calm computing’ (comfortable and effortless living enabled by ubiquitous technologies that act on people’s behalves) that is ‘engaging computing’ (ubiquitous technologies engage people more actively in what they do). It is argued that we ‘simply don’t do smart well yet’, but perhaps that is because we do not have access to comprehensive historical and real-time context for each user that an adaptive system is required to satisfy. Given the way users adapt to new technologies, it is proposed here that the provision of engaging technologies is a necessary step in the journey to ultimately providing calm computing. As users become more engaged with (and accepting of) technology over time they will learn to understand and trust in technological advances. Simultaneously, researchers will learn what users are willing to accept from technology, both in terms of what tasks or decisions they are willing to delegate to automated software and what level of control users demand over their own data. This intermediate step between engaging and calm computing will facilitate the organic growth of large, user-annotated sets of context data (aggregated through user interactions with technology), providing a new historical

7

knowledge base upon which local, personalised, intelligent software can become automated to suit the individual(s) that are relevant to the system. Difficulties in assessing a user’s mood or motivations will be addressed through the inclusion of a user’s real-time social presence within context-aware systems. As such, systems to support the acquisition of CPS context are required. Ubiquitous Computing and web-based research share a common goal: to enable the provision of context-aware applications that enhance a user’s day to day experience. This comprises two main requirements, the acquisition and processing of real-time context data and the satisfaction of users through the provision of intelligent services. While the difficulties involved with satisfying these requirements differ, the common goals of web-based research and Ubiquitous Computing in general allow for a mapping between these challenges. For example, a WSN middleware framework will put emphasis on a feature such as sensor discovery, a Web Service middleware puts emphasis on service discovery. Such a mapping will enable a middleware and programming framework to provide techniques that accommodate CPS context gathering, as well as the rapid development of applications that specify complex and dynamic context requirements. While services exist that facilitate the sharing of WSN (or IoT sensors) data to the web, the publication of private sensor data to an external data broker leads to concerns surrounding user privacy and ownership of data. It is necessary to provide a ‘safe place’ where context processing and intelligent decision-making can take place while adhering to the user defined privacy requirements of the most private data involved. This results in the requirement for web-based data to be integrated with WSN data within a local middleware that supports fully customisable privacy and security services. Such an approach will facilitate fusion of widely diverse context data acquired from physical sensors and the web, as well as publicly accessible information. This thesis acknowledges the challenges in acquiring context from such complex environments. Research and practical developments have been made towards realising a CPS Sensor Web middleware and programming framework that enables the rapid development of context acquisition applications, in a manner that is suited to the development of both the engaging and calm computing paradigms. With a focus on middleware and programming support for context awareness, the objectives and core contributions of this thesis are outlined in the following sections.

1.5

Objectives

Motivated by the desire to enable the provision of CPS context to applications, in a manner that facilitates rapid development and adaptivity, the following goals have been identified for this thesis:

8

1. To explore existing research towards CPS Computing and identify research areas that, when unified, will enable CPS Context-Aware Computing 2. To further explore the identified areas constituting CPS Computing to understand both the benefits and challenges that will emerge from the convergence of these “research fields” and how these areas will converge to realise CPS Computing 3. To review existing approaches towards CPS enablement, focussing on CPS context acquisition and processing; identify an appropriate approach based on the previously identified challenges; and identify open research challenges towards CPS enablement 4. To review the supporting technologies from the distinct research areas that contribute towards CPS Context-Aware Computing, in support of the acquisition and use of CPS context data by developers 5. To design and implement a supporting framework for CPS Context-Aware Computing based on identified best practice from the range of fields explored earlier; while adapting them specifically for CPS context-awareness 6. To demonstrate the effectiveness of the approach, in terms of both usability and its flexibility to enabling context-awareness in a variety of use cases

1.6

Core Contributions

Based on the objectives of this thesis, the following contributions have been made and will be described in full in subsequent chapters: 1. Design of a unifying CPS infrastructure comprised of middleware integrated technologies from the broad fields of WWW and Ubiquitous Computing research 2. WWW-based context source and processing taxonomies identifying the basic WWW building blocks to be accommodated in a CPS sensor middleware 3. Design and implementation of middleware support for the dynamic creation and configuration of Cyber Sensors, which support the acquisition of real-time WWW context, and are treated as equal citizens to their physical sensing counterparts 4. Design and implementation of middleware support for the dynamic creation and configuration of Pipes, which when applied to a sensor realises the Cyber Sensor Pipeline paradigm. In support of dynamically changing application requirements, Pipes accommodate: • Real-time sensor stream processing and refinement 9

• On-demand aggregation of context from static WWW sources and Web Services • On-demand creation and configuration of additional Cyber Sensor Pipelines 5. Development of high-level programming abstractions for CPS context-awareness that are adapted from techniques for the WSN domain and applied to the task of WWW context acquisition 6. Demonstrate the effectiveness of the approach through: • Development of a series of case studies demonstrating functionality as well as the flexibility and extensibility of the Cyber Sensing paradigm • User trials to confirm the usability of Cyber Sensing programming abstractions from the perspective of developers

1.7

Thesis Outline

Each chapter in this thesis serves to meet the objectives stated in Section 1.5. The first two objectives are met in Chapter 2, which explores the impending emergence of CPS Computing from the converging agendas of the broad fields of Ubiquitous Computing and the WWW. CPS Computing is defined from a context-awareness perspective and the challenges and benefits of the paradigm are analysed. The initial outcome of this research is the design of a unifying CPS infrastructure and the specification of requirements for enabling CPS context-awareness. Chapter 3 meets the third objective by analysing enabling technologies for CPS Computing. First, existing approaches towards bridging the CPS divide are identified and their benefits are evaluated with respect to the earlier defined challenges. The shortcomings of existing offerings are identified, informing a deeper analysis of related work with an aim to meeting the requirements of CPS context-awareness in this thesis. This review of related works informs the design of a supporting framework for CPS context acquisition as described in Chapter 4. The basic building blocks, Cyber Sensors and Pipes, are designed to support the seamless integration of WWW data sources and services with Ubiquitous Computing sensors. Higher-level building blocks in the form of programming abstractions enable the rapid development of context acquisition applications. As detailed in Chapter 5, these middleware and programming support components have been implemented as extensions to the SIXTH middleware: a framework for which the design has been influenced by the contributions of this thesis from its conception. The implemented functionality and its flexibility and suitability to acquiring WWW context using Ubiquitous Computing techniques is evaluated first through a series of case studies, which are described in Chapter 6. A second evaluation, focussing on the usability

10

of the approach is detailed in Chapter 7. Finally, Chapter 8 concludes this thesis and proposes directions for future work.

11

Chapter 2

Cyber-Physical-Social Computing Emerging A number of distinct but interconnected advances in technology work towards a partial enablement of Mark Weiser’s Ubiquitous Computing vision. In terms of hardware, mobile devices have become the norm and wearable devices are now more widely accepted. Relatively small devices are now ‘smart’, web-enabled and powered by an operating system that facilitates the deployment and operation of diverse applications on the device. Smart devices most often come equipped with sensors that monitor the device’s location, orientation and facets of its immediate environment. Objects that stream information into the web or can be controlled via the web are often referred to as “Smart Things”, as opposed to the more computationally powerful smart devices. While smart devices are typically held (tablet, phone) or worn (watch, glasses), smaller devices that are optimised for longterm sensing, reporting and intelligent decision-making aim to be embedded or otherwise distributed in an environment. Large-scale sensor networks will be formed through the deployment of vast numbers of such sensors and require middleware support to simplify sensor (re)programming and data-driven application development and deployment. Wearable sensors for monitoring a persons biological state have emerged, unlocking a further realm of data to empower context-aware applications. Wireless Sensor Network (WSN), Mobile Computing, Internet of Things (IoT) and Sensor Web research efforts have enabled these achievements and continue to make strides towards a future of connected devices that underpin personal and global Ambient Intelligent (AmI) and context-aware applications. While Ubiquitous Computing adopts sensor technologies to acquire data for contextawareness, the World Wide web (WWW) produces vast amounts of context through user and service interactions. Just as the WWW is a web of connected, addressable documents, the Web of Data is a web of connected, addressable data. This Web of Data is also known as the Semantic Web, for which Linked Data initiatives aim to represent all entities (people, things, places, 12

etc.) and their modalities (name, colour, GPS location) by unique URIs that are linked by the relationships between each entity. The goal is to maintain a vast open web of data, in standardised formats, upon which reasoning algorithms will infer further knowledge enabling intelligent decision-making. Social Networks and the Social Web aim to promote and facilitate personal profiles and user-contributed information and activities. These user data gathering entities provide Application Programming Interfaces (APIs) that enable the development of personalised services that both enhance a users online experience (e.g. social games) and target them for the benefit of others (e.g. targeted advertising). The IoT and Web of Things (WoT) comprises web-enabled devices that stream realtime sensor readings, typically to a Web Service via an API. Web Services and webenabled devices can now be plugged in to user friendly web programming interfaces such as IFTTT1 or Yahoo! Pipes2 , allowing users to create simple but powerful programs that perform actions based on information from connected entities. By pushing sensed data from physical environments to the web, the Cyber-Physical-Social (CPS) divide is bridged in a manner that makes creating basic CPS application development accessible to users without programming skills. However, in order to develop applications that can autonomously enhance a user’s day to day life, an application requires more in-depth user information such as that acquired by a multitude of heterogeneous sensors deployed in a user’s environments. User-centred services and interfaces such as those proposed in AmI research and global services such as those envisioned by Sensor Web researchers are dependent on the enabling technologies and underlying infrastructures proposed by IoT, Sensor Web and indeed WWW research. The WWW and Ubiquitous Computing are diverse yet converging areas of research, both of which strive to produce personalised and intelligent systems that will be adopted by and meet the needs of users. The challenge addressed in this thesis is in the integration of WWW and Ubiquitous Computing infrastructures in support of the emerging CPS Computing paradigm. This chapter explores the emerging CPS Computing paradigm. In Section 2.1 existing definitions of Cyber-Physical (CP) and CPS Computing are reviewed and the CPS Computing paradigm is explored from the perspective of context-awareness. CPS Computing is found to comprise the distinct, yet converging, research topics within the broad fields of WWW and Ubiquitous Computing, as described in Section 2.2 and Section 2.3 respectively. This chapter concludes in Section 2.4 with a discussion reflecting on the previous sections. The challenges and benefits of CPS Computing are identified, informing the design of CPS Computing infrastructure that integrates each of the enabling technologies introduced throughout this chapter. The role of a supporting framework for CPS Com1 2

https://ifttt.com/ https://Pipes.yahoo.com/Pipes/

13

puting is highlighted within this unifying infrastructure. Finally, the requirements of the supporting framework are defined based on the key challenges identified.

2.1

What is Cyber-Physical-Social Computing?

Figure 2.1: Prototype Architecture of Cyber-Physical Systems [163] CPS Computing is a new paradigm emerging as an evolution of CP Computing. CP Computing is considered to have emerged as the successor to the embedded systems and WSNs that traditionally underpin Ubiquitous Computing research [45]. In Subsection 2.1.1, existing definitions of CP and CPS Computing are reviewed and the definition of most relevance to this thesis is chosen. In Subsection 2.1.2 this definition of CPS Computing is expanded upon and CPS Computing from the perspective of context-awareness is explored.

14

Figure 2.2: Physical and cyber world interactions [37]

2.1.1

Existing Definitions

Tan et al. [163] define a CP system as a “next-generation network-connected collection of loosely-coupled distributed cyber systems and physical systems monitored/controlled by user defined semantic laws”. The authors define cyber systems as collections of control logic and sensor units, while physical systems are defined as collections of actuator units. This is illustrated in Figure 2.1, which depicts the authors’ proposed architecture for CP systems. Sensed information is processed in the cyber world using human-defined laws, resulting in changes to the physical world affected by actuators. While the authors describe the cyber world as an information world that humans interact well with, their research is focussed on technical integration of the cyber and physical worlds rather than the data from these worlds to be used by applications. Similar definitions are proposed by Poovendran [132], Rajkumar et al. [135], Wu et al. [178], and Shi et al. [154]. Wu et al. [178] emphasise the human element by introducing CP-based social networking as a paradigm in which user interactions are enriched through the sharing of sensed data for socialisation and gaming purposes. Conti et al. [37] further emphasise the human element of CP Computing. The authors introduce the concept of Cyber-Physical World (CPW) convergence as a paradigm shift in which “real-world components interact with cyberspace via sensing, computing and communication elements [causing information to flow] from the physical to the cyber-world and vice-versa, adapting 15

the converged world to human behaviour and social dynamics”. In Figure 2.2 the authors illustrate interactions between the physical and cyber worlds, which they envisage will be mediated through social networks. The work of Conti et al. is consistent with the definition of CPS systems offered by Wang [173] and Liu et al. [105]. Wang, having described CP systems as “the tight conjoining of and coordination between computational (or cyber) and physical resources”, went on to define CPS systems as being CP systems “tightly conjoined, coordinated and integrated with human and social characteristics”. Wang notes that since the Internet enabled the presence of cyber space, human living space includes the artificial world as well as the mental and physical worlds. As such, context-aware systems require the integration of data from both of these worlds.

Figure 2.3: Physical -Cyber-Social Computing [153] While the previous research has focussed on technical integration and coordination of the cyber, physical and social worlds, Sheth et al. [153, 152] explore CPS Computing from the perspective of context data. The authors define CPS Computing as a paradigm that “encompasses a holistic treatment of data, information, and knowledge from the [PhysicalCyber-Social] worlds to integrate, correlate, interpret, and provide contextually relevant abstractions to humans”. The authors’ view of CPS Computing as a next generation paradigm that builds on ‘cyber-physical systems’, ‘socio-technical systems’, and ‘cybersocial’ systems, acquiring, from each of these infrastructures, data that is processed to form contextual information. In Figure 2.3 the authors illustrate examples of each of

16

these types of systems, noting that a tighter integration between the three system types will enable CPS systems to be realised. While the previously described research focuses on the combination of technologies that will realise CPS Computing, the definition proposed by Sheth et al. [153] emphasises the use of CPS context. Given the objective of enabling CPS context-awareness, the definition proposed by Sheth et al. is of most relevance to this thesis. In the following section, this definition is adapted for the purposes of this thesis. Further, the authors’ research is extended to define the infrastructures comprising CPS Computing, with an emphasis on conceptual and technological sources of context.

2.1.2

CPS Computing for Context-Awareness

Figure 2.4: Cyber-Physical-Social Computing For the purpose of this thesis, CPS Computing is defined as encompassing: a holistic treatment of data, information, and knowledge from the cyber, physical and social worlds, to provide contextually relevant abstractions to humans and software agents This definition is adapted from that of Sheth et al. [153] to accommodate autonomous decision-making by software agents driving context-aware applications. The overlapping cyber, physical and social worlds are illustrated in Figure 2.4. The Venn diagram marked ‘A’ explores these worlds as context sources in a technology-independent manner. The diagram marked ‘B’ provides a Computer Science research perspective of ‘A’ and is comprised of technologies and research agendas, from the broad fields of the WWW (see Section 2.2) and Ubiquitous Computing (see Section 2.3), which will enable the provision of CPS context. Figure 2.4 can be considered as a reference point for the discussions throughout the remainder of this chapter. The following paragraphs describe each of these overlapping worlds by making a mapping between ‘A’ and ‘B’. 17

Conceptually, the physical world comprises geographical places and physical (nonhuman) objects. The context of these places and objects is acquired by a wide variety of sensor technologies. Sensors embedded in the environment acquire data pertaining to real world phenomena such as light or noise levels. Sensors attached to objects report data relating to both that object and its immediate, potentially changing, location. By combining large numbers of heterogeneous sensors to form sensor networks, researchers strive to form a complete contextual representation of the physical world (or part thereof) upon which applications may respond. The social world represents the human component of CPS Computing rather than solely social interactions as in the research described in Subsection 2.1.1. This separation of concerns allows emphasis to be placed on the personal aspects of mind, body and human senses. As in the physical world, sensor network technologies are deployed to acquire context in the personal world. Sensors that are embedded in clothes and fixed to the human body, either externally or as an implant, form body area networks. These networks report phenomena directly related to the person and, depending on the sensor types, their immediate surroundings also. The intersection of the physical and social worlds relates to human interaction in physical places and with physical things. These interactions may be either personal (one person in isolation) or social (more than one person engaged in the same interaction). The interactions are sometimes supported through technological interfaces, which both produce (through user interaction) and consume (for the purpose of intelligent decision-making) context. Human-owned smart devices typically travel with their user, providing additional means to acquiring the user location context. These devices are also a technological interface for user interactions, again, producing and consuming context. The technologies relating to the acquisition of context from the physical and social worlds largely lie in the category of Ubiquitous Computing research. This research field strives to enable AmI, enabled by data acquired by unobtrusive sensors, delivered in the form of autonomous decision-making by agents and intuitive interfaces for users. The cyber world represents the WWW which comprises websites, services and applications as well as the emerging Semantic Web, which aims to represent information in a machine-consumable manner. It is considered an information world comprised of knowledge bases related to diverse topics as well as the computational capabilities to produce higher-levels of information. At the intersection of the cyber and physical worlds, the WWW enables digital representation and actuation of physical places and objects through emerging paradigms such as the IoT and the Sensor Web. Based on the discussion in Subsection 2.1.1, these paradigms can also be considered CP Computing paradigms. At the intersection of the cyber and social worlds, the WWW enables social interaction with information and

18

services through the Social Web, which is comprised of platforms such as social networking sites and socially-enabled applications and websites. While the Social Web is categorised as WWW research, the Cyber-Physical intersection is largely driven by sensors based research and is categorised as Ubiquitous Computing research. The integration of WWW, the cyber world, and Ubiquitous Computing will unlock an additional realm of information for use by context-aware and AmI applications. The cited research in this section focuses largely on the connectivity of physical devices to cyberspace and the challenges introduced by the integration of complex sensor networks. What is not addressed is how, from an application developers perspective, data from these diverse infrastructures can be acquired and processed in a unified manner. Further, it also becomes necessary to identify additional challenges that arise from the integration of the cyber world specifically. The remainder of this chapter explores the above identified research agendas, under the broad headings of WWW (Section 2.2) and Ubiquitous Computing (Section 2.3), with an aim to revealing more in-depth insights into the converging research agendas constituting the emerging CPS Computing paradigm.

2.2

The World Wide web

Since the birth of the WWW in 1990 it has been transformed from a web of documents to a web of interconnected information, users and services (Web 2.0). Initial Web 1.0 pages could be described as static, quickly evolving into a more dynamic Web 1.0, where web pages are updated automatically by databases that serve information to be displayed in a page. The basic premise of Web 1.0 is based on client-server computing. Documents are named using URLs, created using HTML (within which URLs connect documents to each-other), and stored by servers. Clients machines then access the documents by communicating with servers via HTTP [83]. These Web 1.0 pages, interconnected through hyperlinks, are made discoverable by users through search engines such as Yahoo!

3

and

Google 4 . Web page content is widely varied across global topics of interest and the perspective or incentive of the contributor of the content. For example the goal of a web page could be to support human decision-making by providing biased or unbiased information; or to provide information for knowledge enhancement or life enrichment purposes [21]. User experience on the WWW changed radically with the emergence of Web 2.0. Instead of referring to a document (or web page), now the URL can also refer to an application residing on a server upon which it can be executed. Rather than accessing URLs as a means to read web pages, users can now access media rich, interactive, online applications. Web 2.0, also known as ‘the network as a platform’ [126], ‘wisdom web’, 3 4

http://search.yahoo.com http://google.com

19

‘people-centric web’, ‘participative web’, ‘read/write web’ [122], and ‘Social Web’ [35], has embedded users into the web by providing mechanisms through which they can connect, interact and share. Many Web 2.0 systems now provide programmatic access to data and data-processing services through standardised interfaces, facilitating the Web of Services (WoS), enabling enhanced and personalised applications, which further enhance a user’s web-based experience. The emergence of social networks [50] in particular, has unlocked a wealth of user-contributed data, enabling Big Data researchers to analyse user behaviour within the online environment [108, 110, 28]. The following section describes in more detail the Social Web, which was identified as Cyber-Social Computing in Subsection 2.1.2. The subsequent sections describe the WoS, Semantic Web and research proposals relating to the future of the web.

2.2.1

Social Web

A social network allows users to “(1) construct a public or semi-public profile within a bounded system, (2) articulate a list of other users with whom they share a connection, and (3) view and traverse their list of connections and those made by others within a system” [50]. The terms “social media” and “social networks” are often used interchangeably. However in [90] a distinction is offered stating that social networks allow people to stay connected within online communities (e.g. Facebook) while social media sites allow users to share user created contributions (UCCs) (e.g. YouTube 5 ). It should be noted that social networks and social media sites are now converging in that each type of site typically grows to include more features such as user connectivity (Google+ to YouTube accounts) and UCC sharing (Facebook

7

6

profiles connected

acquisition of Instagram 8 ). Given

this convergence, in this thesis the term “social network” is used to refer to both social media and social networking sites. Individual social networks differ in the ways in which user connections and interactions are supported and also in terms of the visibility of user profiles and interactions with the public web. CPS Computing will involve the use of this information by CPS context-aware applications, so it is useful to explore the types of data on the Social Web further. Schneier et al. [146] define a taxonomy of social networking data . This taxonomy provides insight into how user data is contributed to social platforms, and how much or little control over the content that user has once it has been contributed. The taxonomy is summarised as follows: Service data the data a user must give to a social networking site in order to obtain 5

https://www.youtube.com/ https://plus.google.com/ 7 https://www.facebook.com 8 https://instagram.com/ 6

20

access to it. For example, the user’s legal name, age, and sometimes financial information such as credit card. Disclosed data the user created content that is contributed to a user’s own page. This type of content varies in form, from textual comments or blog entries to media-rich photos, audio, or video content. Entrusted data the user created content that is contributed to another user’s page. This is essentially the same as disclosed data except the user whose page the content is contributed to has control over the content once it is contributed, not the original creator of the content. Incidental data this is content contributed by users who are explicitly connected to another user (e.g., through tagging) or information about another user. Like previous types, the content varies from text to media such as a photo or video of a user that someone else takes and contributes. Again, this is essentially the same as disclosed data except the user that the content is related to does not have control over it, and they did not create it in the first place. Behavioural data is data the site collects about users’ habits by recording what they do and who they do it with. It might include games they play, topics they write about, news articles they access (and what that says about your political leanings) etc. Derived data is data about users which is derived from all the other data. The example provided is “if 20 percent of your friends self identify as gay, you’re likely gay yourself” ([146]). This taxonomy reveals the complexity of monitoring the complete context of an individual user, or indeed a community. Techniques are required for the acquisition of user context that is contributed by a user directly, indirectly and from the perspective of another user. Further, users can post information about topics and locations of interest and may share their physical location with respect to the information they are contributing. CPS Computing requires a means to providing context-aware applications with information relating to these users, topics and locations if and when they become of relevance to the application in real-time. Social networks are increasingly platform independent (supporting mobile devices as well as multiple browsers and platforms) and their integration with existing websites and applications (e.g., social network functionality for commenting on and/or logging in to third-party websites) has made social networks ubiquitous in relation to a user’s WWW experience. This gives rise to the term “Social Web”, where the social aspects of Web 2.0 21

are not confined to the social applications in which a user creates their online persona(s). The Social Web supports an increasingly large range of diverse functions that users perform on the WWW [23, 35, 90]. Examples of these functions include: User interaction involves users establishing and maintaining connections (with other users, brands, communities) through messaging, commenting and promoting other users or brands. User interaction in Web 2.0 is enabled through integration of social functionality in otherwise non-Social Websites, i.e. logging in to third-party websites using a Social Network profile often allows users to interact with each other within the bounds of that third-party site. User participation in the Social Web involves users commenting on socially enabled websites (e.g., news article comments and product reviews) and sharing and tagging user created content (UCC). Tagging of content enables automated formation of folksonomies: information structures that can be used to ease user navigation through diverse content. Collaborative content creation in the Social Web is enabled through socially enhanced online content creation tools such as wikis that allow multiple users to simultaneously work on a shared document, often with added features that facilitate collaborator communication such as commenting and chat services. Social search tools in the Social Web, through collective averaging or voting methods, allows users to find relevant information more easily. Search is also facilitated through user tags provided through user participation with content in social networks and socially enabled third-party sites. User interaction and participation in the Social Web results in the creation of data within contextually relevant ‘cyber environments’, further enhancing the breadth of context accessible from the Social Web. Collaborative content creation results in evolving knowledge bases that have the potential to provide CPS Computing with on-demand, upto-date expertise. Social search tools have emerged as a consequence of the ‘information overload’ on the WWW and are an example of context-awareness in the cyber world. These new functions of the Social Web enable users to unlock data that not only enhance the platforms through which they are performed (e.g. product reviews enhance Amazon’s

9

recommender system) but they also enhance and support new functions per-

formed by businesses and researchers on the WWW: Citizen Sensor Networks researchers have demonstrated the plausibility of citizen sensors (also referred to as social sensors): software sensors that monitor user contribu9

http://www.amazon.co.uk/

22

tions to social networks with an aim to detecting and monitoring regional and global events in real-time [143, 38, 151]. Digital Journalism involves professionals and citizens reporting on incidents in realtime, utilising one or more social network as a medium for sharing this information with online communities. Breaking news is most often found on a social platform such as Twitter before traditional News mediums such as TV stations or news papers. Journalists now listen to social channels to keep informed of news being broken by other journalists and citizens alike [88, 149, 68, 91, 172] Digital Marketing companies create brand personas through websites and social brand profiles. They use these to connect to their target market communities to promote their products and services. Listening to social media channels and other interactions on the Social Web now improves digital marketing in that companies gain wider perspective on their brand’s reputation and also the demands and interests of their target markets [169, 157, 168, 52] eGovernment monitoring and information dissemination has been supported by the Social Web, Twitter in particular, where governments have been seen to listen to social channels to gain insight into real-time events such as Tasmania fires, and also to disseminate support requests and information to citizens [182, 77, 127, 41] Despite the rapid growth of the Social Web, it faces a number of challenges in relation to user acceptance of social technologies and management of user data [90, 28]. Modern social networking platforms host, manage and control user data according to privacy settings configured by users. They act as a data broker allowing third-party service providers access to user data, typically in line with these privacy mechanisms. User misunderstanding of privacy and visibility in social networks can result in personal damage to the user due to indiscreet postings, membership in unconventional or irregular groups, inaccurate profile descriptions and connectivity to suspect users. Further, user provision of comprehensive personal details may be visible to and misused by suspect users and services, threatening the safety of that user. User behaviour, sometimes through false profiles, can be damaging to others due to cyber stalking or bullying, dissemination of false or misleading information and the spread of sensitive information. Services using such complex data must take these challenges into account in order to provide trusted and accurate functionality. Realising CPS Computing will require these challenges to be addressed since the paradigm will see the integration of context derived from sensitive data sources such as sensors in a user’s home. Motivated to overcome such privacy and security problems from a user perspective, distributed mobile and peer-to-peer (P2P) social networks offer social network users a 23

closer to real life online solution to socialising [78, 32]. Such distributed social networks remove the role of a third-party data broker and aim to alleviate user concern regarding unauthorised use of their private information. Regardless of the social platform or architecture of a system analysing user-contributed information, automated processing of such data is problematic in many ways. Human-created content is often unstructured text, informal language and requires validation. Despite the abundance of data made available by the Social Web, analysing online users alone can result in misleading conclusions. People interact offline, online, publicly, privately, and not all relevant users provide accessible content online. Further, due to the erroneous and subjective nature of user contributed data, quantitative analysis alone is not sufficient and costly models such as ontologies are required.

2.2.2

Web of Services

While the WWW maintains collections of static and evolving information and the Social Web unlocks an abundance of user-contributed data and insights, the WoS exposes diverse APIs enabling the development of new applications that are novel in their composition. A Web Service is “a piece of functionality accessible over the Internet with a standard interface” and can be viewed as an object that provides interfaces to access its methods remotely [83]. A Web Service mashup is “generated by combining content, presentation, or application functionality from disparate web sources” [181]. For example, HousingMaps 10

is composed of data from Craigslist

11

and Google Maps

12 .

Web Service mashup patterns include data source mashups and process mashups [99]. Data source mashups combine similar types of information from multiple sources into a single representation and are key to creating data-driven applications such as those described in Subsection 2.2.1. Such services are often enhanced with additional dataprocessing services providing functions such as filtering, personalisation, and targeted realtime monitoring [177]. Process mashups, often referred to as Service Oriented Architecture (SOA) orchestration, enable the composition of applications through the connection of an existing set of services. Increasing numbers of Web Services are becoming available for public consumption through API directories such as ‘Programmableweb’13 . The use of these APIs often results in the creation of new or higher-levels of context in an on-demand basis, usually limited by the number of API calls permitted during a specified time interval. Despite the availability and use of standards in exposing Web Services (e.g., XML standardised the structural definition of data [26] ), developing mashups is time-consuming and problematic. Yu et al. [181] describe the challenges in developing web mashups 10

www.housingmaps.com https://www.craigslist.org 12 http://www.maps.google.com 13 http://www.programmableweb.com/ 11

24

without the use of a development tool, stating that manual mashup development “requires programming skills and intimate knowledge about the schemes and semantics of data sources or the business protocol conventions for message exchange”. This challenge will be amplified in CPS Computing, which requires techniques for gathering context that support a wide range of use cases using diverse semantics. Problems can occur when combining diverse data sources on the web such as different file formats, schemas, naming systems and the technical challenges of accessing data if not exposed via an API or feed [148]. Heterogeneous services with inherently disparate APIs, data types and structures, programming models and patterns causes difficulties for programmers and non-programmers alike [84]. This prevents standard (non-developer) web users from creating mashups to meet their own needs, giving rise to the need for mashup tools and frameworks such as Yahoo! Pipes14 and Google Mashup Editor15 . Such tools speed up the mashup development process and also make mashup development accessible to users with little or no development experience. Mashup tools typically comprise components (objects, services or data feeds to be integrated), composition logic (methods for combining components), a development environment (enabling and simplifying mashup creation by users), and a runtime environment (enabling deployment of a completed mashup) [181]. Researchers have found that while current web mashup tools are reasonable environments for rapid prototyping, they are not mature enough for large-scale applications and that their primary use currently is for private users creating mashups that meet personal needs as opposed to development of the next ‘killer app’ which will require an approach that is both scalable and sustainable. Open challenges identified in relation to web mashup development, and therefore the WoS, as identified in [181, 148, 56, 11, 84] are summarised as follows: • Improved tool support, and end-user-oriented integration paradigms, are required to simplify mashup development by non-programmer users, in particular with inclusion of user interface components • Semantic heterogeneity of data and services needs to be addressed to support more intuitive and user-friendly registration, publication, search and discovery of mashup components • Solutions to support non-programmer management of failures or unexpected behaviours of disparate website and services that comprise mashups • Provision of non-programmer tools for testing and debugging mashups in support of security and correctness of applications 14 15

https://Pipes.yahoo.com/Pipes/ https://developers.google.com/mashup-editor/ (discontinued)

25

• Models for and management of uncertainty in mashup data • Support for non-URL-addressable components within a mashup, e.g., enterprise data (documents, emails, databases) and web scraping tools • Platform independent mashup tools and resulting mashups supporting interaction across devices • Personalisation and user-context aware features within a mashup development framework • Automated mashup generation based on user’s tasks, profile, environment and context While the ultimate goal of mashup development tools is to support ‘non-programmers’, there are a number of core challenges due to semantic and technical heterogeneity that are of primary concern. Like the privacy challenges outlined in Subsection 2.2.1, these challenges are important to resolve if the even more heterogeneous CPS Computing data sources and technologies are to be integrated. Researchers solving the above problems, in particular in relation to heterogeneity, discoverability, and integration of data and services, often turn to Semantic Web technologies, which are discussed in the following section.

2.2.3

The Semantic Web

Figure 2.5: Semantic Web 3.0 [70] In 2009, Hendler proposed that Web 3.0 will be in part the realisation of the semantic web [70] as outlined by Tim Berners-Lee in 2001 [25]. It is stated that Web 3.0 can be viewed as “Semantic Web technologies integrated into, or powering, large-scale web applications”. This is illustrated in Figure 2.5, which identifies the underlying semantic technologies that link Web 2.0 data allowing the formation of a Semantic Web upon which Web 3.0 applications are expected to strive. Web 3.0 applications are said to differ from those we know to be Web 2.0 in that they will become ‘smarter’ due to the ease of 26

performing simple inferences with the support of semantic knowledge bases. The three fundamental parts to the Semantic Web are: 1. Ontological data models (OWL16 , RDFS17 ) and formats (RDF triples18 ) that provide a queryable (SPARQL19 ) semantic foundation to applications (Semantic Web and Linked Data technologies) 2. Services (Web 2.0) 3. Intelligent agents or applications (Web 3.0) Web 2.0 researchers, Social Web and Web of Services in particular, are beginning to adopt Semantic Web technologies to solve some of the problems described in the previous section. For example, while the Social Web offers a platform of “collected intelligence”, the Semantic Social Web is expected to offer “collective intelligence”, which will be generated as an output from semantic reasoning performed upon the Social Web [60, 116]. Such semantic reasoning will assist in addressing the problems in automated analysis of user contributed content and will enhance automated user applications. Semantic annotation of Web Services and data will simplify service discovery and integration in the WoS and the mashup tools that non-programmers use to create custom applications [33, 67, 117]. While Semantic WWW applications are emerging, widespread adoption of Semantic Web technologies is hindered by costly ontology development [115] and the intimidating nature of Semantic Web technologies [57]. Proposed approaches include simplification of Semantic web formats [114], leveraging of the folksonomies, or ‘tag clouds’ that grow organically within Web 2.0 [155, 96, 60], and the design of user applications that create incentive for manual annotation of content such as a Social Semantic Desktop [42, 59, 24, 116].

2.2.4

The Future web

Beyond the Semantic Web, a number of research initiatives leverage related research fields with an aim to shaping a future user-centred web. The Social Semantic Desktop paradigm [42, 59, 24, 116] draws on Semantic Web, Peer-to-Peer Networks and Social Web research, aiming to convert the traditional desktop into “a seamless, networked working environment, by loosening the borders between individual applications and the physical workspace of different users” [24]. A vision of a Smart Internet [123, 56, 160] was introduced in 2009 and is described as an “evolving extension of the internet in which online services and 16

http://www.w3.org/OWL/ http://www.w3.org/RDFS/ 18 http://www.w3.org/TR/n-triples/ 19 http://www.w3.org/SPARQL 17

27

resources are discovered, aggregated and delivered dynamically, automatically and interactively in response to user or group’s evolving concerns and situations, which may involve real-time or proactive performance of tasks that address that user’s goals” [123]. In 2010, Hendler and Berners-Lee [71] argued that the social connected nature of current web systems is an early realisation of web-connected social machines that will achieve much more through collaborative functionalities. It is argued that the future web must “be designed to allow the virtually unlimited interaction of the web of people” [71]. It is envisioned that people will build and share their own social machines. While research efforts towards distributed social networks, the social semantic desktop and the smart internet bring us closer to the vision of user-centred social machines, Web 2.0 persists in the mainstream and Web 3.0 applications are beginning to emerge in the research literature [70]. Initial insights into Web 4.0 and beyond are now offered in research literature and indicate the anticipation within the field that the WWW will evolve into an infrastructure that will enable CPS Computing. Aghaei [7] and Kambil [86] speculate, independently, on what is to come after Web 3.0, agreeing that Web 4.0 will be a realisation of the WWW where intelligent interactions between humans and machines take place, bridging the CPS divide. Kambil offers further insights into Web 5.0, proposing a sensoryemotive web, a WWW that is responsive to users’ emotional state or receptivity to specific information. Concurrently, Internet Architecture researchers are investigating problems with the current architecture of the internet and offering proposals for a future internet architecture that will be more accommodating of how the internet is now used and user expectations of web applications [128, 120, 80, 81]. IoT is considered an integrated part of the future Internet where physical and virtual objects are seamlessly integrated into the information network [171]. Among the requirements of a future internet architecture described by Jain [81] are explicit support for mobile technologies, client-server traffic, distributed services, person-to-person (device independent) communication, security and privacy of location, data and data integrity guarantees, support for real-time services and streaming data. Rather than striving for future iterations of the web, these research efforts are in fact evolving simultaneously, and in many cases their goals and adopted approaches are converging. It is therefore anticipated that the future CPS Web will be realised as a composition of these research efforts and those of researchers involved in Ubiquitous Computing. The following section provides a review of Ubiquitous Computing research including the IoT and Sensor Web paradigms that were categorised as CP Computing in Subsection 2.1.2. The discussion that follows then brings these converging research areas together to form a unified overview of the infrastructure for CPS Computing and to identify the requirements of a supporting framework that will enable the realisation of a CPS Computing paradigm.

28

2.3

Ubiquitous Computing

Ubiquitous Computing involves building highly responsive and self-adaptive systems that aim to increase the integration of technology into the fabric of everyday living [176, 175], for example, smart spaces and environmental monitoring. The implicit goal of Ubiquitous Computing is to assist with and enhance daily life without overwhelming it [5] through the provision of AmI techniques [141]. Enabling this vision requires the availability and use of heterogeneous sensing devices that facilitate data capture in real-time. Ubiquitous Computing [5] is a user-centred paradigm that strives for natural, intuitive user interfaces that leverage implicit user interaction. Such interfaces deliver context-aware applications whose behaviour adapts based on sensed information in real-time.

2.3.1

Ubiquitous Technologies and Applications

Ubiquitous technologies are devices that provide context to drive the behaviour of contextaware applications and/or devices hosting context-aware applications that a user interacts with. The former can be generally described as sensing devices. A sensing device converts a physical phenomenon into an electrical signal [136], e.g. a thermometer is a temperature sensor. A sensor system then, is “an aggregation of sensors, attached to a single platform” such as a weather station [31]. A sensor network is composed of a large number of sensing devices that sense, communicate and process data [9]. While these devices are dedicated to sensing and are not usually interacted with by a user, the types of sensing devices available now are diverse in both form and function. Sensor networks can be wired or wireless, homogeneous in nature or heterogeneous, encompassing a diverse range of devices that work together within a single system or multiple systems. Sensing devices may be equipped with intelligent agents enabling autonomous decision-making within a device and/or network, or they may be more passive and simply report phenomena as instructed. Wearable sensors monitor human biological and chemical state. Mobile sensors monitor physical phenomena but not in a fixed place. Embedded sensors monitor phenomena in a fixed place such as a building or in a fixed object such as a mobile user device or utility, and can therefore have a mobile element. Such user devices enable both ubiquitous sensing and interactions and are typically utilised for personal, local Ubiquitous Computing systems such as smart spaces. Smart spaces aim to provide ubiquitous (hidden) services such as ambient personalised displays, mood lighting, and automated services by embedding intelligence in devices within a utilised space such as a home [40] or office. IoT and WSN researchers aim to enable smart spaces through small to medium size deployments of smart things and sensors within an occupied space. Environmental monitoring systems typically gather a diverse range of data relating

29

to the conditions of an environment enabling the development of intelligent ecosystems that react to diverse phenomena such as pollution [89], wildlife migration [82, 79], natural disaster [34, 142] or intrusion [162, 53]. Such systems require large scale WSNs that can be left in the environment unattended for long periods of time, meaning the provision of sensing devices capable of intelligence is required. The solution offered by the Sensor Web is “web accessible sensor networks and archived sensor data that can be accessed and discovered using standard protocols and APIs” [27]. The challenges of acquiring context in Ubiquitous Computing differ to those of WWW research. While WWW requires dynamic search and aggregation to acquire context relating to specific users, topics, or locations, Ubiquitous Computing relies on sensor discovery and programming abstractions that aggregate context of interest to an application. WWW data originates in a context-rich form, i.e., it is typically combined metadata delivered using a data representation such as XML or JSON. Sensor data, on the other hand, originates in a raw form and requires processing and aggregation before it is useful to an application. Both WWW and Ubiquitous Computing face challenges in terms of uncertainty and trust in data. In WWW the key challenges are in natural language processing, human error and intention. In Ubiquitous Computing, sensors can become damaged, lose connectivity or send erroneous readings. Both WWW and Ubiquitous Computing face the challenge of acquiring ‘complete’ contextual descriptions of an event or situation but this challenge will be somewhat addressed by their integration in CPS Computing. While programming support for using WWW data is delivered in the form of mashup tools that create and aggregate data feeds, Ubiquitous Computing researchers turn to WSN middleware and programming abstractions for the discovery of sensors and aggregation of readings related to the interests of the application.

2.3.2

Internet/Web of Things

IoT, WoT and Sensor Web are terms in the research literature that are often given conflicting definitions. Some IoT definitions [13] focus on internet-enabled devices, while others emphasise the networking capability of such devices, connecting WSNs that underpin Sensor Web research (as described in Section 2.3.3), to the IoT [109]. The IoT is now described as ‘sensors and actuators embedded in physical objects - from roadways to pacemakers - that are linked through wired and wireless networks, using the same Internet Protocol (IP) that connects the Internet” [36]. This seems a natural progression from the IoT vision that started in the RFID community where their internet-enabled devices are limited in computational capabilities [40, 179, 165, 12]. IoT researchers have since begun experimenting with more powerful devices or smart objects that are capable of inter-device communication amongst other capabilities such as reasoning and actuation [36, 92]. Additionally, IoT researchers can lean towards a semantic-oriented view of the 30

IoT, leveraging Semantic Web technologies for the integration and use of IoT devices [87, 156, 166]. WoT research, which is seen as the next stage of the IoT, focusses on use of emerging web standards to facilitate reuse of internet-enabled devices and web mashup tools for the creation of web-based applications driven by IoT sensor data [61, 63, 159, 48]. It is anticipated that the composition of these research efforts will realise the future IoT [171, 13], as illustrated in Figure 2.6 (taken from [13]).

Figure 2.6: Overview of IoT Visions [13] IoT and WoT research bridges the cyber-physical divide by adopting modern web technologies that facilitate sharing of data from internet-connected devices to the web. There are a number of open research challenges within this field as described in [13, 164, 171]. Given IoT research leverages the Internet as a means to bridge the cyber-physical divide, it is not surprising that many of the challenges outlined are the same as those WWW researchers face, although often the issues are more complex given the scale and heterogeneity that characterises the future IoT. Addressing problems such as discovery and composition (of services and devices alike), data integrity (measuring trust and reliability of people on the web as well as devices in the IoT), data integration (require standards for WWW and IoT to be complimentary and need to manage incomplete or inaccurate data from physical devices), security (authentication of people and devices and resilience to attacks) and privacy (WWW and device data ownership, access and use rights), are increasingly important and difficult for IoT.

31

There is also similarity between research problems proposed within the IoT community and Internet Architecture researchers. Within the IoT community, these problems are addressed within a middleware solution, most of which adopt a SOA. The SOA design is borrowed from the Web of Services (see Subsection 2.2.2) and is used in IoT to support a network topology that is both unknown and dynamic through the use of discovery methods based on those in WoS research [164, 13]. While some IoT middleware approaches focus on abstracting devices in the network as services [49, 183, 62], others devote more attention to data abstractions and their integrations with services [72, 3].

2.3.3

Sensor Web

Figure 2.7: Global Sensor Web Vision [27] Much like the evolving scope of IoT research, the Sensor Web [43] paradigm envisions connectivity of entities and users in real and web-based environments through the adoption of large scale heterogeneous sensor-actuator networks. A Sensor Web is defined as “web accessible sensor networks and archived sensor data that can be discovered and accessed using standard protocols and APIs” [27]. Sensor Web researchers strive to solve global problems such as environmental monitoring and early warning systems, meaning the importance of open data and systems in the Sensor Web is of huge importance. The Global Earth Observing System of Systems (GEOSS) community in particular strive to integrate global environmental monitoring systems across organisational and political boundaries [184]. This Sensor Web Enablement (SWE) initiative is driven by the Open Geospatial Consortium where a comprehensive framework has been developed to standardise sensor discovery, sharing, access, etc. Parallel research efforts develop Sensor Web frameworks using non-standardised approaches [4]. Such networks support intelligent sensing and actuating, sometimes through the embedding of agents [170] and sometimes even advocating 32

Autonomic WSNs [111]. They treat all entities as equal citizens regardless of their origin, form or nature [124]. The characteristics, or features, of a Sensor Web are summarised from [184] as follows: • Complex adaptive system (heterogeneous components, complex network structure, large number of nodes, adaptability, self-organising) • Organised as a network of open sensor resources that are able to interact • Pervades the internet • Provides external access to the constituent components that form the Sensor Web (open system) • Consists of sensor resources that are any source of sensor data or meta-data and can take any form • Requires well-defined semantics for discovering, using and managing sensors and observational data As a complex adaptive system that pervades the internet, the Sensor Web bears strong similarities to the IoT (see Subsection 2.3.2), future Internet (see Subsection 2.2.4), and CP Computing (see Subsection 2.1.1). The reliance on Semantic Web (see Subsection 2.2.3) technologies to manage the challenges introduced by heterogeneity is also common to each of these paradigms. Given that the Sensor Web is characterised by sensor resources as any source of data in any form, the Sensor Web could be considered as an enabling technology for CPS Computing. However, like CP Computing, Sensor Web research is focussed largely on sensor networks and is lacking direction in terms of the integration of WWW and sensors for context-awareness. Figure 2.7, taken from [27], illustrates a vision of a global Sensor Web where diverse networks of sensor networks are accessible as a single system. The Sensor Web then, is considered as middleware between the sensors, sensor systems, or networks of sensors, and the applications. This is illustrated in Figure 2.8, which displays three main architectural layers: sensor, Sensor Web (middleware) and applications. It is noted that IoT and WoT are shown as a subset of this overall vision but that perhaps much of IoT and Sensor Web research is now converging in terms of objectives and enabling technologies. Much like IoT and WWW research, middleware for Sensor Web enablement largely adopt SOA design [184, 31, 27]. Challenges for the open Sensor Web, similar to those outlined by WoS and IoT researchers, are summarised from [31]:

33

Figure 2.8: Sensor Web layer stack and located middleware classes [31] • Increasing interoperability - lightweight standards are required, reducing the complexity of integrating standards into Sensor Web frameworks, reducing the barrier to adopting these standards • Autonomous discovery and integration of sensors and services • Data quality, provenance and uncertainty management and assurance • Realising the Human Sensor Web and integrating social networks with the Sensor Web (Citizen sensors) • Enabling the Semantic Sensor Web and Linked Sensor Data As in WWW and IoT research, Semantic Web technologies are being adopted to solve some of the problems common to these and Sensor Web research fields. However, this list of challenges assume a Sensor Web consisting solely of open systems and data. Through integration with online social networks and/or deployment in private spaces, future CPS Sensor Webs are expected to collect data that is highly private (e.g., location), giving rise to concerns regarding security and privacy [15].

2.4

Discussion: Moving Forward

This chapter sought to reveal current state-of-the-art agendas towards CPS Computing. Subsection 2.1.1 described existing definitions of CP and CPS Computing and a definition focusing on CPS context-awareness was chosen in Subsection 2.1.2. CPS from the perspective of context-awareness was explored further and the technologies and research agendas that, when unified, will enable CPS context-awareness were identified. These 34

agendas fall under the broad umbrellas of WWW (Web 2.0, Social Web, Web of Services, Semantic Web, Future web) and Ubiquitous Computing (Sensors, IoT, WoT, Sensor Web) research. Through further investigation of each of these sub-fields it emerged that the goals and challenges faced in each are largely overlapping. This is particularly evident in Subsection 2.2.4, which describes a future web that seamlessly integrates representations of physical objects and people. The ultimate goal of using data to provide personalised services to enhance daily activities is common in both WWW and Ubiquitous Computing, although the output of these research areas may be delivered through diverse types of interfaces. Further, the approaches used by WoS and Semantic Web researchers in particular are being adopted by Ubiquitous Computing researchers in support of solving their key challenges in managing cooperation and interoperation of heterogeneous resources. The following sections discuss further how CPS Computing will move forward based on analysis of the research described throughout this chapter.

2.4.1

Challenges and Benefits of CPS Computing

The original Ubiquitous Computing vision described a future where hidden technology is everywhere and we interact with it as naturally as we do our current environments. Technology would take the chore out of our jobs, allowing us to focus on our own personal interests and domains of expertise. To achieve this aim, Ubiquitous Computing strives to autonomously digitise the information we do not actively choose to share. Wearable technologies monitor our physical state; mobile technologies monitor our location (geographical and environmental), movements and social connections; environmental sensors monitor the state of our external physical surroundings, natural and man-made; medical sensors monitor our physical internal, external and mental states. As computing devices become more powerful and durable, embedded sensors will become the norm. Data obtained by these sensing devices, in combination with web-sourced data, will largely complete the contextual picture of individual user and community situations at any given point in time. Advances in technology and efforts towards a truly pervasive future must respect people and their commodities. Data generated by a person is one such commodity, which is owned by the person from whom it originated. People, stripped of technology, act as their own data brokers. Thoughts formulate in our minds and, typically, are only shared (through speech, actions, or otherwise) when the person who generated the original thought desires and decides it is appropriate. This happens seamlessly and decisions to share are made through a person’s rapid analysis of the current situation with respect to historical context. People are very good at adapting to their surroundings and very naturally filter out thoughts that are not appropriate to share in certain situations and/or with particular audiences. Technology, as an enhancement to our every day lives, is required to provide as 35

seamless and intuitive approaches to managing our digitised data as the ones we already have and use. Technology has the potential to enhance our decision-making process by providing greater situational context than we are able to take in (sense) by ourselves at any given time. There are some key challenges relating to privacy and security that are faced by both WWW and Ubiquitous Computing researchers. While it may be tempting for sensor researchers to adopt techniques from the WWW, the ‘third-party data broker’ scenario described in Subsection 2.2.1 has the potential to hinder user uptake of IoT and Sensor Web Services. Web users are beginning to understand the implications of sharing information on the open web but the fa¸cade of data broker privacy management is misleading in terms of the ease with which a user can control the provenance of their own information. Further, the manual nature of managing one’s online connections is not a true contextual representation of the organic way our natural connections develop. A core challenge for Ubiquitous Computing has traditionally been user adoption of AmI applications. The importance of intuitive interaction is as important now as it was when Weiser proposed the original Ubiquitous Computing vision, yet remains a distant goal. In the case of AmI, if the emphasis on the physical element, that is Ubiquitous Computing, is augmented with a social perspective, this may inform the behaviour of AmI systems and make them more human-centric. CPS AmI techniques will be based on a more complete contextual foundation of its users than the knowledge base achievable through embedded sensors alone. Difficulties in assessing a user’s mood or motivations will be addressed through the inclusion of a user’s real-time Social Web presence within the approach. The traditional vision of calm computing encompasses technologies that stay hidden and make decisions without any user input. Motivated by human ability to exploit their environments, Rogers [138] describes an alternative goal to Weiser’s ‘calm computing’ that is engaging computing, shifting from ‘proactive computing’ to supporting ‘proactive people’. It is also argued that we “simply do not do ‘smart’ well yet” [58], but perhaps that is because we do not have access to comprehensive historical data for individual users such that a system may learn to adapt appropriately. It is proposed here that the provision of engaging technologies is a necessary step in the journey to ultimately providing calm computing. As users become more engaged with (and accepting of) technology, they will learn to understand and trust in technological advances. Simultaneously, researchers will learn what users are willing to accept from technology, both in terms of what tasks or decisions they are willing to delegate to automated systems and what level of control users demand over their own data. This intermediate step between engaging and calm computing will facilitate the organic growth of large user annotated data sets (aggregated through user interactions with technology), providing a new historical knowledge base

36

upon which local, personalised, intelligent systems can become automated to suit AmI users. While it is envisaged that a supporting framework will bridge the CPS divide and manage the heterogeneous data sources and services of the cyber, physical and social worlds, it must be considered how that framework will operate. While services exist that facilitate the sharing of WSN, or IoT, data to the web, the publication of private sensor data to an external data broker will lead to concerns surrounding user privacy and ownership of data. The idea that all user information, including that originating from sensors embedded in private environments, is managed by third-party data brokers goes against modern social expectations and will lead to resistance in terms of adopting these new technologies. Advances in information retrieval, dissemination and capture technologies far exceed those in areas of web data provenance and privacy. A contributor of information to the current WWW must understand the privacy controls of hugely diverse service providers whose terms and conditions may change at any point. Relying on these external data brokers to control and secure information means users cannot be certain of who has access to their information and for what purpose. It is therefore necessary when bridging the CPS divide, to provide a safe place where information processing and intelligent decision-making can take place while adhering to user defined privacy requirements. This results in the requirement for web-based data to be integrated with WSN data within a local distributed framework that supports fully customisable privacy and security services. Such an approach will facilitate fusion of widely diverse user data, integrating WWW, Sensor Web and IoT research efforts to date, as proposed in the following section.

2.4.2

CPS Computing Infrastructure

This section discusses how WWW and Ubiquitous Computing will converge to form CPS Computing. This discussion accounts for the challenges and benefits discussed in the previous section, as well as the research agendas outlined earlier in this chapter. The most widely-accepted definition of Web 3.0 is the Semantic Web offering described in Subsection 2.2.3 and Figure 2.5, which illustrates the integration of semantic technologies into Web 2.0. At this time, IoT technologies are beginning to emerge and are now accessible in terms of cost and ease-of-use. Simultaneously, Web 2.0 technology companies are facing major challenges in terms of user data, privacy and security. Increasingly, users are dissatisfied by unwanted personalised experiences and policy and legal issues relating to data protection highlighted by the media. These challenges may prevent IoT systems from evolving as rapidly as the web evolved to Web 2.0. The success of Web 2.0 was driven by the participatory nature of the Social Web. The more people that use it the better it gets. This model, for the IoT, will serve to increase user concern over data protection should a third-party service provider and host maintain responsibility of managing user 37

Figure 2.9: (A) Open Web 3.0 and WoT (B) Private webs of People co-existing with Open Web 3.0 ([121]) data. In Figure 2.9 (A), Web 3.0 is extended to also accommodate the WoT. It is noted in this illustration that Web 2.0 user and device data is essentially public. Users and devices must adhere to the policies set out by the third-party service provider. Also in parallel is the development of distributed, peer to peer, mobile social networks. The goal here is to allow a user to maintain ownership over their data. Data is encrypted and distributed between each peer in the network, ensuring user data is secure and only accessible to services and/or people whom a user chooses to share it with. While this model would mean the software provider has little to no data to monetize, it will provide a service in a manner that meets user demand and requirements. A similar model for the IoT and Sensor Web paradigms can be combined with these distributed Social Webs to form private Sensor Webs hosted by home cloud systems rather than the service provider, forming CPS machines. Figure 2.9 (B) extends this further to accommodate this vision of a social machine. These are depicted as private Sensor Webs encompassing all things and devices belonging to a user, potentially utilising distributed P2P Social Networks as a portal to their CPS data, services and connections. The figure also illustrates the sharing of data from private Sensor Webs to a web of People. Such data will be abstracted versions of data such as user music preferences. Then, there is the potential of a data market place. ‘Data consumers’ on the WWW such as marketers, researchers, brands, journalists, offer enhanced services or payment for access to targeted, abstracted, aspects of user data such as ‘favourite soft drink’, ‘favourite 38

music genres’, ‘areas of expertise or influence’, ‘locations of interest’. Data consumers also have access to data that users publish publicly to the open web. Users can connect their Sensor Webs when they interact with each other, combining individual Sensor Webs to form groups around families, social relationships, professional relationships, etc. Public, open data is published using Web 2.0 and Internet of things (e.g. opendata.org), enabling the development of applications to solve global problems by anyone, anywhere. Then, private Sensor Web data can be used to configure global services to solve localised and personalised versions of global problems from the user or community perspective. Since the data resides in control of the user and select abstracted information is shared with approved third parties only, all accessible data is of relevance to the system. The system will be required to process data in real-time, revealing abstract information, or insights that they share with approved data consumers. The user requires an interface for monitoring what has been shared with whom and the changes in service operations that resulted from it. This is important to ensure that users are supported in managing their data appropriately given their preferences and requirements. This discussion has attempted to unify the research agendas found in WWW and Ubiquitous Computing research to form the underlying infrastructure for CPS Computing. This is illustrated further in Figure 2.10, which provides a conceptual overview of how these technologies will interoperate. A description of each illustrated component forming the underlying infrastructure for CPS Computing follows: Web > 3.0: The future web that forms a global interface to CPS web applications enabled by the underlying infrastructure Semantic Web and Linked Data: A publicly accessible and queryable model representing the current state of Web 3.0 applications, Web 2.0, services, WoT devices and the web of People. While only this, open, Semantic Web is illustrated in the architecture, it is expected that Semantic Web technologies will also pervade many private data-driven applications also Web 2.0: The interactive, participatory, dynamic Social WoS WoT: A portal to all internet connected devices or things made accessible through Web 2.0 services and integrated into web > 3.0 applications Web of People: The perspective of people that is made accessible to the WWW, i.e., public facing user profiles, content and interactions that can be integrated into Web 2.0 services and web > 3.0 applications Sensor Web: A platform for the management of all sensors, agnostic of origin, form or nature, acquiring data about a user, entity, or environment. User Sensor Webs 39

Figure 2.10: Middleware Enablement for CPS Web > 3.0 ([121]) become social Sensor Webs through interactions within a community of Sensor Webs, potentially driven by a distributed, peer-to-peer, mobile social network Augmented Reality Spaces: immersive software environments in which users and entities interact as seamlessly as they do in physical spaces The supporting framework is highlighted in Figure 2.10 as middleware and programming support. Programming support is required to facilitate the development of CPS context-aware applications in a manner that provides a unified interface to the complex underlying data sources. Middleware then, is responsible for acquiring the data and managing services and their orchestration in support of the application requirements. The role and requirements of this supporting framework are explored further in the following section.

2.4.3

Role of a Supporting Framework for CPS Computing

Having defined the underlying infrastructure that will realise CPS Computing, the role and requirements of a supporting framework for the paradigm can be derived. A supporting framework for CPS Computing must enable monitoring of relevant users, entities and locations in the cyber, physical and social worlds. It must also provide technologies that make acquisition of CPS context of relevance to systems and applications accessible, with 40

a hierarchical balance of flexibility and control versus technical skills, to a range of developers from network engineers to application end-users. These developers will focus on domain expert application logic and the supporting framework must manage all lower-level problems requiring only high level configuration of the monitoring networks. This goal for accessible programming supports was identified in both WWW (Subsection 2.2.2) and Ubiquitous Computing (Subsection 2.3.1) and CPS Computing requires a unified solution. SOA was identified as a common approach to enabling WWW, IoT and Sensor Web initiatives. Despite the differences between WWW and Ubiquitous Computing, a mapping can be made between the concepts and challenges to be addressed by services, enabling a unifying solution to bridge the CPS divide. An adaptation of a layered SOA architecture for CPS enablement is described as follows (adapted from [13]): Objects Sensors, Smart devices, WSNs, Targeted aspects of any software including WWW sites and services Object Abstraction Software that unifies access to and control of objects regardless of original form, converting sensed data to a standardised format Service Management Software services that provide features to simplify deployment and management of a complete infrastructure, e.g., standards, object discovery, configuration and status monitoring. Trust, privacy and security, if not inbuilt to the core of the middleware architecture, are typically provided as services that can be configured as required Service Composition Supports the application developer in composing applications supported by the infrastructure through the composition of predefined services exposed through an API Applications Multiple, concurrent applications competing for infrastructure resources in a way that is managed by the previous layers according to its configuration Bridging the CPS divide introduces some key challenges in each of the ‘object abstraction’, ‘service management’ and ‘service composition’ layers. These challenges are described in more detail in Chapter 3 and motivate further analysis of related work with an aim to achieving the objective of designing a supporting framework for CPS contextawareness.

41

Chapter 3

Realising Cyber-Physical-Social Computing Chapter 2 explored the emerging Cyber-Physical-Social (CPS) Computing paradigm which was defined in Subsection 2.1.2 as encompassing a holistic treatment of data, information, and knowledge from the cyber, physical and social worlds, to provide contextually relevant abstractions to humans and software agents. The underlying infrastructure that will facilitate CPS context-awareness was identified as comprising both World Wide Web (WWW) (Web 2.0, Social Web, Web of Services (WoS) and Semantic Web) and Ubiquitous Computing (Wireless Sensor Networks (WSNs), Web of Things (WoT) Internet of Things (IoT), Sensor Web). An overview of this infrastructure was provided in Subsection 2.4.2, which also highlighted the role of a supporting framework for CPS Computing, with respect to the challenges and benefits of the emerging paradigm as described in Subsection 2.4.1. Key challenges included the requirement of context gathering in a manner that is supportive of users’ privacy and security requirements. In support of the objective to design a supporting framework for CPS context-awareness, the role and architecture of such a framework was explored further in Subsection 2.4.3. Service Oriented Architecture (SOA) was identified as a common approach to supporting frameworks in WWW and Ubiquitous Computing research. A mapping was made between the architectural offerings in the research literature, resulting in a unified layered SOA design for CPS Computing that comprises the layers: objects; object abstraction; service management; service composition; and applications. Bridging the CPS divide introduces some key challenges in each of the ‘object abstraction’, ‘service management’ and ‘service composition’ layers. Section 3.1 describes points of particular importance to the contributions of this thesis; and serve as motivation for discussion on related research as indicated throughout. In Section 3.2, existing approaches to bridging the CPS divide are reviewed and evaluated with respect to the challenges defined in Subsection 2.4.1. The sensor middleware approach emerges as the most suitable 42

approach for enabling CPS context-awareness. Research offerings in this category are evaluated with respect to the previously defined requirements in Subsection 3.2.4. This evaluation reveals that existing systems do not meet the requirements to enable dynamic creation and configuration of sensors monitoring WWW resources. Further, they do not provide high-level abstractions to simplify the dynamic creation of collections of sensors. With an aim to identifying existing WWW and Ubiquitous Computing research that will unify to meet these requirements, Section 3.3 explores techniques for Web Service mashup development and WSN programming. This chapter concludes in Section 3.4 with a discussion on how WWW and Ubiquitous Computing techniques will be unified in the design of middleware and programming support for CPS context-awareness.

3.1

SOA Requirements for CPS Context-Awareness

Unified object abstraction:

The object abstraction layer requires the design of a

unified interface to complex underlying objects. In the case of CPS Computing, these objects include physical sensors, networks of sensors (as described in Subsection 2.3.1), websites, services, Application Programming Interfaces (APIs) and feeds (as described in Section 2.2). The unified interface will be used by users such as network engineers (managing a deployment), as well as software comprising the management services in the next layer. It is also required that the data produced by both WWW and sensor resources be transformed into a unified format for use by both application developers and data-processing services within the framework. A key concern for CPS Computing is the approach that is taken to unifying WWW and sensor resources within a single system. The approaches adopted in related research efforts are explored in more detail in Section 3.2, with particular attention to the challenges CPS Computing faces, as outlined in Subsection 2.4.1. The approach most suited to enabling CPS context-awareness is chosen based on this analysis; and existing techniques in this category are evaluated in Subsection 3.2.4, with respect to the requirements described here. WWW service management:

The service management layer traditionally includes a

means to discover new sensors as they enter a network; to discover new sensors as they become contextually relevant to an application; and to re-task sensors based on changing application requirements. Given that CPS Computing also includes the use of objects monitoring WWW resources, the mechanics of management services for these objects needs to be considered. The key concern is who or what creates the object that monitors a WWW resource and how and when can it be (re)configured? While network engineers could be responsible for creating the objects at deployment time, there must also be support for programmatically creating and configuring the objects at runtime in support

43

of changing application needs. Additional management services that must be applied to WWW resources, but are beyond the scope of this thesis, include: resource constraint management (API call limits, server failures, etc.), trust and uncertainty management (natural language processing, reputation algorithms, etc.). Related research efforts are evaluated with respect to these requirements in Subsection 3.2.4. Comprehensive CPS context acquisition: While the lower layers are aimed at providing supports to services and users such as network engineers, the service composition layer is aimed at supporting orchestration of services both programmatically and manually by applications and their developers. For CPS Computing a key challenge is in the aggregation of comprehensive and relevant context, from both WWW and sensor resources. Approaches exist in both WWW and Ubiquitous Computing research but do not explicitly support a unified technique with respect to the requirements of the SOA layers. From WWW research, mashup tools are the common approach to facilitating data filtering and aggregation, while Ubiquitous Computing research focuses on WSN programming abstractions for the identification of relevant sensor nodes. These approaches are explored in more detail in Subsection 3.3.1 and Subsection 3.3.2 respectively, with an aim to designing the general-purpose, unified approach that is required by CPS Computing but missing from existing research. CPS Computing requires a unified approach which will serve both end-users and systematic configuration of objects within the framework.

3.2

Approaches to Bridging the CPS Divide

Rosi et al [139] reviewed existing approaches to integrating social networks with Ubiquitous Computing systems. The authors categorise the approaches as (1) extracting data from social networks, (2) exploiting social networks as a socio-pervasive middleware, (3) pervasive overlays on social networks, and (4) creating new application-specific socio-pervasive networks. This section reviews approaches to bridging the CPS divide in categories based on the method of WWW - sensor integration. First, WWW-accessible devices and networks, where the aim is to incorporate ubiquitous technologies with the WoS (as described in Subsection 2.2.2). Second, approaches that incorporate ubiquitous technologies with the Social Web (as described in Subsection 2.2.1). Finally, approaches that leverage Web technologies within middleware for Ubiquitous Computing (as described in Section 2.3). Each of these approaches are critically analysed with respect to the benefits and challenges of CPS Computing described in Subsection 2.4.1. An appropriate approach is identified and evaluated further in Subsection 3.2.4.

44

3.2.1

WWW Accessible Devices and Networks

This approach essentially brings WSN and other sensor or actuator equipped devices into what is envisaged as the programmable WoS. Many WoT and Sensor Web offerings fit into this category through adopting web standards to enable access to and use of physical sensing devices and infrastructures through traditional, or purpose built, Web Service architectures. At a basic level, sensor devices are connected to the Web via an API provided by an IoT framework such as Xively1 , or equally using standards such as those defined by the Sensor Web Enablement (SWE) initiative [27]. Rouached et al. [140] extend a Sensor Web middleware, 52 North 2 , to validate their proposal for the adoption of the lightweight Representational State Transfer (REST) Web Services concept and the usage of JavaScript Object Notation (JSON) format as an alternative to the verbose XML for exchanged messages in the Sensor Web. The authors note that “the sensor Web is to sensor resources what the WWW is to general information sources, an infrastructure allowing users to easily share their sensor resources in a welldefined way”. By describing sensor resources and data using the JSON format, sensor data is in a format that is commonly used for describing data in the WWW. However, the middleware does not explicitly accommodate the integration of WWW data sources, focusing on the processing and presentation of data emitted from physical sensors alone. SenSer [129] is a generic middleware framework that allows for web access and management of Sensor networks by virtualising their functionalities as services, in a way that is independent of programming language and sensor network platform. It provides a Javabased implementation that exposes the framework as two Web Services, cleanly separating regular user operations from administrative operations. A similar approach is adopted by Akribopoulos et al. [8], with emphasis on integrating small programmable objects in the WoT. While some approaches in this category facilitate the processing of fused sensor streams, no explicit support is provided for the integration of data originating in the WWW. In this way, this category is considered an enabler for Cyber-Physical computing rather than CPS Computing. There is, however, the potential to adapt mashup development techniques to accommodate sensor data that has been transformed to a standardised web format such as JSON. Despite this potential, the method of integration of WWW data with sensor data to be exposed as CPS services in the WoS will be hindered by existing privacy and security concerns in the WWW. Users will be reluctant to engage with CPS applications should they be delivered in this manner. It is proposed that such an approach is suited to integration of public or open global networks, in line with the goals of the open Sensor Web ( Subsection 2.3.3) but not for networks that are sensitive in nature, including 1 2

https://xively.com/ http://52north.org

45

devices or software environments connected to individual users.

3.2.2

Utilising and Extending Social Networks

While the previous discussion summarised the unification of Ubiquitous Computing with the WoS, approaches in this category focus on leveraging the Social Web. Most research of this nature makes use of the existing infrastructure of Social Networks for sharing sensor readings and managing problematic aspects of Ubiquitous Computing research such as communications, security, privacy etc. For example, Rahman [134] presents a framework that uses open source hardware and software to merge Body Sensor Networks (BSNs) with social networks. It captures events from a BSN and sends them to a list of remote interested parties. Similarily, Kranz et al. [93] extends social networks with technical networks such as WSNs to demonstrate the ability of social platforms to not only connect humans to each other but also things to humans. Breslin et al. [29] further explores the research areas where sensors and social networks can interface, from sensors providing contextual information in context-aware and personalised social applications, to using social networks as ‘storage infrastructures’ for sensor information. S-Sensors [19] envisages a range of sensor-to-web integration models to improve the utilisation of sensory infrastructures and data via the advanced coordination and collaboration of sensor nodes and web applications. The authors propose to employ microblogging to publish and share sensory data and resources; and provide a framework to globally share locally-measured sensory readings. It is stated that privacy (e.g., private biosensor readings) will be coherent with the existing infrastructure of the internet and will not require special control options. Citizen sensing was identified in Subsection 2.2.1 as a new research function enabled by the Social Web. Demirbas et. al. [44] propose that microblogging services such as Twitter3 can provide an open publish-subscribe infrastructure for sensors and smartphones, and pave the way for ubiquitous crowd-sourced sensing and collaboration applications. The authors design and implement a crowd sourced sensing and collaboration system over Twitter, and demonstrated the system using two case studies: weather radar and noise mapping. Given the rapid growth and evolution of Web 2.0 and the Social Web, social networks present themselves as an ideal platform to extend for the sharing and dissemination of a wide range of sensor data. However, the privacy and security challenges that are presently facing most social networks will prevent their evolution towards CPS networks. As described in Subsection 2.4.1, social network users are becoming more aware of their privacy and the providers of those platforms are being challenged by their users (and the media) 3

http://www.Twitter.com

46

on a more frequent basis. The existing model where the provider of the platform acts as the data broker for all the platforms users, will need to be addressed before the Social Web will evolve to a CPS Web. As described in Subsection 2.2.1, peer-to-peer mobile social networks are beginning to emerge as a result of these issues. It is proposed that the ideas and techniques presented by the authors in this category will be more likely to succeed by leveraging these emerging types of social networks.

3.2.3

Extension of the Sensor Middleware Paradigm

The previous two categories of CPS Computing enablement essentially push WSN data and services to the existing WWW infrastructure. This category differs in that the opposite approach is taken, i.e., WWW data and services are brought to the middleware level. While research in the previous two categories typically emphasise integration with either the Social Web or the WoS, this category offers approaches that are inclusive of both. Specifically, WWW data and services are integrated using a unifying software sensor paradigm. An example that is closely related to the previous category is offered by Beach et al. [20]. The authors identify mobile social networks as an important new direction of research in mobile computing and show how an expanded definition of mobile social networks, that includes sensor networks, can enable new context-aware applications. They propose that future mobile social networks will be defined as ‘any distributed system that combines mobile, social and sensing components’. The authors offer SocialFusion as a system capable of integrating diverse mobile, social and sensing input streams and effectuating the appropriate context-aware output action. Social Fusion collects, organises, processes and filters this data. Then, a bank of classifiers operates on the assembled raw data to attempt to infer meaningful context from the data. Finally, a recommendation engine employs data mining to generate context-aware output actions for users. Construct [47, 158] is a standards-based sensor fusion middleware that integrates data from both physical and virtual sensors. Virtual sensors in Construct can be written in a wide variety of programming languages and obtain information from the WWW, local network or local computer. Virtual sensors and other entities are discovered by the middleware using the ZeroConf4 open standard. Physical and virtual sensors in Construct are treated as equal citizens with respect to their role as inputs to uncertain reasoning algorithms. Construct employs the RDF open standard for modelling of all data that supports SPARQL5 querying of the context of the system. The Global Sensor Networks (GSN) platform aims to offer a flexible middleware that enables the dynamic integration and management of heterogeneous sensor networks and 4 5

http://www.zeroconf.org/ http://www.w3.org/sparql

47

the produced data streams [3, 4, 2]. The access layer of GSN makes no assumptions of sensor networks other than the connectivity of sink nodes to a base computer through use of a software wrapper that conforms to the GSN API. These wrappers enable the definition of virtual sensors in XML6 , allowing for the inclusion of WWW resources as virtual sensors within the middleware. Using a container-based architecture, local and remote virtual sensors in GSN abstract the implementation details of sensor data access and define the data stream processing to be performed. Users can then build a data-oriented ‘sensor internet’ consisting of sensor networks connected via GSN through a combination of virtual sensors, their data streams and the associated SQL query processing. Users can access their GSN deployment as a Web Service, retrieving data in XML or comma separated value (CSV) format. The Linked Sensor Middleware (LSM) [130] is a platform that brings together sensed data and the Semantic Web (described in Subsection 2.2.3). Like GSN, its access layer adopts the concept of wrappers to enable integration of heterogeneous sources, including web-based sources such as RSS feeds and social networking APIs. Wrappers that enable the collection of data from sensors are termed ‘physical wrappers’. Mediate wrappers in LSM enable connectivity of other platforms, such as GSN, and transform diverse data formats into RDF. Linked data wrappers provide access to the sensor data resources by exposing the relational data structure in RDF, as in the approach adopted by Construct. Owing to the method of integration being the reverse of the first two categories, the sensor middleware approach is not limited by the privacy and security issues of the WWW, assuming that appropriate, user-oriented, privacy and security services are employed by the middleware. While the SocialFusion middleware is focussed solely on human-centred CPS Computing driven by social connectedness, Construct, GSN and LSM offer more generalpurpose frameworks that can be applied in both social and non-social use cases. Despite the latter three middlewares integrating WWW resources as ‘equal citizens’, they fail to adequately support the management of virtual sensors with respect to their optimisation for CPS context gathering. Construct, GSN and LSM are evaluated with respect to the SOA layer requirements in the following section.

3.2.4

Evaluation of Existing Approaches

The approaches to enabling CPS Computing were discussed in the previous three sections and analysed with respect to the challenges outlined in Subsection 2.4.1. The approach most likely to succeed for a broad range of use cases, assuming research on privacy and security mechanisms progresses, is the sensor middleware approach. In this approach, WWW resources are accommodated in a sensor middleware using a unifying software sensor abstraction. The key benefit to this approach is the ability to deploy the sensor 6

http://www.w3.org/xml

48

middleware locally (e.g., home or office) or indeed as a Web Service. Mechanisms relating to access, security and privacy must then be customised based on the requirements of the owner of the deployment. Table 3.1 highlights the requirements of CPS middleware with respect to the WWW integration requirements at each middleware layer, as described at the start of this chapter. This section discuss the features of three sensor middlewares, Construct [47, 158] , GSN [3, 4, 2], and LSM [130], introduced in Subsection 3.2.3 with respect to these requirements. Table 3.1: Middleware characteristics with respect to CPS sensor middleware requirements Sensor Middleware CONSTRUCT

GSN

LSM

WWW Middleware Integration Middleware WWW

Any language software

Integration Method

& Zero-Conf for discovery

Wrapper using GSN API

Wrapper using LSM API

Virtual Sensor

Virtual Sensor

Virtual Sensor

Data Format

RDF

XML, CSV

RDF

Data Access

SPARQL

Web Service

SPARQL

Object Abstraction Method

WWW Service Management Manual creation/deployment Dynamic creation & configuration of sensors

Zero-conf discovery No

Virtual Sensor XML Specification No

Virtual Sensor XML Specification No

WWW Context Acquisition Real time stream processing/aggregation Abstractions for dynamic collections of sensors

N/A

Virtual sensors XML & SQL

Visual mashup interface

No

No

No

WWW Middleware Integration:

All three of the sensor middlewares are identified

as meeting the requirements of the object abstraction layer. This is achieved by the incorporation of WWW resources within the middleware as virtual sensors. In the case of Construct, virtual sensors can be created in any programming language and are connected to the middleware through the use of the open standard Zero-Conf. Using Zero-Conf, Construct discovers sensors and converts sensor readings to RDF. The data is integrated into a global distributed model of the system with the support of a gossiping algorithm for data dissemination. Users then, query the model using SPARQL to acquire context for use in applications. GSN provides an API for the development of wrappers that accommodate either sensor or WWW resources within the middleware. Individual virtual sensors are then defined using XML to enable targeted monitoring of the WWW resource. The middleware, and the data from its constituent sensors, can be accessed via REST as a Web Service allowing developers to retrieve XML or CSV data for use in applications. A similar approach 49

is adopted for integrating WWW resources in LSM. WWW resources are integrated by creating wrappers using the API and virtual sensors are defined using XML. Data management in LSM however, is more similar to the approach used in Construct. Data is transformed to RDF, added to a semantic model, and SPARQL is used by application developers for the acquisition of context. WWW Service Management and Context Acquisition:

In Construct, virtual

sensors are created at deployment time and discovered using Zero-Conf, while GSN and LSM require the specification of XML-based deployment descriptors for virtual sensors. None of the offerings support dynamic sensor creation at run time. While it can be considered that these middleware offerings treat WWW resources as equal citizens to sensors through the virtual sensor abstraction, they fail to accommodate the fundamental differences of the underlying resources. Specifically, virtual sensors are designed as software representations of non-changing sensor types that ‘exist’ in the real world. WWW resources are vast in terms of data availability and can be monitored using a wide range of parameters to facilitate targeted monitoring. They benefit from not requiring a physical device to be monitored and as such the ability to create virtual sensors ‘on-the-fly’ will add great flexibility to developers seeking to meet the context requirements of applications. While GSN and LSM support filtering and aggregation through SQL and a visual interface respectively, these manual approaches do not support high-level specification of dynamic context acquisition requirements. For example, the need to create new sensors or configure sensors based on data acquired from other sensors in the deployment. The key objective of this thesis is to design a supporting framework for CPS contextawareness. This not only requires the integration of WWW resources as equal citizens within a sensor middleware, but also for the unique characteristics of WWW resources to be leveraged to deliver of optimised CPS context-gathering support. The following section investigates WWW mashup and WSN programming abstractions in support of the design decisions of the contributions of this thesis.

3.3

Towards Programming Support for CPS Context-Acquisition

While WWW resources integrated with a sensor middleware enable unified object abstraction within a supporting framework for CPS Computing, additional support is required to simplify access to comprehensive yet refined context of relevance to an application. For CPS Computing, this will include dynamic creation of objects for monitoring WWW resources as well as the filtering and processing of their real-time data streams. No sensor middleware has been identified as supporting this requirement. This section reviews

50

approaches from both WWW and Ubiquitous Computing research, which aim to simplify context acquisition by developers. From the WWW perspective, mashup development tools (see Subsection 3.3.1 offer support to programmers and Web 2.0 users in the development of applications based on the composition of data feeds and Web Services. WSN researchers from the Ubiquitous Computing domain typically adopt WSN programming abstractions (see Subsection 3.3.2) as a technique to simplify the process of acquiring data from complex sensor networks.

3.3.1

Web Service Mashup Development

The core programming abstraction provided in Web 2.0 is the Web Service (introduced in Subsection 2.2.2). CPS context-awareness requires the integration of Web Services in a sensor middleware and the support for creating ‘CPS mashups’. Web Services enable developers of context-rich applications to perform complex analysis or interactions within their system, without needing to understand or implement the functionality of pre-existing Web Services that are utilised. The difficulty involved in designing and developing Web Services depends on a number of factors including the scale of the service and its requirements. An important feature of Web Services that justifies their use as a programming abstraction is their loosely-coupled and interoperable nature. Web Services provide developers with the ability to produce complex systems by integrating composite Web Services (i.e., multiple services used in cooperation) with their own application logic. These features of Web Services largely influenced the adoption of SOAs in Ubiquitous Computing (Subsection 2.4.3). While Web Service Composition as a methodology for application development introduces great flexibility, extensibility, and interoperability to the development process, there are still a number of issues to be managed. A Web Service is a software system that resides on a server that has some quantifiable processing power. The service itself could be stateful or stateless. A stateful service needs to maintain some memory, which makes memory capacity of the server important also. Many Web Services manage these issues by enforcing usage limits. This logical approach insures that the service lifetime is not jeopardised by multiple applications simultaneously over-loading the system. It also however, imposes further complications on the application development process. As is the case when managing multiple data sources, an application that incorporates multiple diverse Web Services is required to manage these limitations while hormonizing the output of each Web Service in order to ensure a usable service that adheres to the restrictions of the services. Yu et al. [181] emphasise the challenges in developing web mashups without the use of a development tool, stating that manual mashup development “requires programming skills and intimate knowledge about the schemas and semantics of data sources or the 51

business protocol conventions for message exchange”. While modern Web technologies have somewhat simplified the process, it is acknowledged that mashup tools will benefit development, speeding up the process but also allowing end-users to compose their own mashups. The following sections explore mashup development environments with respect to their characteristics; the levels of abstractions they offer; and the approaches adopted by existing tools. 3.3.1.1

Characteristics of Mashup tools

Yu et al. [181] and Maximilien et al. [112] survey existing mashup development tools, providing insights into their characteristics and the functionality that is required. Yu et al. find that mashup tools typically differ in two complementary aspects: the mashup paradigm as the basis of the approach; and the software instrument that implements the chosen approach. For CPS Computing, the latter is determined by the underlying middleware of which the programming framework is an extension. The former is characterised by looking at the objects of integration (components) and how such objects are orchestrated (composition logic). In a sensor middleware approach to supporting CPS context-awareness, a key component is a sensor (WWW or physical), or indeed a service (Web Service or internal to the middleware). The component model determines the nature of components and influences how they can be composed. Yu et al. describe the component model as being characterised by its type (e.g., data or application logic); interface (e.g., API in language or mark-up); and extensibility (ability of users to create new components or extend the component model to accommodate new operations). For CPS Computing, sensors have a data type while services have either a data or application logic type; some services simply add context while others perform some action. The interface is determined by that exposed by the object in the middleware, e.g., XML in the case of virtual sensors in GSN and LSM. Extensibility is also determined by the middleware design and is a typical feature of Service Oriented Middlewares like GSN and LSM. The authors note that a well-defined component interface facilitates reusability, while a flexible component interface ensures extensibility. Once the components are well-defined and assuming they are readily available, the composition model determines how these components are integrated to form the mashup. Yu et al. describe the composition model as being characterised by its output type (e.g., data or application logic); orchestration (e.g., flow-based or event-based); and data-passing style (e.g., data-flow, blackboard). The output type is the output of the final component as influenced by the preceding components in the composition. Orchestration in CPS Computing should be flexible and accommodate both flow-based and event-based integrations. For example, continuous processing based on a stream of data versus triggered processing based on a detected event. The data-passing style will vary between CPS supporting 52

frameworks depending on their goals. Maximilien et al. [112] outline the tasks typically involved in mashing up Web Services which include data mediation and process mediation. Data mediation involves converting, transforming and combining data elements from one or more components and is informed by the output type of the component models. Process mediation choreographs different APIs to create a new process and is informed by the orchestration and data-passing style characteristics of the composition model. These tasks are not dissimilar to those specified in the requirements for the SOA layers of CPS Computing. 3.3.1.2

Levels of Abstraction

Mashup development tools offer varying levels of abstraction to support a wide range of end-users. Grammel et al. [56] acknowledge the relationship between technical abilities and the range of applications that can be developed, stating that “since raising the levels of abstraction requires encapsulating concepts, the number of choices available on higherlevels of abstraction is usually restricted”. The authors identify three levels of abstraction to determine the level of technical and programming abilities required of developers of mashups: High level of abstraction: no programming knowledge is required, but the flexibility is usually restricted to reusing and configuring mashups that are developed by others. This is supported in several ways: (1) reuse of complete mashups created by others (2) high-level mashup and widget parameterisation (3) automatic reuse of data extractors for websites and (4) programming by example Intermediate level of abstraction: knowledge about concepts such as data flow, data types or UI widgets is required but the technological details of these concepts are encapsulated in notations. This is usually supported through visual data-flow languages Low level abstraction: programming knowledge is required but the greatest flexibility is achieved. Textual Domain Specific Language (DSL) editors for languages such as HTML, Javascript and custom scripting languages are used to represent mashups or elements of mashups Integration of different abstraction levels: due to the trade-off between level of abstraction and flexibility in designing mashups, this provides many benefits. Having several abstraction levels allows the user to choose the right level of abstraction for their goal and skill. It also positively affects the ease-of-learning the functionality of a system by offering a gentle slope of complexity, and it enables reuse of elements from lower-levels of abstraction in higher-levels of abstraction 53

The research of Grammel et al. indicates the emphasis towards end-user (rather than programmer) support in mashup development. While end-user support is not the objective of this thesis, the levels of abstraction can be related to user interaction with the SOA layers for CPS Computing. The high-level abstraction indicates the need for context-gathering services that can be configured to produce data relating to particular semantics from a sensing infrastructure. Such a service could be delivered in the form of configurable templates, programs or visual interfaces. This abstraction also specifies the need for reusable services for extracting WWW data. This is also required of the object abstraction layer in the SOA for CPS Computing. The intermediate level of abstraction requires the ability for users to have more control over the composition of the services. Support for manual service orchestration by manipulating and integrating individual objects is required. The low-level abstraction requires the greatest flexibility. In terms of CPS mashups, this would require the ability to integrate new sources of data and custom services. This can be supported easily in a SOA, as evidenced by the use of wrapper APIs in GSN and LSM (Subsection 3.2.3). The integration of different abstraction levels within a single framework will support the development of the skill sets of users who may begin by using higher-level abstractions. These users will be able to start developing more customised applications as they learn more about the individual components and how they are composed and orchestrated. The following section reviews the approaches adopted by existing WWW mashup tools. 3.3.1.3

Mashup Tools in Practice

Microsoft Popfly (discontinued) was a mashup development environment that combined development perspectives that exposed different levels of complexity, serving users of varying technical abilities. Its main perspective was a view in which 3D blocks could be wired together, that was essentially a visual data flow language. Additional views allowed configuring and reusing mashup blocks as well as lower-level programming. Yahoo! Pipes7 focus on supporting end-users in the creation of information mashups using a visual data-flow language. The Pipes allow users to aggregate and transform Web Services, pages, and feeds. The results display in feeds, in lists, on maps, or as photo viewers, depending on the output data. Deri Pipes

8

is the visual mashup editor used by the LSM middleware introduced in

Subsection 3.2.3. The design of Deri Pipes is inspired by Yahoo! Pipes and allows the visual implementation of customised services and information streams by processing and combining WWW sources using a set of simple operators. 7 8

https://Pipes.yahoo.com/Pipes/ http://Pipes.deri.org/

54

Maximilien et al. [113] present a more technical alternative to visual mashup programming in the Swashup DSL domain-specific language. Swashup DSL unifies the most common service models and facilitates service composition and integration into end-useroriented WWW applications. A domain-specific language is a ‘mini’ language built on top of a hosting language that provides a common syntax and semantics to represent concepts and behaviours in a particular domain. Swashup DSL directly represents in the syntax, the concepts necessary to cover three main components of their conceptual model for mashups: data and mediation; service APIs, their protocols and choreography; and a means to generate web applications with customised user interfaces for the resulting mashups. Naming conventions are used to simplify the language’s usage and to make the resulting code more compact. The approaches taken to developing WWW mashup environments can be directly related to the SOA layers for CPS Computing as described at the start of this chapter. Despite the emphasis on manual and visual programming interventions for ‘non-programmer’ end-users, there also exists research efforts towards programmatic composition of mashups. Data-flow processing is identified as a common approach that is well suited to the need for acquisition of CPS context in real time. However, applying the mashup paradigm directly to CPS context gathering would be tedious due to the large numbers of sensors that would require individual integration. The next section reviews approaches to managing this problem from Ubiquitous Computing research.

3.3.2

WSN Programming Abstractions

The contributions of this thesis are most concerned with the types of programming abstractions are typically adopted by WSN middleware and programming framework developers to abstract the complicated nature of underlying WSNs and simplify the data acquisition process for WSN application developers. Mottola et. a., [131, 118] state that for WSNs to emerge from research labs and impact society at large they must address the challenge of developing methodologies, techniques and abstractions for an improved application development process. Wang et al. [174] describe programming abstractions as being the foundation of WSN middleware, providing (1) high-level programming interfaces to the application developer, enabling the developer to focus on application logic without the distraction of problematic WSN infrastructures and (2) the basis of developing desirable middleware services. Despite the different nature of WSN programming to WWW mashup development, the aspects of consideration in the research literature are similar. For example, Wang et al. [174] describe abstraction level, programming paradigm and interface type as aspects of key consideration for the development of WSN programming abstractions. In WSN terms, the abstraction level refers to how the application developer views the underlying sensor 55

system. Low-level abstractions support the configuration and acquisition of data from individual nodes, while high-level abstractions provide a single programmable substrate and an intermediate between the developer and the underlying sensing infrastructure, or some part of it. The programming paradigm refers to the model of programming applications and is characterised in terms of an application’s data-collection feature (event-driven, continuous, or query-based). The interface type, like in mashup tools, refers to the style of the programming interface. The choice of abstraction level, appropriate programming paradigm and applicable interface are said to depend on the specific application requirements, the underlying WSN infrastructure and the abilities of the user who is the developer of the application. 3.3.2.1

Users of WSN Programming Abstractions

The target user for WSN programming abstractions is typically required to have greater technical abilities than those target by WWW mashup tools. Mottola et al. [131, 118] argue that three user types for WSN programming abstractions exist: the ‘domain expert’, the ‘WSN geek’, and the ‘WSN technician’. The domain expert is concerned with data quality and application logic. They view the WSN as a macro-component delivering a service. They are required to have good technical, programming skills but not to be experts in networking. They require WSN programming to be highly abstracted, masking the WSN infrastructure and its inherent complexities. The WSN geek is described as the person who develops software for and deploys WSNs. They are more concerned with collecting vast amounts of data efficiently than the higher-level concern of data quality. They are typically skilled at embedded systems programming and protocol design, and require lower-level abstractions that allow them more control over the underlying WSN infrastructure. The third user, the WSN technician, is a role that does not currently exist but the authors expect will be required for the development of complex systems beyond the abilities of domain experts. Like WWW mashup tools, the higher the level of WSN programming abstraction provided, the lesser the need for the user to have specialised technical abilities. However, even high-level WSN programming abstractions require programming ability on the part of the user. The specific types of programming abstractions found in the research literature are explored in more detail in the following section. 3.3.2.2

Types of Programming Abstractions

In the research literature, programming abstraction models are most often classified in terms of the level (vertical) and type (horizontal) of abstraction supported in terms of developer requirements to manage the underlying WSN infrastructure and their view of the network as a data source. The abstraction level is similar to that described in 56

WWW mashup research (Subsection 3.3.1.2), while the abstraction type adds the notion of semantics regarding the logic or location of sensors. Hadim et al. [66] divide WSN programming abstractions into two main subclasses: global behaviour (macro-programming) and local behaviour. Macro-programming allows the developer to program WSN global behaviour at a high-level specification, typically viewing the WSN as a single entity leaving the individual node programs and configurations to be automatically generated by the supporting middleware. The local behaviour subclass allows WSN application behaviour to be defined based on sensor readings from a specific location, or more generally, nodes are addressed according to the data produced. These categories of programming abstractions are consistent with the categories presented by Sugihara and Gupta [161]. Sugihara and Gupta also describe the goal of macro-programming as to realise programming from a macroscopic viewpoint such that every node and data can be accessed without considering low-level communication between nodes. Sugihara and Gupta further categorise high-level programming models into two sublevels: network-level and group-level abstractions. The authors consider network-level abstractions to be synonymous with macro-programming, in which an entire network is viewed and managed as a single abstract machine. As in [66], group-level abstractions are further classified into two categories of horizontal abstractions: neighbourhood and logical groups. These horizontal abstractions are summarised as follows: Regional abstractions define groups of sensor nodes based on the physical closeness of sensors. These neighbourhood-based groups are typically defined by locality and consist of a node and its neighbours. While membership of nodes to a neighbourhoodbased group is said to be mostly static, mobile and ad-hoc networks require dynamic management of such groups as nodes enter and leave the region. Logical abstractions define groups of sensors nodes that are defined by the commonalities of logical properties between nodes, e.g., node type and sensed modalities. Membership of nodes to logical groups is dynamic when membership is determined by dynamic property values. Laukkarinen et al. [97] survey vertical WSN programming abstractions for application development. Three levels of abstraction are classified from the existing literature: node, network and infrastructure abstractions. Infrastructure abstractions are novel in this research and challenge the macro-programming paradigm, which will be required to integrate heterogeneous network types. The vertical abstraction types are defined as follows: Node abstractions: the resource constrained embedded node hardware and communication protocols are abstracted with node abstraction that executes applications on each physical node 57

Network abstraction: the distributed node network is abstracted from data-interested users. At this level distributed nodes cooperate to provide services such as data access through queries and data-processing in-network through aggregation and fusion Infrastructure abstraction: multiple heterogeneous sensor networks are abstracted behind one interface Relating to WWW mashup research, infrastructure, network and node-level abstractions are considered the WSN equivalent of high, intermediate, and low levels of abstraction respectively. The low level offers the most control but requires the most knowledge of sensor networks. The high level offers the least control but requires the least expertise and aims to provide the ease-of-use appropriate for domain expert developers. The horizontal abstractions (regional and logical types) are not present in WWW mashup research. The inclusion of these abstraction types has the potential to solve the problem relating to diverse application semantics (Subsection 3.3.1), without the need for time-consuming ontology development. The following section reviews the use of WSN programming abstractions by existing offerings in the research literature. 3.3.2.3

WSN Programming Abstractions in Practice

Abdelzaher et al. [1] introduce EnviroTrack, an object-based distributed middleware system that abstracts groups of sensors by logical ‘tracking objects’, which maintain aggregate environmental state. Such objects may be logically attached to moving entities in the physical environment, in order to monitor the state of the tracked entity. Envirosuite [107] extends Envirotrack and is a programming framework that introduces a new paradigm called ‘environmental immersive programming’, to abstract distributed interactions with the environment. Environmentally immersive programming refers to an object-based programming model in which individual objects represent physical elements in the external environment. Envirosuite has been demonstrated using only homogeneous networks and so is categorised as a logical network-level abstraction. A similar approach is adopted in the OASIS middleware and programming framework, which also benefits from a modular, service oriented design. OASIS [94] is an objectcentric, ambient aware sensor network applications development model. It defines the logical elements necessary to support a wide variety of WSN data-flow applications. It also defines a programming framework that facilitates a separation of concerns for application development through a multilayer development process. Application developers do not need any expertise in sensor network programming and use a domain-specific service library to specify global network behaviour, from the perspective of the physical phenomena of interest. Given its SOA design, OASIS supports heterogeneous network types and is considered a regional infrastructure-level abstraction. OASIS also accommodates the 58

integration of Web Services by application developers, although WWW resources are not supported as equal citizens as sensors within the middleware. Mottola et. al. [119] present SPIDEY as a declarative language for specifying logical neighbourhoods in WSNs. A logical neighbourhood includes nearby nodes that satisfy predicates over their static (e.g. type) or dynamic (e.g. sensed values) characteristics. Logical nodes (the application-level representation of a physical node) are defined by node templates created using SPIDEY. The node template specifies which portion of a node’s data and characteristics are made available to the definition of any logical neighbourhood. Similarly, logical neighbourhood is defined by a neighbourhood template, which specifies the predicate (membership function) that determines whether a node belongs to the logical neighbourhood. SPIDEY also supports data aggregation and hierarchical composition of neighbourhoods, enabling their notion of a virtual sensor. For example, a logical neighbourhood that collects and averages the temperature readings from a set of sensors can report the aggregate information as a single reading. SPIDEY has been demonstrated as support heterogeneity and is considered a logical-regional infrastructure-level abstraction. Gummad et. al. [64] describe Kairos’ macro-programming model, which allows developers to write a centralised application in the language of their choice to specify the global behaviour of a distributed sensor network. Kairos provides three constructs that allow programmers to implicitly express distributed data flow and distributed control flow. The ‘node abstraction’ allows programmers to explicitly manipulate nodes and lists of nodes. The ‘one-hop neighbour’ abstraction allows programmers to retrieve the list of one-hop neighbours of a node by simply calling a provided function. The ‘remote data access’ abstraction allows programmers to read from variables at nodes. Kairos has been demonstrated as supporting heterogeneity and also supports hierarchical abstractions through the use of its node abstraction. Kairos is categorised as a regional infrastructure-level abstraction, with support for node-level interactions. While WWW mashup tools maintain flexible interfaces to maximise reusability, they also suffer from the complications users face in terms of the diverse semantics of both underlying services and data and domain-specific applications. WSN programmers faced a similar challenge and it is interesting to note that each of the WSN programming abstractions in practice adopted a hybrid level and type approach to abstraction design. The location and logic of sensors were a natural approach to adding the notion of semantics to sensor network abstractions given the limited functionality of sensors. While it is noted that the semantics of WWW resources are vastly more complicated, it is proposed that regional and logical abstractions may also be applied to simplify the acquisition of WWW context.

59

3.4

Discussion: Informing Design Decisions

This chapter sought to build on Chapter 2 in understanding the requirements that need to be met to realise CPS context-awareness. Chapter 2 introduced the emerging CPS Computing paradigm; the underlying infrastructure it is comprised of (Subsection 2.4.2); the challenges the paradigm faces (Subsection 2.4.1); and the role of a supporting framework to realise CPS context-awareness (Subsection 2.4.3). These requirements were defined further in Section 3.1, with respect to bridging the CPS divide. The key challenges arise from the requirement to treat WWW resources as equal citizens to physical sensors; but to also provide optimised support that leverages the unique features of WWW resources. Existing approaches to bridging the CPS divide were explored in Section 3.2 and were identified as WWW-accessible devices and networks (Subsection 3.2.1); extensions to social networks (Subsection 3.2.2); and extensions to sensor middleware (Subsection 3.2.3). Privacy and security issues pertaining to the WWW resulted in the sensor middleware approach being chosen as an appropriate paradigm for enabling CPS context-awareness. This is not to say, however, that the other approaches do not have a role in CPS Computing. Peer-to-peer CPS social networks have the potential to enable personal social machines as described in Subsection 2.4.2; and WWW-accessible devices and networks will form globally accessible open Sensor Webs. Research offerings in the sensor middleware category were evaluated in Subsection 3.2.4 with respect to the SOA requirements. While each of the offerings met the requirements of the object abstraction layer, the approach adopted by each meant the requirements of the higher-level layers could not be met. A virtual sensor approach to unifying WWW resources and sensor resources, meant these data sources were treated as equal citizens by the middlewares. However, it also meant that the nature of WWW resources were not exploited to provide optimised context-acquisition support. In particular, it is not possible to create and reconfigure virtual sensors at runtime; and as a consequence, higher-level context acquisition services can not be supported. The contributions of this thesis address the former issue in Section 4.2 by extending the virtual sensor approach to accommodate dynamic creation and configuration at runtime. As a result of latter challenge, Web Service mashup tools and WSN programming abstractions were investigated with an aim to revealing approaches that could be applied to CPS context-gathering. WWW mashup tools are characterised by their component models and composition models, which were related to CPS SOA requirements in Subsection 3.3.1.1. While components in WWW mashup tools are typically Web Services or data feeds, for CPS Computing the components will also extend to physical sensors and networks of sensors. A key characteristic of a mashup component is its type, which is either data or application logic. Reflecting on the nature of WWW data and services as described in Section 2.2, a range

60

of data life-cycles are identified that must be considered in the composition of CPS objects. Information can be relatively static or informational; evolve over time; be produced as a consequence of an event; be produced by users; or can be produced on demand by services in response to a query. Web Services also have the ability to perform actions continuously or in response to a request. The discussion on WSN programming abstractions in Subsection 3.3.2 also revealed the need to accommodate both sensors that stream data continuously or based on events; and sensors that respond to individual queries. For CPS context-awareness it is proposed here that each of these data sources be supported and for their orchestration to be designed in line with the natural life-cycle of the data. In support of this goal, sources of WWW data are analysed further in Subsection 4.1.1, informing the specification of the basic building blocks, or component model, for WWW integration in a sensor middleware in Section 4.2 . Both WWW mashup tools and WSN programming abstractions provide varying levels of abstraction in support of development by users with a range of technical abilities. WSN research defines additional types of abstractions adding the power of regional and logical semantics. Adopting a sensor approach to WWW resource monitoring makes the WSN programming abstraction a natural fit to providing optimised CPS context-gathering support. While it is noted that WWW semantics are much broader than those of limited sensor devices, it is proposed that the adoption of this approach will facilitate rapid and comprehensive context acquisition and Pipeline orchestration without the need for time-consuming ontology development. The design of CPS programming abstractions is provided in Section 4.4. The next two chapters describe in detail the design and implementation of the above contributions.

61

Chapter 4

Design of a Supporting Framework for Cyber-Physical-Social Computing Chapter 3 provided a detailed description of related work, informing the design of a supporting framework for Cyber-Physical-Social (CPS) Computing. A sensor middleware approach to bridging the CPS divide was chosen in Subsection 3.2.4 and the key challenges to be addressed by the contributions of this thesis were identified by evaluation of existing frameworks with respect to the key CPS Service Oriented Architecture (SOA) requirements defined in Section 3.1. Existing sensor middleware approaches to bridging the CPS divide integrate World Wide Web (WWW) resources as virtual sensors within the middleware. Virtual sensors represent both physical sensors and WWW resources as equal citizens. While this approach satisfies the need for unified object abstraction, it fails to provide programmatic support for the inclusion of WWW resources in an on-demand and adaptable manner. This is a key requirement for enabling CPS context-awareness and is a pre-requisite to supporting high-level programming abstractions for the specification of complex context acquisition requirements. This chapter describes the design of sensor middleware components that aim to satisfy these requirements. Section 4.1 analyses sources of WWW data to identify the basic building blocks that require integration in a sensor middleware. In addition to these building blocks, an existing context taxonomy is extended to accommodate the findings of this analysis and inform future research in this area. The basic building blocks inform the design of a Cyber Sensor Pipeline approach, described in detail in Section 4.2 and Section 4.3. The Cyber Sensor Pipeline paradigm is influenced by both WWW mashup and WSN programming abstraction techniques. The

62

individual components can be orchestrated to create custom mashups, or Pipelines. A Cyber Sensor Pipeline is also considered equivalent to a node-level abstraction from WSN research. The semantic, horizontal abstractions of WSN research is extended to accommodate a ‘user’ abstraction type; and applied to the Cyber Sensor and Pipe components. This feature facilitates seamless orchestration of Cyber Sensor Pipelines and also accommodates the design of the contributions to follow. Higher-level CPS context acquisition abstractions are inspired by sensor network and infrastructure-level abstractions. As described in Section 4.4, these abstractions provide a single component as a configurable object that produces a homogeneous network, or heterogeneous infrastructure of Cyber Sensor Pipelines. These networks or infrastructures use the previously described components as a basis for creating new Cyber Sensor Pipelines at runtime, as new contexts of interest are discovered.

4.1

Basic WWW Building Blocks for CPS Middleware

A sensor middleware approach to bridging the CPS divide was chosen in Subsection 3.2.4. Prior to the integration of WWW resources with a sensor middleware, the types of WWW data must first be well understood. In Subsection 4.1.1 the unique characteristics of WWW sources of data are analysed. This informs the extension of a context-taxonomy for CPS Computing, in support of the additional challenges imposed by the integration of these data sources with sensor data. Subsection 4.1.2 reviews types of data-processing services typically used in the case of both WWW and WSN research. The resulting taxonomies inform the design of a Cyber Sensor Pipeline paradigm in Section 4.2 and Section 4.3. The Cyber Sensor Pipeline approach is considered equivalent to a node-level programming abstraction from WSN research. It satisfies the object abstraction requirements defined in Section 3.1 and is designed to be extensible in support of higher-level programming abstractions.

4.1.1

Sources of Data

Figure 4.1 illustrates a taxonomy ([121]) of web-based sources of data based on how the data originated. It provides an overview of WWW resources, motivating a discussion on the challenges of incorporating these resources with a sensor middleware. Computational Resources are Web Services that, given a piece of information, perform some computation to provide additional context. The computed information may be static, i.e., it does not change no matter how many times the query is made, or it may be dynamic, i.e., the computed information may change as the same query is made over time. Both static and dynamic computational resources require an

63

Figure 4.1: Taxonomy of WWW-based Sources of Context explicit request for information and are used in an on-demand basis. An example of a static computational resource is a geocoding API, which will always return the appropriate location name given a pair of latitude, longitude coordinates. An example of a dynamic computational resource is a social reputation API that, given a users social profile identifier, will return a reputation metric representing that users computed reputation at that given point in time. The reputation of computational resources is dependent on that of the providers and may or may not hold overall computation types provided by that resource. Static Resources While all web-based resources are considered dynamic, in terms of having the potential to change, the first category is considered to provide static information and relates to websites where the information is, in theory, always valid. Explicitly, this means that the information provided does not change over time. Such information is typically factual or historic (e.g., archives), meaning these resources are useful to services seeking concept definitions or information related to a concept. These resources will not usually require regular monitoring, rather they would be queried for information as a need arises. The reputability of information acquired from static resources will typically depend on the reputability of the resource as the contributor is most often anonymous, or disconnected from any Social Web profile. Some static resources, such as news archives, may have associated user information that can be used in assessing reputability of source. Acquiring data from static resources typically involves web scraping since most static resources do not offer mechanisms for acquiring data, such as RSS Feeds or APIs. Static resources that do offer API access include online dictionary services. Evolutionary resources are the dynamic counterpart of static resources. The data provided is informational in nature but becomes enhanced over time. Examples

64

Table 4.1: Classification of WWW-based Context Sources Type

Definition

Examples

Services that combine context Computational

and application logic to

Klouta, Peer Indexb,

provide more information

Alchemyc

about data Static

Evolutionary

Event driven

Knowledge or fact that

Informative or factual documents,

remains true over time

Web 1.0, Dictionary APIs

Knowledge or fact that changes over time

documents, Web 1.0 sites, Web 2.0 feeds

Autonomous or manual

Weather APIs, Traffic/News feeds,

reporting in response

IoT (e.g., Xivelyd, Thing Speake),

to event or schedule

Sensor Web

User provided data where User driven

Informative or factual

user desire to update

Klout: https://klout.com/

b

Peer Index: http://peerindex.com/

c

Alchemy: http://www.Alchemyapi.com/

d

Xively: https://xively.com/

e

Thing Speak: https://thingspeak.com/

Social Networking platforms, socially enabled Web 2.0

instigates change a

Social Web, including

and a future Web of People

include participatory websites such as Wikipedia1 , websites that describe evolving academic concepts, and e-commerce websites in which prices and items change over time. While resources like Wikipedia are considered socially enabled, the focus is on content and not users, meaning reputation of information is dependent on the resource itself rather than any person. In some cases the contributor may have their social profile connected to a resource enabling some personal reputation information to be used in data integrity profiling. Diverse approaches to data acquisition from evolutionary resources are required including web scraping, data feed consumption and API integration. Event-driven resources are websites, services or applications that consistently provide new information in response to a real world event, either in the physical world, or cyber space. Event types include weather, news, traffic, disaster reporting, and 1

http://wikipedia.org

65

financial stocks. While some resources require web scraping, this category offers many resources that are accessible via a data feed or API. Web of Things (WoT) resources are web portals that provide access to raw sensor streams that are provided by contributors to the Internet of Things (IoT) via an API. Acquiring this data is also realised through an API. Sometimes, raw data is abstracted accommodating metadata in the acquired information, giving the data more meaning for use in an application. The integrity of the information lies in the technical characteristics of the physical sensing device and how reliable it is at reporting values that are an accurate representation of the real phenomena it is sensing. The technical characteristics of the IoT platform used for disseminating data will impact on reliability. For example, server outages would result in no data being acquired despite a fully functioning sensor. If the contributor is identifiable in the Social Web, their reputation may impact the choice of whether to use an IoT sensor. This is due to the ability of contributors to connect simulated sensors to the IoT without the knowledge of the IoT resource itself. Sensor web resources are web portals that provide access to the sensor streams of WSNs. In the context of WWW resources, these sensor streams are accessible via API calls. Removing the WWW requirement, sensor streams from any Sensor Web middleware could be considered Sensor web resources. The integrity of data acquired from event-driven resources is dependent on the reputation of the resource unless the information is explicitly connected to a human reporter via the Social Web. User-driven resources comprise the Social Web and a future Web of People - should it emerge. Resources from other categories whose content is connected to a Social Web profile can be considered hybrid resources. Examples of user-driven resources include social networking platforms such as Twitter and Facebook, blogs such as those on the blogger.com platform and socially connected content from other resource types (e.g., user reviews on the otherwise evolutionary Amazon.com website). While the most widely adopted (in terms of users) platforms offer API access to information, these APIs are often limited in terms of the amount of data that can be consumed (e.g., Twitter Streaming API), the frequency at which information can be requested (i.e., API call rates or limits) and the types of information that are accessible (e.g., at the time of writing, the Google+ API can not provide information for a specified region). The integrity of user-driven information is directly related to the reputation of the user and often that of their immediate social network. As realised from the work of Schneier, [146], there is a key difference between user-provided information and information about a user. The complete context of a users Social Web presence includes all types of data described in Schneier’s taxonomy: service data, disclosed data, entrusted data, incidental data, behavioural data, and derived data. 66

For all of these resource types, the technical challenges of acquiring data must be managed by a supporting framework so that an application developer may concentrate on how to use or respond to information within their application logic. The supporting framework will be responsible for: 1. Unification of diverse data access procedures (API calls, feed consumption, web scraping, etc.) 2. Unification of the diverse data formats from these heterogeneous web resources 3. Ensuring interactions with web resources adhere to constraints set out by API ratelimits 4. Ensure re-use of data from web resources, reducing interactions with the resource, ensuring they are not overloaded through multiple simultaneous applications The web resource descriptions above focus on the origin of data. Each type of resource has the potential to reveal information about context from a number of different perspectives. This is of benefit to Context-Aware Computing that, by its nature, requires as complete as possible a contextual model upon which to make decisions. It also introduces new challenges, however, in terms of building complex context models and weighting information based on their perspectives and reputation. Non-social resources that do not involve physical sensing devices have the potential to reveal information about multiple locations and entities, potentially relating to some period in time. Static resources are more likely to have a historical time component, if any. Time is important in terms of evolutionary resources in which information is likely to become invalid as new information is provided. Event-driven resources are largely time based and can relate to multiple locations, entities and concepts. Time is also of great importance to IoT and Sensor web resources that monitor real world phenomena and whose data becomes stale relatively quickly. Computational resources are more complex again, in that they may provide additional information that would change given a second computation, or remain the same. The work of Rosi et al. who adopt a social sensing paradigm within their middleware framework [139] was included in the earlier literature review (Section 3.2). The authors also proposed the concept of ‘pervasive social context’ [147], similar to the definition of ‘situated social context’ by Endler et al. [51]. Both works emphasis the spatial dimension as being where a social interaction physically takes place. However, this is focussed on use of social sensors and ignores the remote perspective of spatial context, i.e., social sensors that do not report from within a geographic space are ignored, even if they produce important information relating to that space. Further, users are more frequently immersed in software environments such as the Social Web, meaning characteristics of these environments are

67

also of relevance to a users context. Such software environments will become increasingly important to CPS Computing as augmented and virtual reality systems continue to evolve. While developing complex context models is beyond the scope of this thesis, facilitating the development of such models is very much an objective. It is noted that context modelling becomes more challenging in the CPS Computing realm and the discussion here aims to contribute to further research in this domain. A context taxonomy that is adapted from [147] and [51] is offered as STEPP: Spaces Determines the cyber and physical spaces in which context is being produced, or being described by context Times The period in time context data refers to and/or for which it is valid Entities The entities, including conceptual entities, that are described by the context People The named or anonymous individuals or communities described by context and/or producing context Perspective How and why context has been generated - describes the relationship between the context producer and the context information The main aim in providing a new taxonomy is to identify the complexity of context within CPS Computing. In particular, spatial context now refers to perspectives of environments as well as physical phenomena emitted within an environment. Consider, for example, Social Web users reporting about environmental conditions in a region they visited recently, when they are now physically located in their home region. Context representing such a scenario has a remote, personal perspective of a target location that may be indicative of the perception of that place from people from the users location. Scenarios of this nature justify closer inspection through targeted monitoring of the environment from remote perspectives. Such context may also have multiple temporal aspects to it such as ‘now’, last week, and information that originated 10 years, ago but remains valid at this point. In CPS Computing a single piece of web-sourced information may relate to multiple entities, some of which may be conceptual in nature such as a topic of interest. As identified by previous authors, people can both produce and consume context. The perspective dimension in this taxonomy is similar to the information dimension used by [147]. Perspective expands on the originator of data (e.g., evolutionary resource or Sensor Web) by describing its relationship to the context it produced. Its purpose is to provide insight into credibility and intended perception of information by aligning a goal with an information source. A taxonomy identified earlier by [145] lists some interesting goal types that have the potential to form a basis for such a dimension. These include: Informed decision 68

- biased, Informed decision -unbiased, Life enrichment, Online learning, Entertainment, Knowledge. Additional important challenges relevant at this point but that are considered beyond the scope of this thesis include: • Techniques for the management and interrogation of temporal context data • Natural language processing to extract meaning from textual information in a digital, programmable format • Techniques for measuring the accuracy or certainty of information as well as reputation of users and resources • Integrated modelling techniques for creating dynamic contextual models based on acquired real-time data

4.1.2

Data Processing Services

While the previous section focussed on web resources that maintain or compute information of value to applications the focus here is on the functions provided by services that result in added value for applications. This is in keeping with the goals of Web 2.0 mashup tools. In some cases the service types listed below leverage the web resources previously described. Otherwise, the services perform local computation based on required input; resulting in an output or action performed by the service. The main aim in this section is to identify further building blocks of CPS middleware, which have the potential to be integrated into composite applications akin to mashups. The following types of dataprocessing services have been derived from the research efforts described in Subsection 3.3.1 and Subsection 3.3.2. Aggregation Services enable the combination of related information from diverse sources to form an abstracted view of a concept such as those realised in data source mashups and feed aggregation tools. The output of an aggregation service is information from heterogeneous sources combined into a single, unified information object. Filtering Services allow a focussed view of information by isolating information that does, or does not, meet a specified requirement. The output of a filtering service is the ‘interesting’ information. Summary Services consume information and provide an abstracted representation of the information in a single value. Examples of summarised data include the mean, mode, sum, minimum, maximum, etc., of that value with respect to other values for 69

the same property. The output of a summary service is a metric associated with the values. Transformation Services take information in one format and return it in another, supporting interoperation of information from heterogeneous resources, which may not comply to data representation standards or may require a value in an alternate formate. Examples include transforming XML to JSON or a location value from ‘Dublin’ to the coordinates of Dublin, Ireland. The output from a transformation service is the new value for a modality specified as requiring transformation. Viewing Services provide an alternate user interface for the viewing of information. The output of the service is a user interface component that updates according to information it receives in real-time. Interaction Services provide an interface that allows a user to engage with an application through collaboration, content tagging, rating, or reviewing, for example. The output of the service is a user interface component that adapts, or reconfigures, another service based on values the user inputs. Modelling Services build and manage a contextual representation of real-time data in a format that can be queried by applications. While some modelling services use Semantic Web standards such as RDFS and OWL, others use context modelling tools such as JCAF. This is an example of a composite service that relies on a transformation service to convert information to the appropriate format for the model (e.g. RDF) and then updates the context model with that information.

4.2

CPS Sensing: Leveraging the Cyber Sensor Paradigm

Sensor networks have been the focus of a significant research effort in the last decade, primarily due their potential as the key enabling technology in Ubiquitous Computing systems. At first sight, websites, services, applications and platforms that constitute Web 2.0 may seem to have nothing in common with physical sensors that are concerned with physical phenomena in the real world. However, this overlooks the software perspective; in this perspective, a sensor is agnostic in terms of application domain. In principle, a sensor can embrace any data producing entity. In practice, it must be programmed for each data production category. Thus, if sensor networks are considered as an instance of the Internet-of-things (IoT) it can be seen that the sensor principles translates quite well, though not of course completely, from the physical world to cyber space. It must be noted that the discussion so far is consistent with the approach advocated by [139]. However, for the purposes of this thesis, a richer harnessing of the sensor paradigm is required that accommodates all of the web resources previously described. 70

A difficulty commonly experienced by those researching sensor networks concerns how to develop suitable abstractions that encapsulate the inner workings of the sensor and sensor network and, thus, protect end-users from the myriad of issues that essentially belong to the realms of embedded computing and networking technologies. A consensus has emerged in recent years that the middleware paradigm offers a suitable vehicle through which these abstractions are best managed. Thus, the middleware takes care of many of the network management activities in light of end-user policy directives. All that is required is that the software component of the sensor platform be engineered for the category of sensor in question. This component will respond to events For example, changes in temperature, perform preliminary processing on the sensor before transmitting the data. Routing, tagging with appropriate metadata and storing this data is all handled by the middleware framework in a transparent fashion. How the data is subsequently processed and visualised is outside the scope of the core middleware functionality and must be supported through a programming framework that extends and abstracts these core functions. The Context Source taxonomy illustrated in Figure 4.1 not only provides a structured visualisation of how and where data is produced but it illustrates the natural flow of data accessible via the WWW. Whether data is produced by measuring physical elements (using physical sensing devices), web users providing updates, or real-time information disseminated about events as and when they occur, this low-level context data can now be passed through various services, transforming this “low-level” sensed data into abstract knowledge that can be used to drive applications. Cyber sensor is a term already used in fields relating to cyber crime and network security usually relating to the monitoring or detecting of data that identifies a malicious event in cyber space [103, 104]. This data is a form of context. We expand this understanding of Cyber Sensors to include the monitoring of any contextual data in cyber space. As revealed through related research, terms used to described context data gathered from online sources include virtual sensors [69, 85, 2] and social sensors [139, 125, 10]. Virtual sensors in Ubiquitous Computing often refer to a software version of a physical sensor provided by middleware as an abstraction for use by developers of Ubiquitous Computing systems. As was found in Subsection 3.2.4, this term is also used to describe WWW resources integrated in a sensor middleware, without any form of optimisation for context acquisition. For this reason we use the term Cyber Sensor and consider it an extension of the virtual sensor approach. We argue that Social Sensors are in fact a subset of Cyber Sensors, representing user-driven resources, rather than a complete categorisation of web-based context sources [121, 124]. As described in Subsection 2.3.1, physical sensors can be defined as hardware components that have the capability of monitoring the environment they are physically located in. This research offers the following definition for Cyber Sensors:

71

A Cyber Sensor is a (re)taskable software component that monitors the environment it is programmatically connected to, acquiring context related to users, entities, and locations, of relevance to an application. Note the definition states that Cyber Sensors monitor contexts of interest to an application. This is acheived through the configuration, or tasking, or a Cyber Sensor to target specific modalities of interest, for example, a particular user on a social network. Physical sensor networks often provide consistent, and aim for complete, monitoring of physical environments and their constituent entities and users. Intelligent sensor networks enable targeted and aggregated sensing of an environment, perhaps turning on sensors in a particular region in response to a detected event. In terms of application development, related research (Subsection 3.3.2.3) revealed that sensor networks are best accessed by developers through logical and regional abstractions, through node, network and infrastructure-level abstractions. The Cyber Sensor paradigm can be considered a node-level abstraction. Extending the concept of logical and regional abstractions for web resources, the following types of Cyber Sensors can be defined as Location Sensors, Entity Sensors and User Sensors. A Location Sensor monitors the environment it is programmatically connected to, acquiring context related to users, entities and environments from the perspective of a defined location. A location can be a geographical region or a personal space and is defined by its ability to be described by a set of geographical coordinates. An Entity Sensor monitors the environment it is programmatically connected to, acquiring context related to users, entities and environments from the perspective of a defined entity. An entity can be an environmental condition (such as lighting or temperature conditions), a physical object or device in the world, or a concept that represents a phenomenon, event or topic. Entities can be described using keywords that capture their semantic meaning. A User Sensor monitors the environment it is programmatically connected to, acquiring context related to users, entities and environments from the perspective of a defined user. A user is a digital representation of a person made accessible from the environment that maintains a profile portraying some of this persons characteristics and interactions, for example, a Facebook or Twitter profile. A user is described by their unique identifier within a specified environment, typically a user ID or profile name. These categories of Cyber Sensors will enable targeted monitoring of web resources in which information changes over time. Location and Entity sensors will enable a developer

72

to access information from the perspective of their applications interests while User sensors will enable a personal perspective to be obtained from user-driven resources. The Cyber Sensor paradigm is not complete without defining the concepts of a Cyber Sensor Network and a Cyber Sensor Infrastructure. These concepts will enable the provision of network-level and infrastructure-level abstractions as defined in Subsection 3.3.2.3. At this point, it is re-iterated that a Cyber Sensor monitors the WWW environment it is programmatically connected to. A Cyber Sensor Network is then defined as follows: A Cyber Sensor Network is a collection of Cyber Sensors, each monitoring the environment the network is programmatically connected to, acquiring context related to users, entities and locations of interest to an application. The above definition of a Cyber Sensor Network is derived from a homogeneous WSN that is limited to a single sensing technology. In terms of a Cyber Sensor Network, the sensing technology is directly tied to the environment being monitored (e.g., Twitter Application Programming Interface (API) for the Twitter environment). This mapping will be useful in the implementation of a unifying technique, as web resource interactions will need to be managed by a middleware in a similar manner to interactions with a physical sensor network. A Cyber Sensor Network programming abstraction will provide a single programmable interface to the creation and management of a collection of Cyber Sensors. Given Cyber Sensors are software sensors that require creation (rather than discovery as with physical sensors), the middleware will be responsible for creating a collection of Cyber Sensors based on the configuration of the network-level abstraction. As described in Subsection 3.3.2.2, an infrastructure abstraction for physical sensors provides a single interface to “multiple heterogeneous sensor networks”, removing the tie to a particular type of sensing technology and facilitating the development of applications driven by data from any type of network. There is a clear hierarchy in WSN programming abstractions in which an infrastructure is comprised of networks and networks are comprised of nodes. Since Cyber Sensors can be dynamically and individually created at runtime, this approach is adapted to seamlessly facilitate the creation of a single Cyber Sensor, or collections of Cyber Sensors, depending on application requirements. As such, Cyber Sensor Infrastructure is defined as follows: A Cyber Sensor Infrastructure is a collection of Cyber Sensors, each monitoring the environment they are programmatically connected to, acquiring context from each environment related to users, entities and locations of interest to an application. A Cyber Sensor Network can be related to an intelligent sensor network in that its monitoring functions are targeted. Cyber sensors will be created ‘on the fly’ at runtime, 73

meaning that as new entities (including concepts), users, or environments become of interest to an application, a Cyber Sensor Network can be deployed autonomously to monitor and discover related contexts. The same concepts apply to Cyber Sensor Infrastructures where the creation of sensors at run time is not dependent on a particular web resource, but leverages all resources accessible to the infrastructure.

4.3

Cyber Sensor Pipelines: Integrating the Web of Services

At this point, the Cyber Sensor paradigm only accommodates web resources that provide access to real-time (‘live’) information. This is an appropriate mapping to physical sensors but omits valuable information accessible via the emerging Web of Services (WoS). Such resources include ‘static’ and ‘computational’ web resources. Related research (Subsection 3.3.1) found mashup tools to focus on discovery and composition of existing Web Services in order to compose new applications in which added value comes from novelty of the composition. This is closely related to the goals of mashup tools where the aim is to provide simple to use interfaces that support the creation of mashups based on existing Web Services. Similarly, SOA service management is required to provide a discovery service to enable applications access to sensors. A unified middleware discovery service will apply to both sensors and services. Pipeline, or data-flow, programming emerged from Subsection 3.3.1.1 as an approach to integrating data streams and services that is both scalable, extensible and intuitive. Previously, WWW-based data resources and the functions of data-processing services were defined. A Pipe is a software component that performs some function based on real-time data input, and outputs the same data, augmented data, or no data depending on its function. A Pipeline is an ordered series of Pipes. Pipeline programming is a natural approach for data sources and services of this nature and can be accommodated within a middleware through an extension of the Cyber Sensor paradigm. A Pipe can receive sensor data as an input and in turn aggregate additional context based on the sensor data value. The differentiator between the sensor and the Pipe in this scenario is that the sensor stream is continuous, user-driven, or event-driven; while the Pipe responds only when queried on-demand. Using this model offers the potential for sensor resources supporting queries to be modelled as Pipes for integration in sensor Pipelines. The following extended definition of a Cyber Sensor describes its role in the Cyber Sensor Pipeline paradigm: A Cyber Sensor is a (re) taskable component whose configuration determines 74

the context it targets and how it reports information. A Cyber Sensor can be tasked to pass its data through a Pipeline before reporting to an application. Defining this function of Cyber Sensors requires a middleware to manage the integration of static and computational web resources, acting on a sensor stream, requiring the unification of data acquisition protocols, data formats and management of interactions with remote resources in a manner that ensures an application receives data in a timely manner.

4.4

Programming Abstractions for CPS Sensor Networks

There is a clear and purposeful mapping between Cyber Sensors, networks and infrastructures and node, network and infrastructure-level programming abstractions from WSN research described in Subsection 3.3.2.2. Such programming abstractions are described as ‘vertical’ abstractions in that they each provide a higher-level, more abstract view of an underlying infrastructure. Complimentary WSN programming logical and regional programming abstractions are ‘horizontal’ in that the level of abstraction is not impacted but the perspective of data exploration is configured according to semantic context requirements of applications. These horizontal abstractions are accommodated and extended in the Cyber Sensor paradigm through the definition of entity, location and user sensors. It is important for CPS Computing to accommodate the user sensor paradigm to meet the needs of social sensors and user-centered system development identified in related research. While the basic concepts of Cyber Sensors, networks and infrastructures were introduced earlier in this chapter, this section focusses on their use by a developer. The hierarchy of vertical abstractions is similar to that described in Subsection 3.3.2.3 and stated as a requirement for mashup tools in Subsection 3.3.1.2. It is noted that hierarchical WSN programming abstractions opt for node-network or node-infrastructure rather than nodenetwork-infrastructure hierarchy’s. The latter hierarchy is adopted here to allow simplified focus on a single WWW resource (e.g., Twitter), although it is anticipated that future developers of CPS context-aware applications will not require (or want to know) an understanding of the underlying infrastructure. The definition of Cyber Sensors as being (re)taskable sensing software enabling targeted monitoring requires Cyber Sensors to be specifically created and configured for particular sensing tasks. This problem is similar to the challenge of providing programming abstractions for developing applications based on data from WSNs. Manually tasking each individual node is time-consuming and technically challenging, programming abstractions aim to make this process easier resulting in WSN programming tools that are accessible to developers without network engineering skills. This similarity provides an opportunity to develop programming abstractions that enable the creation of CPS monitoring infrastruc75

tures, leveraging all of the data resources previously described to build adaptable sensing infrastructures. Table 4.2: Cyber Sensor Programming Abstractions Vertical Abstractions Node-Level Network-Level Infrastructure-Level

Horizontal (Semantic) Abstractions Logical

Regional

Social

Entity Sensor

Location Sensor

User Sensor

Entity-Focussed

Location-Focussed

User-Focussed

Sensor Network

Sensor Network

Sensor Network

Entity Focussed

Location-Focussed

User-Focussed

Sensor Infrastructure

Sensor Infrastructure

Sensor Infrastructure

As noted by researchers in both WWW mashup (Subsection 3.3.1.2) and WSN Programming Abstraction fields (Subsection 3.3.2.1), the more abstract the programming interface is for a programmer, the easier it is (in theory) for a programmer to work with heterogeneous underlying data sources, but the less flexibility the developer has in terms of what they can achieve. The design of programming abstractions here leverages the Cyber Sensor, network and infrastructure concepts to define programming abstractions, with a clear mapping to WSN programming abstractions, which offer programming alternatives for developers of diverse abilities. These programming abstractions are summarised in Table 4.2 and defined as follows:

Figure 4.2: Developer Perspective of Node Level Abstraction for Cyber Sensor Pipelines

76

Node Level Abstractions

The lowest level abstraction is node-level. They provide

developers with a single point of access to data from an environment from the perspective of an entity, location or user. This is enabled through the provision of Entity and Location sensors for all web resources and User sensors for all Social web resources. The developer interface to such sensors is required to be unified, supporting developer requirement for flexibility but also simplifying sensor configuration through the transformation of complex data formats. For example, allowing a web resource that requires a bounding box to be specified for a location to be monitored using a Location Sensor configured using a location name. This abstraction is illustrated in Figure 4.2. In this example, the developer tasks the abstraction to monitor the sentiment of all “Ebola” entities found in the Twitter Environment. The middleware is then required to produce an Entity Sensor for the Twitter environment and task it to produce information relating to “Ebola”. It then creates a Pipe that will interact with Alchemy to retrieve sentiment information about “Ebola” data as it is acquired by the sensor. This data is then fused together before being sent to the application.

Figure 4.3: Developer Perspective of Network Level Abstraction for Cyber Sensor Pipelines Network-Level Abstractions Network-level abstractions provide developers with a single point of access to data from an environment from the perspective of a defined entity, location or user, as well as additional entities, locations or users discovered to be relevant to an application at runtime. This is illustrated in Figure 4.3, which depicts a use case where a developer configured a network abstraction to monitor the sentiment 77

of all Twitter users related to the entity “Ebola ”. In this example, an Entity Sensor is created to monitor “Ebola” entities on Twitter. This is considered the ‘base sensor’ of the network. For each entity identified, a unique User ID is extracted and used in the generation of a User Sensor to monitor that users presence in the Twitter Environment. As in the previous example, a Sentiment Aggregator Pipe is utilised. In this use case however, a unique Pipe is created for each User Sensor generated. The data sent to the application includes data relating to all users related to “Ebola”, including information regarding the sentiment of each user data.

Figure 4.4: Developer Perspective of Infrastructure Level Abstraction for Cyber Sensor Pipelines Infrastructure-Level Abstractions Infrastructure-level abstractions provide the same functionality as network-Level abstractions without tying the developer to a particular environment. This is illustrated in Figure 4.4 in which the developer wants to monitor the sentiment of Twitter users in hot countries. The base sensor in this example is an Entity Sensor monitoring the entity “hot” in a Weather environment. This Entity sensor produces information relating to all locations that are related to the entity “hot”. From this base sensor, location information is extracted and used in the generation of Twitter Location Sensors to monitor each detected location. From these Twitter Location Sensors, user ID information is extracted and used in the generation of User Sensors for which sentiment information is retrieved providing the application with the data it is interested in. 78

The above Cyber Sensor programming abstractions complete the design of the contributions of this thesis. These design decisions are realised through the implementation of extensions to an existing Sensor Middleware to accommodate diverse WWW resources.

4.5

Discussion

This chapter described the design of middleware components that aim to satisfy the requirements missing from existing sensor middlewares, aiming to bridge the CPS divide. Given the focus on extending a sensor middleware, Section 4.1 analysed WWW data sources and processing services to identify the basic building blocks that require middleware integration. In addition, an existing context taxonomy was extended to accommodate the findings of this analysis, informing future research in this area. The above analysis comprises the second contribution defined in Section 1.6, in turn informing the design of the third and fourth contributions, namely, Cyber Sensors and Pipes. The Cyber Sensor Pipeline approach is described in detail in Section 4.2 and Section 4.3. The Cyber Sensor is considered an extension of the virtual sensor paradigm and supports dynamic creation and configuration of WWW sensors at runtime. While the Cyber Sensor accommodates the integration of real-time data sources, the Pipe enables on-demand acquisition of context from static information sources and Web Services; realtime data-processing and filtering; and on-demand creation and configuration of additional Cyber Sensor Pipelines. The Cyber Sensor Pipeline paradigm is influenced by both WWW mashup and WSN programming abstraction techniques. The individual components can be orchestrated to create custom mashups, or Pipelines. A Cyber Sensor Pipeline is also considered equivalent to a node-level abstraction from WSN research. The semantic (horizontal) abstractions of WSN research are extended to accommodate a ‘user’ abstraction type and applied to the Cyber Sensor and Pipe components. This feature facilitates seamless orchestration of Cyber Sensor Pipelines and also accommodates the design of the contributions to follow. The fifth contribution of this thesis involves higher-level CPS context acquisition abstractions that are inspired by sensor network and infrastructure-level abstractions. As described in Section 4.4, these abstractions provide a single component as a configurable object that produces a homogeneous network, or heterogeneous infrastructure of Cyber Sensor Pipelines. These networks and infrastructures use the previously described components as a basis for creating new Cyber Sensor Pipelines at runtime - as new contexts of interest are discovered. Chapter 5 describes the implementation of these contributions as extensions to a sensor middleware called SIXTH. The SIXTH middleware (introduced in Section 5.1) was influenced by the contributions of this thesis from its conception; and was developed in

79

collaboration with other researchers.

80

Chapter 5

Detailed Design and Implementation The previous chapter described the design of middleware components that aim to satisfy the requirements missing from existing sensor middlewares (described in Section 3.1) aiming to bridge the Cyber-Physical-Social (CPS) divide. This chapter completes contributions three, four and five (Section 1.6) by describing the implementation of these components as extensions to a sensor middleware called SIXTH. The SIXTH middleware is described in Section 5.1; its core features of relevance to this thesis are described; and an evaluation of its features and the extensions designed in Chapter 4, with respect to CPS context-awareness is provided. The Cyber Sensor Pipeline approach was described in detail in Section 4.2 and Section 4.3. A Cyber Sensor is considered an extension of the virtual sensor paradigm and accommodates the integration of real-time data sources. Subsection 5.3.1 completes the third contribution of this thesis by describing the implementation of the extensions to SIXTH that accommodate World Wide Web (WWW) data sources as Cyber Sensors within the sensor middleware. A Pipe enables on-demand acquisition of context from static information sources and Web Services; real-time data-processing and filtering; and on-demand creation and configuration of additional Cyber Sensors and Pipes. The individual components can be orchestrated to create custom Cyber Sensor Pipelines. Subsection 5.3.3 details the implementation of Pipes within SIXTH, completing the fourth contribution of this thesis. A Cyber Sensor Pipeline is also considered equivalent to a node-level abstraction from Wireless Sensor Network (WSN) research. The semantic (horizontal) abstractions of WSN research are extended to accommodate a ‘user’ abstraction type, which is also applied to the Cyber Sensor and Pipe components. This feature facilitates seamless orchestration of Cyber Sensor Pipelines and also accommodates the design of the higher-level abstractions. As described in Section 4.4, these abstractions provide a single component as a 81

configurable object that produces a homogeneous network, or heterogeneous infrastructure of Cyber Sensor Pipelines. These networks and infrastructures use the previously described components as a basis for creating new Cyber Sensor Pipelines at runtime - as new contexts of interest are discovered. The implementation of these high-level programming abstractions is detailed in Subsection 5.3.2, completing the fifth contribution of this thesis. Each of the above contributions serve to achieve the fifth objective of this thesis, to design and implement a supporting framework for CPS Context-Aware Computing (Section 1.5). The sixth objective, to determine the effectiveness of the approach, requires a human friendly interface to each of the contributions to be evaluated. In support of this aim, Subsection 5.3.4 describes the implementation of a JAVA client Application Programming Interface (API) that facilitates programmatic access to SIXTH for the creation of CPS context-gathering applications. This API is used in evaluation of both the functionality (Chapter 6) and the usability (Chapter 7) of the components contributed by this thesis towards the supporting framework.

5.1

SIXTH: Sensor Web Middleware

SIXTH is a Java-based service-oriented Sensor Web middleware built on top of the Open Service Gateway initiative1 component framework, which facilitates the creation of dynamic, modular service platforms. The primary goal of SIXTH is to support the development and deployment of ubiquitous sensing applications. OSGi enables SIXTH to support plug and play interoperability of heterogeneous data sources and services; which are key requirements of a supporting framework for CPS Computing (Subsection 2.4.3). The SIXTH architecture is illustrated in Figure 5.1. A brief overview of the key components of the SIXTH middleware is as follows: • Adaptors: provide seamless interaction with heterogeneous sensor platforms • Sensor Nodes: virtual sensor representations of individual physical devices that may contain multiple sensors monitoring multiple modalities (e.g.,light, temperature, humidity) • Sensor: virtual representation of an individual sensor connected to a physical device • Sensor Data: common data format for representing data from heterogeneous data sources • Receivers: conduits for received data - sensor data, middleware events, re-tasking events etc 1

http://www.osgi.org/

82

Figure 5.1: SIXTH Architecture • Discovery: controls data dissemination, the data retention policy and access to retaskable components The main component of the SIXTH middleware that is of importance to this thesis is the adaptor, the core of which is realised as an OSGi bundle enabling new adaptors to be injected into a SIXTH instance at runtime. Adaptors facilitate the collection of sensor data from heterogeneous WSNs by abstracting and managing interactions with these remote resources. An adaptor provides uniform access mechanisms and ensures interactions adhere to the diverse constraints of heterogeneous data sources. SIXTH also support run83

time re-tasking of sensors to suit dynamic application demands. This is achieved through the sending and receiving of SIXTH re-tasking messages. SIXTH uses a virtual sensor abstraction for representing physical sensors within the middleware. Further, SIXTH adaptors transform heterogeneous data formats into the SIXTH message interface format, facilitating data independence from sensor platform implementation. SIXTH is extensible and application developers build upon, and extend, the common core in the development of the requisite application-specific functionality. To this end, SIXTH supports the provision of user definable data retention policies, custom sensor data representations, and custom sensor node representations. The middleware has been designed as a distributed system, whereby various components of the system are distributed over a network, but are accessed as though operating locally. This is achieved through the use of service discovery and data receiver services provided by the middleware core. The discovery service is a key component of the SIXTH software architecture. It is responsible for providing sensor status notifications to event subscribers. Status changes include sensor arrivals and time-outs. The distributed nature of SIXTH, along with its support for customisation, means the middleware can adhere to custom privacy and security definitions. This flexibility makes SIXTH well positioned to overcome the challenges described in Subsection 2.4.1 and to be leveraged as a supporting framework integrating any of the infrastructures described in Subsection 2.4.2. To limit the workload necessary to create an adaptor, a default implementation has been developed. This implementation handles discovery service registration and maintenance, the creation and management of virtual sensor objects, data access, and sensor data forwarding. SIXTH also provides convenient interfaces and default implementations for its key components. In this thesis, these default implementations and interfaces are extended to facilitate real-time CPS sensing and on-demand data aggregation and processing as described in Section 4.2. The following section extends the initial evaluation of related work in Subsection 3.2.4 to include SIXTH, highlighting its support for CPS context-acquisition as enabled by Cyber Sensor Pipelines, Networks and Infrastructures previously introduced. Details of the implementation of these contributions follows in the remainder of this chapter.

5.1.1

SIXTH Evaluation

Table 5.1 compares the pre-existing features of SIXTH (in the SIXTH column) with related middleware approaches identified previously in Subsection 3.2.4. Extensions to these features that are delivered as part of this thesis are listed in the CPS SIXTH column. The features contributed by this research are designed to meet the unified object abstraction, WWW service management and comprehensive CPS context-acquisition requirements for CPS Computing (Section 3.1); and are developed as extensions to the pre-existing SIXTH 84

Table 5.1: SIXTH evaluation as a CPS Middleware Sensor Middleware Construct

GSN

LSM

SIXTH

CPS SIXTH

WWW Middleware Integration Middleware-WWW

Any language

Integration

& Zero-Conf

Method

for discovery

Wrapper using

Wrapper using

Adaptor using

Cyber Sensor Adaptor,

GSN API

LSM API

SIXTH API

Pipe Adaptor

Virtual Sensor

Virtual Sensor

Virtual Sensor

Virtual Sensor

Cyber Sensor

Data Format

RDF

XML, CSV

RDF

Data Access

SPARQL

Web Service

SPARQL

Object Abstraction Method

SIXTH Sensor Data XML, Web Service

Cyber Sensor Data Java API

WWW Service Management Manual Creation/ Deployment

Zero-Conf

Virtual Sensor

Virtual Sensor

Re-Tasking

discovery

in XML

in XML

in XML

No

No

No

Dynamic Creation/ Configuration

Java API

Re-Tasking

Java API,

in XML

Pipe

WWW Context Acquisition Real-time Stream Processing/ Aggregation

N/A

Virtual Sensor

Visual Pipeline

in XML, SQL

Mashup Interface

No

Abstractions for Dynamic Collections

Cyber Sensor Pipelines Cyber Sensor Network,

No

No

No

of Sensors

No

Cyber Sensor Infrastructure Abstractions

components. 5.1.1.1 SIXTH:

Requirement 1: Unified Object Abstraction The SIXTH method of integrating WWW resources with the middleware is

most similar to the Wrapper approach adopted in GSN and LSM. A SIXTH Adaptor is responsible for interacting with a remote resource (e.g., WSN or WWW resource); and a virtual sensor is the unifying object abstraction representing all WWW and physical sensors within the middleware. While Contruct, GSN and LSM support standard data formats such as RDF and XML, SIXTH adaptors transform heterogeneous data formats into the custom SIXTH sensor data format, facilitating data independence from sensor platform implementation. As a result, SIXTH is capable of accommodating any of the standardised data formats, as well as bespoke data formats that do not adhere to any standard. CPS SIXTH:

The unified object abstraction requirement is met in this thesis by

exploiting the extensibility and flexibility of the SIXTH middleware. The adaptor and virtual sensor components are not just used to accommodate WWW resources as in GSN and LSM, but are extended to accommodate the seamless integration of real-time and on-

85

demand WWW data sources within the sensor middleware as Cyber Sensors and Pipes. SIXTH Sensor Data greatly simplifies the integration of WWW data, which is often diverse in format despite the availability of standard web data formats (as described in Subsection 2.2.2). While SIXTH Sensor Data resolves the complications of integrating sensor and WWW data for CPS context-awareness, in this thesis it is extended to facilitate the development of the Cyber Sensor Pipelines and Cyber Sensor Programming abstractions previously designed. 5.1.1.2 SIXTH:

Requirement 2: WWW Service Management SIXTH provides a re-tasking service that supports runtime adaptivity of sen-

sors and networks. Upon receipt of a re-tasking message, an Adaptor will perform the relevant action, for example configuring the sampling frequency of a sensor. CPS SIXTH:

While Cyber Sensors are treated as equal citizens within the SIXTH

sensor middleware, they are fundamentally different to physical sensors. A key difference that can be exploited by the middleware is their requirement to be explicitly created (rather than discovered). SIXTH re-tasking messages provide a means to programmatically create and re-task Cyber Sensor Pipelines, Networks and Infrastructures at runtime. A feature not supported by other sensor middlewares. 5.1.1.3

Requirement 3: Comprehensive CPS Context Acquisition

CPS SIXTH:

Techniques for comprehensive CPS context acquisition are not provided

by any other middleware. This requirement is met in this thesis by extending the existing features of SIXTH and exploiting its support for dynamic run-time adaptivity through re-tasking messages. 5.1.1.4

Discussion

The SIXTH sensor middleware provides an extensible and flexible architecture that is accommodating of the Cyber Sensor Pipelines, Networks and Infrastructures designed in Chapter 4. The feature for dynamic sensor re-tasking in SIXTH is a key differentiator between it and other sensor middlewares. This feature is exploited to facilitate the extension of SIXTH for CPS context-acquisition, resulting in a feature set that meets the requirements outline in Section 3.1. These features are developed as extensions to the SIXTH components described in Table 5.1. The implementation of the SIXTH Adaptor, Virtual Sensor and Sensor Data is provided in Section 5.2. This is followed with implementation details of the CPS extensions to SIXTH in Section 5.3.

86

5.2

Key SIXTH Components

Owing to its use of OSGi and an inherently extensible design, SIXTH provides an ideal platform for facilitating CPS context-awareness. The Adaptor in SIXTH is the component that enables integration of diverse data sources and services. The Adaptor is introduced in Subsection 5.2.1 and its use for WSNs in SIXTH is described in Subsection 5.2.2. Virtual Sensors, described in Subsection 5.2.3, provide a uniform representation of any type of sensor. Finally SIXTH Sensor Data, described in Subsection 5.2.4, unify the format of diverse WSN data sources. Each of these components are extended to support CPS context-acquisition using the middleware.

5.2.1

IAdaptor

The SIXTH IAdaptor interface defines a baseline for all adaptors supported by the middleware. As illustrated in Figure 5.2, there are two interfaces which extend IAdaptor. The first, ISensorAdaptor, provides a base from which all WSN Adaptors in SIXTH should be developed. The second, IPipeAdaptor, was developed to support the inclusion of Pipe Adaptors for aggregation of data from on-demand WWW resources. Regardless of type, all SIXTH adaptors must mantain an XML description of its characteristics and components (I.e., sensors or Pipes) and must handle the receipt of re-tasking messages.

5.2.2

SIXTH Sensor Adaptor

The Sensor Adaptor is the basis for developing all Adaptors for integrating WSNs in SIXTH. Figure 5.3 illustrates the implementation of the Sensor Adaptor and the SIXTH components it is associated with. The Adaptor leverages the core Discovery, Receiver and Retention Policy services in order to interact with other SIXTH components, produce Virtual Sensors that are discoverable by SIXTH components, receive re-tasking messages with specified commands, and manage the lifetime of Virtual Sensors and Sensor Data within the middleware. The Adaptor manages access to Sensor Data, which is produced by sensors and accessible via the Virtual Sensor for a sensor node. The Virtual Sensor and Sensor Data components are analysed in more detail in Subsection 5.2.3 and Subsection 5.2.4 respectively. A developer implementing a new Sensor Adaptor need only implement a method to act on re-tasking messages and a method to provide information about the capabilities of the Adaptor. The Adaptor description is provided as an XML specification which describes the properties of the various sensors that are supported by the Adaptor. This is illustrated in Figure 5.4, which also depicts the AdaptorInformation class that is responsible for maintaining the sensor specifications as well as information such as the Adaptors name and type. 87

Figure 5.2: UML Class diagram illustrating the IAdaptor, ISensorAdaptor and IPipeAdaptor Interfaces in SIXTH Given each adaptor is required to specify this information in its implementation, this approach is useful in defining the types of Cyber Sensors supported by a Cyber Sensor Adaptor. This approach is also applicable to Pipe Adaptors.

5.2.3

Virtual Sensors

Each SIXTH adaptor forms the interface between a developer and a homogeneous network of sensor nodes. These devices are represented in the middleware as Virtual Sensors implemented using the ISensorNode interface. A physical sensor node is equipped with one

88

Figure 5.3: UML Class Diagram for the AbstractSensorAdaptor in SIXTH

Figure 5.4: SIXTH AdaptorInformation or more sensors that monitors a specific modality such as light, temperature, or humidity. These sensors are represented in SIXTH by the ISensor interface. This relationship

89

Figure 5.5: UML Class Diagram for ISensorNode SIXTH between a Virtual Sensor and other SIXTH components is illustrated in Figure 5.5. Like Adaptors, Virtual Sensors are semantically modelled by a description object. The Virtual Sensor description is extracted from the Adaptor Information and describes the taskable properties of the sensor node. The Virtual Sensor manages data from all of the sensors that are attached to its physical node. Diverse sensor data is unified using the custom SIXTH data format.

5.2.4

SIXTH Sensor Data

Figure 5.6 illustrates the implementation of SIXTH’s custom Sensor Data in the context of a Dummy Sensor Adaptor. The Adaptor simulates sensor readings by producing random numbers. A Virtual Sensor is tasked and its sensor node acquires data and transforms it to the Sensor Data format. This data can then be subscribed to by a receiver via the Adaptor.

90

Figure 5.6: UML Class Diagram for Sensor Data Sensor Data provides a collection of options to provide convenient means for instantiation and for ease-of-use with diverse data formats. Among these is the very flexible option of a Map for managing complex sensor data values. WWW resources often provide a lot of data in response to a single request and this data will easily be unified through its representation as a Map of values, regardless of its original form. This approach will enable data to be reformatted for use by an application that requires data in any specific format. Examples of such data formats include XML2 , JSON3 and SensorML4 .

5.3

Extending SIXTH for Cyber Sensor Pipelines

By providing interfaces and default implementations for Adaptors, Sensors and SensorData, SIXTH provides an ideal sensor middleware for WWW integration. The extensions that form contributions of this thesis were highlighted in Subsection 5.1.1; and the implementation of these extensions to the components described in Section 5.2 is illustrated in Figure 5.7. Real-time WWW data sources are integrated by the Cyber Sensor Adaptor which has been implemented as an extension to the Sensor Adaptor described in Subsection 5.2.2. The Cyber Sensor Adaptor is responsible for the creation and management of Cyber Sensors and Cyber Sensor Data, which are extensions to the original SIXTH Virtual Sensor (described in Subsection 5.2.3) and Sensor Data (described in Subsection 5.2.4) 2

http://www.w3.org/XML/ http://json.org/ 4 http://www.opengeospatial.org/standards/sensorml 3

91

Figure 5.7: CPS extensions to SIXTH core respectively. The implementation of these components are described in more detail in Subsection 5.3.1. The Cyber Sensor Adaptor is also responsible for the management of network and infrastructure-level abstractions, the implementation of which are described in Subsection 5.3.2. These objects form the higher-level abstractions and, based on their configuration, use Pipes to produce new Cyber Sensor Pipelines at runtime. Like Cyber Sensors, Pipes are created and configured by the receipt and processing of re-tasking messages by the adaptor. Pipes can be applied to Cyber Sensors as a property. Cyber Sensors then, maintain a list of Pipes through which it should pass data before the information is sent to an application. The implementation of Pipe Adaptors and Pipes is described in Subsection 5.3.3. In order to evaluate these contributions a Java client API was developed. This API allows a developer to write simple Java code to interact with SIXTH, removing any concern for writing XML configurations or communication with the middleware. The implementation of the API is described in Subsection 5.3.4.

92

5.3.1

Cyber Sensor Adaptor and Cyber Sensors

Figure 5.8: UML Class Diagram illustrating AbstractCyberSensorAdaptor implementation Figure 5.8 illustrates the implementation of the Cyber Sensor Adaptor. As described in Subsection 5.2.2, all Sensor Adaptors in SIXTH accept and respond to re-tasking messages. These XML messages contain a command for instructing the adaptor to perform a particular operation. To facilitate dynamic creation of Cyber Sensors, Networks and Infrastructures at runtime, the Cyber Sensor Adaptor supports the following command types: CreateSensor the application requires a new sensor of a specified type CreateNetwork the application requires a new network with a base sensor of a specified 93

type CreateInfrastructure the application requires a new infrastructure with a base sensor of a specified type ConfigureSensor the application requires a sensor, specified by its unique id, to be re-tasked according to specified property values ConfigureNetwork the application requires a network, specified by its unique id, to be re-tasked according to specified property values ConfigureInfrastructure the application requires an infrastructure, specified by its unique id, to be re-tasked according to specified property values The adaptor performs the required actions by interacting with the appropriate components illustrated in Figure 5.8. Each of these components are described in detail in the remainder of this section. 5.3.1.1

Cyber Sensors

Figure 5.9 illustrates the implementation of a Cyber Sensor, which extends the Virtual Sensor described in Subsection 5.2.3. The Cyber Sensor implementation has been extended further to support Location, Entity and User Cyber Sensors as defined in the design of horizontal node-level abstractions in Section 4.2. A Cyber Sensor maintains a list of properties that can be configured at runtime by sending re-tasking messages to SIXTH. The properties common to all Cyber Sensors are: State:

the current state of the sensor. Permitted values are ‘on’ and ‘off’

Frequency:

the Integer periodic sampling rate using which the sensor is to send data

to an application Pipes:

the list of Pipe ID’s, if any, representing the Pipes that a sensor should pass data

through before sending data to an application Network ID: the network abstraction that this sensor belongs to, if any Infrastructure ID: the infrastructure abstraction that this sensor belongs to, if any The hoorizontal-node-level abstractions define additional properties of relevance to those particular sensor types. The User Cyber Sensor defines the properties user name and user ID so that the sensor can be configured to monitor a user of interest. The Entity Cyber Sensor defines the keyword property so a text value can be used to target an entity of relevance to the application. The Location Cyber Sensor defines the properties 94

Figure 5.9: UML Class Diagram illustrating Node Level Abstraction implementation as Cyber Sensors location name, location latitude and longitude, and location bounding box coordinates. By configuring any of these properties via a re-tasking message, a developer can create a Cyber Sensor that enables targeted monitoring of a WWW environment.

95

Like Virtual Sensors, the properties of a Cyber Sensor are described using a basic XML specification that partially comprises the description of the Adaptor that manages it. A unique default specification is provided for all types of Cyber Sensors currently implemented. For example Listing 5.1 illustrates the specification of a User Cyber Sensor, including its unique user ID and user name properties. In Subsection 5.3.1.2 this specification exploited to provide a flexible means of dynamically creating Cyber Sensors. 1



2

3

4

5

6

7

8

9



Listing 5.1: XML Specification for UserCyberSensor The Sensor Stream is key to enabling the performance of Cyber Sensor Pipelines. A Cyber Sensor uses a Sensor Stream to send data to an application. If a sensors Pipes property has been configured, the Sensor Stream will retrieve the appropriate Pipe(s), pass the data through each Pipe then, if the data has not been filtered out, send the data to the application. The Sensor Stream updates the data to integrate aggregated or calculated information. A record of the actions performed on the data can also be included in support of provenance management. The implementation of Pipes is described in Subsection 5.3.3.

Figure 5.10: UML Class Diagram for Cyber Sensor Data and Data Model As described in Subsection 5.2.4, SIXTH Sensor Data provides a suitable means for representing complex WWW sourced data as it can be formatted in a Map. A parser utility has been developed to support adaptor developers in transforming XML and JSON to a Map object containing key, value pairs formatted as Strings. The key for each entry in the map is a ‘.’ separated String forming a path to the appropriate value which is determined from the original data format. As illustrated in Figure 5.10, the Cyber Sensor 96

Data extension of Sensor Data includes a Data Model, aiming to further simplify data access. The Data Model provides a mapping to the key modalities that have been defined on the basis of the horizontal node-level abstractions. The adaptor sets the following paths, for modalities that exist in the target environment, according to the original data formats of their WWW resource: Entity Path:

The key that will enable the Entity value to be retrieved from the data

Map Location Name Path:

The key that will enable the location name value to be retrieved

from the data Map Latitude Path:

The key that will enable the latitude value to be retrieved from the

data Map Longitude Path:

The key that will enable the longitude value to be retrieved from the

data Map User ID Path:

The key that will enable the user ID value to be retrieved from the data

Map User Name Path:

The key that will enable the user Name value to be retrieved from

the data Map These mappings simplify developer access to the core modalities defined by the horizontal programming abstractions. Access is also unified so developers can access data originating in diverse formats using the same Data Model mappings. Further, the Data Model allows Pipes to retrieve appropriate modality values upon which to act without explicit configuration by the user to point the Pipe to that data in the Map. Further research is required to identify modalities that are (1) of significance to a wide range of CPS environments and (2) of importance to specific application types and domains. 5.3.1.2

Cyber Sensor Factory

The Cyber Sensor Factory is used by the Cyber Sensor Adaptor for the creation of new Cyber Sensors at runtime. The Adaptor sets up the Sensor Factory using the Adaptor Information which contains the unique Cyber Sensor Specifications as described in Subsection 5.3.1.1. The Sensor Factory then, upon request from the Adaptor, will create a Cyber Sensor of a specified type, or a generic Cyber Sensor if no supported types have been specfied. This approach ensures the extended implementation remains flexible and extensible.

97

Figure 5.11: UML Class Diagram illustrating CyberSensorFactory implementation 5.3.1.3

Adaptor Stream

While the Sensor Stream is used to send data from an individual Cyber Sensor to an application, the Adaptor Stream is responsible for interacting with the WWW resource to acquire the data for all Cyber Sensors connected to the Adaptor. The aim in providing the Adaptor Stream is to provide a base implementation that removes common workload involved as a concern for developers of new Cyber Sensor Adaptors. The implementation of the base Adaptor Stream is illustrated in Figure 5.12. Newly created Cyber Sensors are registered by the Cyber Sensor Adaptor with the Adaptor Stream. The Adaptor Stream unifies the demands of all Cyber Sensors belonging to the Adaptor. This is achieved through the management of Aggregate Properties which have been designed to unify the values of individual properties. Specifically, a single Aggregate Property unifies the configured values of all Cyber Sensors for a single Property. For developers of new Cyber Sensor Adaptors it is required to implement the interactions with the target WWW environment. Helper methods are provided so that the developer need only be concerned with getting to the point of using provided utilities to transform data to Cyber Sensor Data format. Once the data is transformed, it is forwarded to the appropriate Cyber Sensors, as derived from the sensor specifications.

5.3.2

Network and Infrastructure Programming Abstractions

The previously described components integrated through the Cyber Sensor Adaptor form the basis for the node-level abstraction as defined in Section 4.3. The high-level network and infrastructure programming abstractions are also managed by the Adaptor. The network-level abstraction enables autonomous Cyber Sensor production driven by realtime data acquired by a base sensor of a specified type. The implementation of this abstraction is illustrated in Figure 5.13. 98

Figure 5.12: UML Class Diagram illustrating AbstractAdaptorStream implementation Upon receipt of a re-tasking message containing a ‘CreateNetwork’ command, the Adaptor extracts the sensor-type information from the message. The type of sensor specified determines the type of base sensor to be created from which the network will be expanded. The Adaptor creates the base sensor, using the Cyber Sensor Factory. The base sensor then, is used for the creation of the Network Abstraction. The abstraction, like a Cyber Sensor, has defined retaskable properties. These properties allow sensor production rules to be configured, for example, the GenerateUsers property accepts values ‘on’ and ‘off’. If the value is set to on, user sensors will be produced as a result of user data being extracted from the base sensor data. A full list of accepted properties can be found in Appendix A.1. The Infrastructure Abstraction operates in a similar manner to the Network Abstrac-

99

Figure 5.13: UML Class Diagram for Network Abstraction

Figure 5.14: UML Class Diagram for Infrastructure Abstraction tion, although its implementation is more complex due to the greater flexibility it offers application developers. An Infrastructure Abstraction is capable of producing Cyber Sensors for any WWW resource while a Network Abstraction is constrained to a single WWW

100

resource. This is reflected in the extensive list of properties supported by the InfrastructureAbstraction as evidenced in Figure 5.14. For example, the GenerateUsers property accepts a value that is the name of the Adaptor from which to generate User Cyber Sensors or the value ‘all’ if all possible User Cyber Sensors are to be generated. If the value is set, user sensors will be produced from the specified network, or all networks, as a result of user data being extracted from the base sensor data. A full list of accepted properties can be found in Appendix A.2. The addition of Network Abstraction and Infrastructure Abstraction as ‘wrappers’ to the Node Abstraction completes the tiered abstraction model as designed in Section 4.4. Throughout the detailed description of these contributions, Pipes have been referred to as being applicable to Cyber Sensors and as being leveraged as dynamic sensor production services. The following section covers the detailed design and implementation of Pipes, starting with a description of the Pipe Adaptor as the SIXTH integration method for on-demand WWW resources.

5.3.3

SIXTH Pipe Adaptor

The Pipe Adaptor, as illustrated in Figure 5.15, provides a base implementation of the IAdaptor interface for the development of Pipe Adaptors in SIXTH. The implementation of Pipe Adaptor bears many similarities to the implementation of the Cyber Sensor Adaptor. However, Pipe Adaptor is not an abstract class, rather it provides an extensible and fully functioning Adaptor that facilitates the production of all types of Pipes that do not require interactions with external resources. Examples of such Pipes include filters and summarisers. These types of Pipes process data through internal functions that exclude or summarise data based on configurable rules and can be created and configured through re-tasking messages interpreted by the Adaptor. Aggregator Pipes on the other hand, connect information retrieved from on-demand WWW resources. Creating a new Pipe Adaptor is similar to creating a new Cyber Sensor Adaptor. The developer simply provides the Adaptor Specification that describes the characteristics of the supported Pipes and implements the method to be performed based on data that passes through the Pipe. Like the Cyber Sensor Adaptor, a Factory is used for the dynamic production of Pipes. 5.3.3.1

Pipe Services

Figure 5.16 illustrates the Pipes that have been developed as part of the Cyber Sensor Pipelines extension to SIXTH. The Pipe types provide diversity in function. The Pipe names provide semantic interconnections between Sensors and Pipes in line with the horizontal (entity, location, user) programming abstractions implemented previously. This

101

Figure 5.15: UML Class Diagram for Pipe Adaptor is enabled by the Data Model component that has been included in Cyber Sensor Data. Basic functionality has been implemented for each of the Pipes illustrated in Figure 5.16. Pipe is the basis for creating all new types of Pipe services in SIXTH. Like a Cyber Sensor, a new Pipe is setup using a Specification which details the type, name and retaskable property types the Pipe will accept. For a generic Pipe, the ‘Modality’ property must be set to specify the key to the specific data value the Pipe should process. The default method for acting on the data simply calls the perform Pipe function method in Pipe Adaptor from which it was created. 1

p u b l i c c l a s s F i l t e r P i p e e x t e n d s Pipe {

2

.

3

@Override

4

p u b l i c I S e n s o r D a t a PipeData ( I S e n s o r D a t a data ) {

5

Map dataMap = data . g e t V a l u e s ( ) ;

6

S t r i n g m o d a l i t y V a l u e = ( dataMap . g e t ( g e t M o d a l i t y P a t h ( ) ) ) . toLowerCase () ;

7

i f ( checkMatches ) {

102

Figure 5.16: UML Class Diagram for IPipeService i f ( m o d a l i t y V a l u e . e q u a l s I g n o r e C a s e ( matches ) ) {

8

meetsThreshold = true ;

9

dataMap . put ( getType ( )+ ’ . ’+getName ( )+ ’ . ’+MATCHES PROPERTY,

10

matches ) ; } e l s e { return null ;}

11

}

12 13

.

14

data . r e p l a c e V a l u e s ( dataMap ) ;

15

// i f

16

r e t u r n s u p e r . PipeData ( data ) ; }

17 18

t h e r e i s more work , a d a p t o r w i l l t a k e c a r e o f i t

}

Listing 5.2: Implementation of FilterPipe.java The development of new Pipe types extending this implementation follows a methodological process and leverages recursive calls to the super class enabling new Pipes to exploit the functionality of the super class while providing custom implementations that extend this functionality. For example, Listing 5.2 illustrates the implementation of a Filter Pipe that extends the basic Pipe to allow data to be filtered out if it does not meet 103

rules that are determined by the configuration of the Pipes’ properties. The properties in this example allow a user to define rules that state a modality value must match, or not match, specified values. The modality path property is inherited from the basic Pipe. The necessity to configure the modality path motivated the design of Pipe abstractions that extend the ‘entity’, ‘location’ and ‘user’ sensor abstractions to the Pipe concept. For each unique Pipe type, an extended version is created to handle these types which are reflected in the Pipe name. For example, an Entity Filter Pipe is an extension to a Filter Pipe that provides additional properties and methods that are relevant to ‘entity’ type modalities. This development pattern repeats for all types of Pipes. A description of each Pipe developed is available in Appendix B.

5.3.4

Developer Interactions with SIXTH

Figure 5.17: SIXTH DiscoveryReceiver The previous sections described the core components of SIXTH and the extensions provided to accommodate Cyber Sensor Pipelines, Networks and Infrastructures with

104

SIXTH. This section focuses on developer interactions with SIXTH in terms of what a developer needs to do to create, configure and access data from the components already introduced. Figure 5.17 illustrates the core SIXTH services a developer must use to discover and access SIXTH Adaptors, Sensors, Pipes and Data. Using these services, an application can keep track of what adaptors are available, send XML re-tasking messages to create and re-task components, and receive notifications and data. Listing 5.3 illustrates a portion of this process. A notification of a successfully created Cyber Sensor is received and an XML re-tasking message is constructed and sent to SIXTH. 1

@Override

2

p u b l i c v o i d r e c e i v e ( ISensorNode node , S t r i n g e v e n t ) { i f ( e v e n t . e q u a l s ( ’NEW’ ) && node . g et S e n s o r T yp e ( ) . e q u a l s ( ’ e n t i t y ’ ) && node .

3

getNetwork ( ) . e q u a l s ( ’ T w i t t e r ’ ) ) { S t r i n g x m l S t r i n g = ’ ’

4 5

+ ’ ’

6

+ ’

1 ’

7

+ ’

e b o l a

’

8

+ ’

on

’ ; A d a p t o r I n f o r m a t i o n a d p a t o r I n f o = node . getAdaptor ( ) . g e t I n f o ( ) ;

9

I D i s c o v e r y d i s c o v e r y = SIXTH . g e t D i s c o v e r y H a n d l e r ( ) . g e t D i s c o v e r y (

10

SIXTH . PASS) ; I R e t a s k i n g S e r v i c e re−t a s k i n g S e r v i c e = d i s c o v e r y . g e t R e t a s k i n g S e r v i c e

11

() ; 12

RetaskingMsg msg = RetaskingMsg . r e t a s k ( x m l S t r i n g ) ;

13

msg . s e t A d a p t o r ( a d p a t o r I n f o ) ; b o o l e a n s u c c e s s = re−t a s k i n g S e r v i c e . r e t a s k ( msg ) ;

14

}

15 16

}

Listing 5.3: Implementation required retrieve the ID of and configure a Twitter Entity Sensor A more complete list of XML messages is provided in Appendix C. Applications that interact with SIXTH in this manner become complex very quickly and it is challenging for any developer to manage the availability of resources, construction of XML messages and sending and receiving of re-tasking messages. This challenge, and the necessity to evaluate the contributions of this thesis, results in the requirement of a client API to simplify developer interactions with SIXTH. A basic client API was designed for the purpose of evaluating the CPS extensions to SIXTH. The design and implementation of this API follows and full Java-docs are for the API are provided in Appendix D.

105

Figure 5.18: UML Class Diagram illustrating the implementation of the SIXTH DiscoveryReceiver in the client API 5.3.4.1

client API

There are three core challenges that need to be removed as concerns for developers and these inform the design of the client API. First, the availability of SIXTH adaptors needs to be managed in a manner that allows developers to develop WWW context-acquisition applications without concern for the underlying middleware and the dynamic nature of Adaptor discovery. Second, programmatic means to creating and configuring Cyber Sensors, Pipes, Networks and Services is required. XML configurations enable flexibility and extensibility within the middleware but does not provide a human friendly interface. Finally, a basic means to accessing data produced by Cyber Sensors is required. To achieve these goals a client side SIXTH Discovery/Receiver utility was implemented. This is illustrated in Figure 5.18. It models currently available Sensor and Pipe adaptors and facilitates the sending of re-tasking messages to SIXTH. This implementation masks the developer from the complexity of working with a dynamic sensor middleware. Programmatic access to creating and configuring SIXTH components is then provided in the AbstractApplication, which leverages the Discovery/Receiver methods to perform all required interactions with SIXTH. It provides a series of factory style methods for creating SIXTH components. As illustrated in Figure 5.19, the create methods return an object of the appropriate type, using which a developer can configure the component as desired, at any point in the execution of their application. 106

Figure 5.19: UML Class Diagram illustrating the Functions of the AbstractApplication class in the client API

Figure 5.20: UML Class Diagram illustrating the Building Blocks for developers using the SIXTH client API Listing 5.4 illustrates the implementation of a Cyber Sensor Pipeline using the API. A Twitter Entity Sensor is configured to monitor ‘Ebola’ and two Pipes are applied to the sensor. The first aggregates reputation information (from Klout) relating to the user that provided the information. The second Pipe, a Filter Pipe, retains only data that 107

Figure 5.21: UML Sequence Diagram illustrating the creation of a Cyber Sensor using the SIXTH client API was provided by a user with a reputation score higher than 20. Line 8, in which the Twitter Entity Sensor is created, is illustrated using a sequence diagram in Figure 5.21. While the sequence diagram displays only processes occurring within the client API and the extensions made to SIXTH core, it illustrates the level of complexity the client API masks for the user. The same can be said for subsequent lines of code as well as the tasks of creating and configuring Network and Infrastructures using the client API. 1

p u b l i c c l a s s App e x t e n d s A b s t r a c t A p p l i c a t i o n {

2

p r i v a t e S e n s o r myTwitterSensor ;

3

p u b l i c App( BundleContext c o n t e x t ) { super ( context ) ;

4 5

}

6

@Override

7

protected void setup ( ) {

8

myTwitterSensor = c r e a t e E n t i t y S e n s o r ( ‘ e n t i t y ’ ) ;

108

myTwitterSensor . t a s k ( ‘ e b o l a ’ ) ;

9 10

Pipe k l o u t P i p e = c r e a t e U s e r A g g r e g a t o r P i p e ( ‘ k l o u t ’ , ‘

11

Twitter KloutScore ’ ) ; 12 13

Pipe f i l t e r P i p e = c r e a t e N u m e r i c F i l t e r P i p e ( ‘ b a s i c ’ ) ;

14

f i l t e r P i p e . task ( ‘ modality ’ , ‘ k l o u t s c o r e ’ ) ;

15

f i l t e r P i p e . t a s k ( ‘ g r e a t e r than ’ , ‘ 2 0 ’ ) ;

16

myTwitterSensor . a p p l y P i p e l i n e ( k l o u t P i p e , f i l t e r P i p e ) ;

17

}

18

@Override

19

p u b l i c v o i d respondToData ( S e n s o r s e n s o r , I S e n s o r D a t a data ) { i f ( s e n s o r . e q u a l s ( myTwitterSensor ) ) {

20

// i know t h i s data i s from a T w i t t e r u s e r with a k l o u t s c o r e

21

h i g h e r than 20 //now i can do something with t h i s i n f o r m a t i o n

22

}

23 24

}

25 26

}

Listing 5.4: Basic template for applications extending AbstractApplication.java

5.4

Discussion

This chapter described the implementations details of the Cyber Sensor Pipeline, Network and Infrastructure which were motivated and designed in subsequent chapters. The contributions were implemented as extensions to the SIXTH sensor middleware which was described in detail in Section 5.1. The core components of SIXTH to be extended included the Adaptor, Virtual Sensor and Sensor Data abstractions. Using these components as the starting point resulted in the development of the Cyber Sensor Adaptor, Cyber Sensor and Cyber Sensor Data as the WWW integration method for SIXTH (see Subsection 5.3.1). In support of data-aggregation, data-processing and dynamic use of middleware services, Pipes were incorporated through the extension of the SIXTH Adaptor to support a Pipe Adaptor (see Subsection 5.3.3). These basic building blocks are the foundation for creating Cyber Sensor Pipelines as defined in Section 4.3. SIXTH, with these extensions, satisfy the ‘unified object abstraction’ and ‘WWW service management’ requirements for CPS context-awareness (Section 3.1). Higher-level programming abstractions were designed (Section 4.4) to meet the third requirement to support ‘comprehensive CPS context acquisition’. These network and infrastructure programming abstractions were implemented as features of the Cyber Sensor Adaptor (see Subsection 5.3.2) and are reusable for any future Cyber Adaptor implementation added 109

to SIXTH. While the middleware comparison in Subsection 5.1.1 provided evidence for SIXTH meeting the requirements for enabling CPS context-awareness (as introduced in Section 3.1), there are a number of aspects relating to the implementation of the middleware extensions that should be considered in future research. The contributions of this thesis have been designed with a developer in mind rather than from a performance perspective. Each of response and processing times as well as the throughput rates will depend on the performance of the individual components comprising a Cyber Sensor Pipeline. Within a single Pipeline, the processing time will be as a direct consequence of the slowest process. For example, a simple Pipeline consisting of a single sensor and a single Pipe. The sensor sends new data every second. The Pipe is constrained to making one API call every five seconds and it takes 500 milliseconds for its process to complete. In this scenario, the Pipe is processing data at a much slower rate than it is being produced and therefore slowing down the response to the application. For some types of WWW data, a few seconds delay is acceptable in terms of data freshness. For real-time data this type of delay will become problematic. Further, for a long-term deployment the impact of such delays needs to be assessed and managed. The horizontal abstractions in which Entity, Location and User Cyber Sensors were developed illustrates the reusability of the core sensor implementation. This ethos pervades the design and implementation decisions, an exemplar of which is the Pipe component. The Pipe component is implemented as a basic shell of a Pipe service that can be extended for the creation of any Pipe type. Its reusability is evident in the implementation of the different types of Pipes including filters and aggregators. These implementations are extended further to implement Pipes that are defined by the type of data they act on, for example the Entity Filter Pipe. The use of these abstractions has the potential to be adopted throughout the middleware to optimise provided services. For example, recommendation of discovered CPS sensors and services based on semantic compatibility; automated orchestration of compatible sensor Pipelines; and replaceability of sensors/services that achieve the same goal to optimise reliability and manage uncertainty. Next, the contributions are evaluated in terms of (1) functional suitability and (2) usability. The functional suitability evaluation is provided as a series of case studies (Chapter 6). The latter part of the evaluation is performed as a usability study (Chapter 7).

110

Chapter 6

Evaluation Part 1: Case Studies Each chapter in this thesis has served to describe the contributions introduced in Section 1.6, which meet the objectives outlined in Section 1.5. The first two objectives required an exploration of related research. This was provided in Chapter 2, which, aiming to define the emerging Cyber-Physical-Social (CPS) context-awareness problem space, investigated existing definitions towards CPS Computing (Section 2.1). Having defined CPS Computing, and identified the technical infrastructures it comprises, these unique infrastructures were investigated further (Section 2.2). The outcome of this research was the design of a middleware enabled unifying CPS infrastructure (Subsection 2.4.2) and forms the first contribution of this thesis. Part of the first contribution included a description of the challenges CPS Computing will face (Subsection 2.4.1), as well as the role and requirements of a CPS enabling middleware (Subsection 2.4.3). The third objective required an investigation of existing approaches to bridging the CPS divide and used the former findings as a basis for critically analysing the research offerings. Identifying sensor middleware integration as the most appropriate methodology, Subsection 3.2.4 evaluated existing middleware frameworks with respect to the previously defined requirements. Not finding an approach that meets all the stated requirements, the fourth objective was to review supporting technologies from both World Wide Web (WWW) and Ubiquitous Computing research. This review of WWW mashup techniques and Wirless Sensor Network (WSN) programming abstractions (Section 3.3), combined with the findings of the earlier research, informed the design of WWW context-source and processing taxonomies (Section 4.1) as the second contribution of this thesis. Focussing specifically on the integration of the existing WWW with a sensor middleware, the fifth objective of this thesis is to design and implement a supporting framework for CPS context-awareness. This is acheived in Chapter 4 and Chapter 5, by the delivery of three further contributions. The first is the design and implementation of support for the dynamic creation and configuration of Cyber Sensors as equal citizens within a sensor 111

middleware (Section 4.2 and Subsection 5.3.1). This is complimented by the contribution of support for the dynamic creation and configuration as Pipes for the realisation of Cyber Sensor Pipelines (Section 4.3 and Subsection 5.3.3). Leveraging Cyber Sensor Pipelines, the penultimate contribution of this thesis is high-level programming abstractions for CPS context acquisition (Section 4.4 and Subsection 5.3.2). The final objective of this thesis is to determine the effectiveness of the designed approach to enabling CPS context-awareness. Each of the technical contributions of this thesis were designed and implemented as part of the SIXTH middleware. The middleware framework evaluation with respect to CPS Computing requirements was updated to include SIXTH (Subsection 5.1.1). The evaluation found that SIXTH, as a result of the contributions of this thesis, met all of the stated requirements. Table 6.1: Functionality of thesis contributions in case study demonstrations WWW Middleware Case Study: Adaptor

WWW

WWW Context

Integration & Management

Extensibility

Cyber Sensor Reuse of

Adaptor

of

entity and location

(Section 6.1)

Adaptor

sensor abstractions

Ubiquitous

Demonstrates

Mapping

heterogeneous

(Section 6.2)

adaptors

Acquisition

Cyber

Pipe

Sensor

(Dynamic

Data

Retasking)

Cyber Pipe (Processing)

Pipe (Aggregation)

Pipeline

Simplifies development of AdaptorStream

Generation

Geo mapping

of UI

simplified by

based on

using

sensor specifications

data model Basic Pipe

Disaster Monitoring (Section 6.3)

Basic Pipe Adaptor for on-demand middleware services

Horizontal abstractions for acquiring context

Geo mapping simplified by using data model

Horizontal

to

abstractions

dynamically

enable

create new

seamless orchestration of

contextually

Cyber Sensor

relevant

Pipelines

sensors Basic Pipe Epidemic

Pipe Adaptor

Monitoring

for aggregation

& Analysis

of context from

(Section 6.4)

on-demand resources

Sensor

Use of

to

Horizontal

Data Model

dynamically

abstractions

to extract

create new

for acquiring context

information from

contextually

received data

relevant

Aggregator Basic Pipe

Pipes to acquire

to summarise

contextually

data in

relevant

real-time

data from Web Services

sensors

Horizontal abstractions enable seamless orchestration of Cyber Sensor Pipelines

Basic Pipes

Autonomous

Pipe Adaptor

Community

for aggregation

Curation

of context from

(Section 6.5

on-demand resources

Basic Pipe

to summarise

Use of

to

data in

Horizontal

Data Model

dynamically

real-time

abstractions

to extract

create new

and

for acquiring context

information from

contextually

to filter out

received data

relevant

contextually

sensors

irrelevant

Aggregator Pipes to acquire contextually relevant data from Web Services

Horizontal abstractions enable seamless orchestration of Cyber Sensor Pipelines

data

This chapter focuses on demonstrating the effectiveness of the basic building blocks as functions for CPS context-acquisition. A series of case studies are described that were designed to provide evidence for the contributions of this thesis, and their alignment, with the requirements of a supporting framework for CPS context-awareness. Table 6.1 provides a roadmap through these case studies and the features they each implement, listed under the requirements each feature was designed to meet. Adaptors (Cyber Sensor 112

Adaptor, Pipe Adaptor, and their extensions); the Cyber Sensor abstraction (and the horizontal location, entity and user abstractions); and Cyber Sensor Data were designed to satisfy the WWW middleware integration requirement. The WWW service management requirement was largely met by SIXTH without extension, but enhanced by the Pipe for dynamic re-tasking. Features satisfying the context-acquisition requirement are Pipes for the processing and aggregation of context; and the Cyber Sensor Pipeline feature which is evidenced through the seamless orchestration of the Cyber Sensor and Pipe components. The extensible and flexible nature of SIXTH enabled its extension for CPS Computing and the first Case Study (Section 6.1) provides evidence that this extensibility has been maintained. Developers simply extend the default implementation, AbstractCyberAdaptor, and use (or extend and use) the Cyber Sensor and Cyber Sensor Data components in their development. The remaining case studies move from a middleware development to middleware use perspective. The second case study, Ubiquitous Mapping (Section 6.2), comprises a simple mapping application that adds markers to a map for the location of every Cyber Sensor produced by the middleware. This is followed by a demonstration of the basic Pipes included in the default Pipe Adaptor in the third case study (Section 6.3). In this case study, new User Cyber Sensors are created to monitor Twitter users who are found contextually relevant to the bush fires in Tasmania. The semantic abstractions are used in this case study to seamlessly orchestrate Cyber Sensor Pipelines. The final two case studies emphasise the usefulness of Pipes for processing data in real-time as well as acquiring contextually relevant data from Web Services. The epidemic monitoring and analysis case study (Section 6.4) uses Cyber Sensor Pipelines to monitor and analyse the reporting of medical epidemics on Twitter and Facebook in realtime. Cyber Sensor Pipelines are setup to acquire illness related information; monitor the users reporting the information; and acquire sentiment context using a Web Service. The Pipelines result in a collection of contextually relevant User Cyber Sensor Pipelines, acquiring sentiment and reputation context relating to the users and their textual updates. The autonomous community curation case study (Section 6.5) uses the very same techniques as in the previous example, but for the purpose of identifying ‘Olympics experts’. These latter two case studies, in particular, demonstrate the full range of basic building blocks and provide evidence for their application to diverse application scenarios.

6.1

Developing an Adaptor to monitor a Web Environment

Listing 6.1 shows the complete implementation of a new Adaptor for monitoring a real-time WWW resource. The Adaptor extends the default implementation, AbstractCyberAdaptor, only requiring a developer to create one new class, an Adaptor stream, and to complete

113

the ‘init’, ‘getSpecification’ and ‘getDataModel’ methods. This use case requires no implementation of the delegate method which is reserved for scenarios where a developer wishes to support sensor types or components other than those provided by the core middleware. 1 2

p u b l i c c l a s s MyAdaptor e x t e n d s AbstractCyberAdaptor {

3

p u b l i c s t a t i c S t r i n g e n t i t y P a t h =‘ data . weather . weatherDesc ’ ;

4

..

5

p r i v a t e s t a t i c DataModel model ;

6

p u b l i c MyAdaptor ( BundleContext c o n t e x t , S t r i n g type ) {

7

s u p e r ( c o n t e x t , type ) ;

8

i n i t ( c o n t e x t , type ) ; }

9 10

@Override

11

p u b l i c v o i d i n i t ( BundleContext c o n t e x t , S t r i n g type ) {

12

t h i s . context = context ;

13

streamAdaptor = new MyAdaptorStream ( t h i s ) ;

14

setupSensorFactory ( t h i s ) ;

15

model = new DataModel ( e n t i t y P a t h , locationNamePath , l a t i t u d e P a t h ,

16

s u p e r . i n i t ( c o n t e x t , type ) ;

longitudePath ) ;

17

}

18

@Override

19

public String getSpecification () {

20

r e t u r n ‘< a d a p t o r D e s c r i p t i o n name=\ ‘ w o r l d w e a t h e r \’> ’+

21

E n t i t y C y b e r S e n s o r . SPEC+L o c a t i o n C y b e r S e n s o r . SPEC+ ‘< output type =\ ‘ t e x t \ ’/> ’ ;

22 23

}

24

@Override

25

p u b l i c v o i d d e l e g a t e ( S t r i n g message ) {}

26

@Override

27

p u b l i c DataModel getDataModel ( ) { r e t u r n model ;

28

}

29 30

}

Listing 6.1: Complete implementation of Adaptor The getSpecification method is required to return an XML String that describes the adaptor and the sensor types it supports. The Application Programming Interface (API) supports queries by location which will facilitate location sensors. The API provides ‘entity’ data in the form of weather keywords (and descriptions). The API does not support querying by keyword so the adaptor will be responsible for abstracting this in order to enable the use of entity sensors with this adaptor. As illustrated in lines 18-23 of Listing 6.1, the adaptor specification can be defined by specifying the adaptor name, output type and including the static specifications of the supported sensor types. Making 114

use of the Entity Sensor and Location Sensor abstractions, relieves a developer of having to develop those components. Line 14 in the sample implementation sets up a sensor factory that makes use of the adaptor specification to determine which objects to use when creating new Cyber Sensors. A DataModel object maintains a path to each of the core semantics of relevance to the sensors. Line 3 of Listing 6.1 illustrates an example path setup for the World Weather environment. Each path is a ‘.’ separated series of keys that can be used to find the appropriate value in the SensorData map. The model is initialised in line 15 and made accessible via the getter method in lines 26-29. Line 13 in Listing 6.1 initialises a new stream adaptor, ‘MyAdaptorStream’, which the developer is responsible for creating. A stream adaptor manages sensor requirements and interactions with the WWW environment. RunnableAbstractAdaptorStream.java is the version of the AdaptorStream that is used when the WWW environment does not already provide access to data as a stream. Extending this default implementation leaves the developer with only the ‘run()’ method to implement. The super class takes care of sensor registration, aggregation of properties and ensuring sensors receive appropriate data. The developer is responsible for performing the required interactions with the World Weather API, based on the aggregate sensor properties. The first aggregate property of concern in the sensor frequency. A simple while loop controls the thread, which starts when any sensor frequency is greater than zero. For each iteration of this loop, the aggregate properties are updated and appropriate interactions with the WWW resource are made. Listing 6.2 illustrates the implementation of this process for a location sensor monitoring the World Weather API. The AggregateProperty object for the location name property (line 2) holds the aggregated values and the IDs of the sensors interested in each value. The aggregated values are then retrieved and a call made to the API to satisfy each unique sensor requirement. The sensor IDs for each value is retrieved from the AggregateProperty object and, using this information, the sensor is retrieved from the sensorMap maintained by the superclass. A CyberSensorData object is constructed including a map of the values acquired; providing metadata about the WWW environment and the sensor, including the DataModel as defined by the adaptor. Finally, the ‘pushDataToSensor(...)’ method is called to allow the abstract class to determine when and if this data should be sent from the sensor. 1

i f ( a g g r e g a t e P r o p e r t i e s . containsKey ( LocationCyberSensor . LOCATION NAME PROPERTY) ) {

2

AggregateProperty locationNames = a g g r e g a t e P r o p e r t i e s . get (

3

Set v a l u e s = l o c a t i o n N a m e s . g e t V a l u e s ( ) ;

4

for ( String string : values ) {

L o c a t i o n C y b e r S e n s o r .LOCATION NAME PROPERTY) ;

5

Map weatherMap = g e t W e a t h e r F o r L o c a t i o n ( s t r i n g ) ;

115

L i s t s e n s o r s = l o c a t i o n N a m e s . g e t S e n s o r s F o r V a l u e ( s t r i n g ) ;

6

for ( Integer integer : sensors ) {

7

L o c a t i o n C y b e r S e n s o r s e n s o r = ( L o c a t i o n C y b e r S e n s o r ) sensorMap

8

. get ( integer ) ; i f ( s e n s o r != n u l l ) {

9 10

CyberSensorData data = new CyberSensorData ( s e n s o r .

11

s e n s o r . g e t S e ns o r T y p e ( ) + ‘: ’+s e n s o r . g e t S e n s e d M o d a l i t y ( ) ,

getNetwork ( ) , s e n s o r . getID ( ) , weatherMap , I S e n s o r D a t a . TYPE PERIODIC, System . c u r r e n t T i m e M i l l i s ( ) , a d a p t o r . getDataModel ( ) ) ; pushDataToSensor ( s e n s o r , data ) ;

12

}

13

}

14

}

15 16

}

17

.

18

.

19 20

private

Map g e t W e a t h e r F o r L o c a t i o n ( S t r i n g locationName ) { S t r i n g u r l = ‘ h t t p : / / a p i . w o r l d w e a t h e r o n l i n e . com/ f r e e / v1 / weather . ashx

21

? q= ’+locationName+ ’&format=xml&num of days=1&i n c l u d e l o c a t i o n=y e s& key=X ’ ; 22

U r l R e q u e s t e r r e q u e s t e r = new U r l R e q u e s t e r ( ) ;

23

S t r i n g r e s p o n s e = r e q u e s t e r . makeRequest ( u r l ) ;

24

Map responseMap = new XmlToMapConverter ( ) . p a r s e (

25

r e t u r n responseMap ;

response ) ;

26

}

Listing 6.2: Performing interactions for configured location sensors The same pattern is followed for the entity sensor although there is extra work to be done to support queries by keyword, as illustrated in Listing 6.3. SIXTH core includes a number of services that can be used by developers, one of which leverages a geocoding API and supports applications in retrieving coordinates for ‘global coverage’ for use cases like this one. The developer simply uses the ‘WorldCoordinates’ class to retrieve a Map of coordinates (centroid of every country). Then, using the same process as above, the developer simply adds in an extra step to query the web API for each country and check if any of the keywords are found in the resulting data. If yes, the appropriate sensors are retrieved, CyberSensorData object constructed and the data pushed to the sensors. If not the data is discarded. 1 2

i f ( a g g r e g a t e P r o p e r t i e s . c o n t a i n s K e y ( E n t i t y C y b e r S e n s o r .KEYWORD PROPERTY) ) {

3

WorldCoordinates world = new WorldCoordinates ( ) ;

4

f i n a l Map c o o r d i n a t e s = world . getMap ( ) ;

116

final

5

A g g r e g a t e P r o p e r t y keywords = a g g r e g a t e P r o p e r t i e s . g e t (

E n t i t y C y b e r S e n s o r .KEYWORD PROPERTY) ; 6

f i n a l Set v a l u e s = keywords . g e t V a l u e s ( ) ;

7

f o r ( S t r i n g country : c o o r d i n a t e s . keySet ( ) ) {

8

// f o r e v e r y c o u n t r y r e q u e s t t h e data

9

Double [ ] c o o r d s = c o o r d i n a t e s . g e t ( c o u n t r y ) ; Map weatherMap = g e t W e a t h e r F o r L o c a t i o n (

10

coords [ 0 ] , coords [ 1 ] ) ; 11

S t r i n g weatherText = weatherMap . g e t ( a d a p t o r . getDataModel ( ) .

12

for ( String string : values ) {

getEntityPath ( ) ) ;

13

s t r i n g = s t r i n g . toLowerCase ( ) ;

14

weatherText = weatherText . toLowerCase ( ) ;

15

i f ( weatherText . c o n t a i n s ( s t r i n g ) ) {

16

L i s t s e n s o r s = keywords . g e t S e n s o r s F o r V a l u e (

17

for ( Integer integer : sensors ) {

string ) ;

18

EntityCyberSensor sensor = ( EntityCyberSensor )

19

i f ( s e n s o r != n u l l ) {

sensorMap . g e t ( i n t e g e r ) ; CyberSensorData data = new CyberSensorData (

20

s e n s o r . getNetwork ( ) , s e n s o r . getID ( ) , s e n s o r . g e t S e ns o r T y p e ( ) + ‘: ’+s e n s o r . g e t S e n s e d M o d a l i t y ( ) , weatherMap , I S e n s o r D a t a . TYPE PERIODIC, System . c u r r e n t T i m e M i l l i s ( ) , a d a p t o r . getDataModel () ) ; pushDataToSensor ( s e n s o r , data ) ;

21

}

22

}

23

}

24

}

25

}

26 27

}

Listing 6.3: Performing interactions for configured entity sensors The remaining implementation is in the initialisation method where the stream adaptor is set up with a reference to the adaptor, as is the sensor factory, which will leverage the adaptor specification defined earlier. Then the DataModel is created using the paths previously defined. Finally the constructor of the super class is called so that the abstracted tasks can be performed. 1 2 3

p u b l i c v o i d s t a r t ( BundleContext bundleContext ) throws E x c e p t i o n { A c t i v a t o r . c o n t e x t = bundleContext ;

117

MyWeatherAdaptor a d a p t o r = new MyWeatherAdaptor ( bundleContext ,

4

‘

worldweather ’ ) ; }

5

Listing 6.4: Creating the new adaptor object in Activator.java Finally, Listing 6.4 illustrates the final requirement of a developer creating a new Cyber Sensor Adaptor. The Activator is responsible for starting the Adaptor. Line 4 in the start method creates the newly developed Adaptor when the bundle is loaded. With this final step complete, the new adaptor can now be used by applications through re-tasking messages or the client API as described in Chapter 5. This case study demonstrated the ease with which a new Adaptor for monitoring a realtime WWW resource can be created. The horizontal sensor node abstractions, Location and Entity Sensors, simplified the development process by removing the need to create additional XML specifications. The DataModel object supported the creation of Entity Sensors for the Location-centric WWW resource by providing a simple means to accessing the entity related information. Creating new Pipe Adaptors for acquiring static WWW data and using Web Services follows a similar process, using the default PipeAdaptor implementation as the basis of new Pipe Adaptors.

6.2

Ubiquitous Mapping Application

Figure 6.1: Ubiquitous Mapping Application The Ubiquitous Mapping application (Figure 6.1) provides a client side user interface for the production, configuration and visualisation of diverse cyber (and physical) sensors 118

in real-time. The application uses Open Street Map

1

to provide a visualisation of the

world and has been built as an Eclipse based application. Three sensor types were selected for demonstrating the support SIXTH provides for heterogeneous data sources. Included are event-driven, user-driven and Internet of Things (IoT) web resources. Yahoo! Weather: Event-driven sensing

For this case study a Yahoo! Weather

adaptor provides a connection between SIXTH and the Yahoo! Weather API. The API allows RSS requests to be made for geographical locations and returns an XML response containing metadata elements describing forecast information for that location. In the SIXTH Yahoo! Weather adaptor additional functionality is provided that allows queries to be made based on a predefined list of weather keywords (chosen based on those specified by the Yahoo! API) and/or weather. Locations can be configured in lat/long, String name or WOEID format and SIXTH manages the conversion of the former two to the latter when required. Functionality has also been included that allows a user to monitor weather for their current location by translating their IP address into a geographical location. These features of the adaptor enable it to support the monitoring of Yahoo! Weather using both Location and Entity Sensors. In the ubiquitous mapping application, when a user selects a node on the map a request is sent to SIXTH to produce a sensor that monitors weather information at that location at a configurable frequency. The SIXTH adaptor then streams the relevant information to the client application making it viewable to the user. Twitter: User driven sensing

SIXTH provides an adaptor for the microblogging

website Twitter to satisfy the user driven aspect of this case study. Twitter provides a real-time API for querying and streaming of user-provided data, i.e., tweets. The API provides functionality allowing queries to be made based on a number of attributes including locations, keywords and user IDs. This makes Twitter a suitable environment for targeted monitoring through Location, Entity and User Sensors. Since this case study is map-based, a requirement of the data sensed from Twitter is that it contains geotagged information. Unless a specific location is specified by a user, not all tweets streamed by SIXTH are guaranteed to have location information. This means that when a user wants to map tweets relating to a keyword, anywhere in the world, the application is required to parse the data received from SIXTH and display only tweets that satisfy the geotag requirement. Not all applications will require geotagged information so rather than only sending geotagged responses to the client SIXTH allows a user to configure a sensor to monitor‘Earth’, which it translates into the bounding box coordinates covering all countries. Since the configuration of the location property for a 1

http://www.openstreetmap.org/

119

sensor requires the response to contain only data provided from within a specific bounding box, this means that all tweets received from SIXTH will contain geotags. When a new Twitter sensor is configured in this case study application, the SIXTH adaptor produces the sensor that performs the required queries and filtering to retrieve the appropriate data. The data is then streamed back to the application where nodes dynamically appear on the map in real-time. Pachube : IoT sensing

2

Xively, formally known as COSM, is an IoT platform that

provides an API facilitating streaming of physical sensor data from personal deployments. These data streams can be made either public or private and are accessible to consumers via APIs or HTTP requests. While the original source of data maintained in these environments is most often physical sensors it is also the case that Cyber Sensor data is provided. An adaptor for this type of data source has been developed for SIXTH and incorporated into the ubiquitous mapping application. IoT WWW environments and physical sensor network deployments differ in relation to the semantic enhancement of the data and access mechanisms. An IoT data provider sends the raw sensor data in tagged semi-structured (usually XML or JSON) feeds. Since the data can be acquired through a web-based interface, a SIXTH cyber adaptor manages API call limits rather than low-level connectivity issues inherent with SIXTH adaptors for direct physical sensor networks. The Ubiquitous Mapping application was initially developed for the mapping of physical sensors within a geographical location. Given Cyber Sensors and Physical Sensors are interacted with using the same SIXTH XML re-tasking messages, the application seamlessly supports the use of both Cyber Sensors and Physical Sensors without requiring any redevelopment. The configuration widgets are generated automatically by the application and are based on the XML description of the Adaptor as described in the previous case study. Again, these are features that are common to both traditional and Cyber Sensors, allowing for seamless support of both by the application.

6.3

Monitoring Real World Disasters: Tasmania 2013

A recurring news story during the development of this thesis was that of a serious outbreak of bush fires in Tasmania. The summer of 2013 was the hottest on record in Tasmania and the potential for fire outbreaks was significant. However, the scale of the bush fires and their longevity suggested the gravity of the situation, including the extensive damage to property as well as the ongoing risk to human health, and, indeed, life. This story was 2

Since

this

case

study

was

developed,

https://xively.com/dev/docs/api/

120

Pachube

has

been

re-branded

as

Xively

widely reported in the international press. However, in the case of a journalist based in the northern hemisphere whose budget would not extend to travelling to Tasmania, and who might like to offer their perspective on the story, the question arises as to how this may be facilitated. One potential approach is to design and deploy a Cyber Sensor Network that would allow the journalist to collate factual (and secondary) information as it related to this ongoing story, and thus put their own perspective on it for their readers. This motivates the development of a use case to illustrate the potential of the Cyber Sensor Pipeline approach in support of this scenario. SIXTH adaptors for the following WWW environments are used in this case study: COSM

3

COSM provides an API that allows IoT sensors to be acquired, based on lo-

cation or keyword based queries. For this scenario COSM is monitored using a Location Sensor that targets Tasmania. This sensor provides access to all publicly available IoT sensors located in Tasmania and made accessibly by COSM. Yahoo! Weather

4

Yahoo! Weather provides access to real-time weather information

for locations queried by a unique identifier. For this use case, a Location Sensor that targets a series of locations in Tasmania is used to reveal current and forecasted weather conditions in the region. Tasmania Bush Fire Alerts

5

Tasmania Fire Service provide a bush fire alert service

through their website. An adaptor was created that uses web scraping tools to access the information and transform it in the SIXTH custom data format. In this use case an Entity Sensor monitors all of this information. Twitter

6

The SIXTH Twitter adaptor supports monitoring of the Twitter environment

through targeted Location, Entity and User Sensors. In this use case Location and Entity Sensors monitor the Tasmania region and bush fire related keywords respectively. From these data streams, User Sensors are created to target the users who are found to be of interest from the former two sensors. The keywords used to configure the Entity Sensor are ‘Tasmania’, ‘fire’, ‘bush fire’ and ‘emergency’. 1

@Override

2

protected void setup ( ) {

3

S t r i n g keywords = ‘ Tasmania ; f i r e ; bush f i r e ; emergency ’ ;

4 5

S e n s o r w e a t h e r S e n s o r = c r e a t e L o c a t i o n S e n s o r ‘ yahooweather ’ ) ;

6

w e a t h e r S e n s o r . t a s k ( ‘ Tasmania ’ ) ;

7

weatherSensor . s t a r t ( ) ;

8 9 10

Sensor f i r e S e n s o r = createEntitySensor ( ‘ Tasmaniabushfire ’ ) ; f i r e S e n s o r . task ( ‘ a l l ’ ) ;

121

fireSensor . start () ;

11 12 13

// monitor u s e r s t w e e t i n g about t h e keywords

14

Sensor TwitterEntitySensor = createEntitySensor ( ‘ Twitter ’ ) ;

15

T w i t t e r E n t i t y S e n s o r . t a s k ( keywords ) ;

16

Pipe e n t i t y U s e r S e n s o r C r e a t o r = c r e a t e E n t i t y P i p e ( ‘ b a s i c ’ , ‘ s e n s o r C r e a t o r ’

17

TwitterEntitySensor . applyPipe ( entityUserSensorCreator ) ;

18

TwitterEntitySensor . s t a r t () ;

, ‘ user ’ ) ;

19 20

// monitor u s e r s t w e e t i n g from t h e l o c a t i o n

21

Sensor TwitterLocationSensor = createLocationSensor ( ‘ Twitter ’ ) ;

22

T w i t t e r L o c a t i o n S e n s o r . t a s k ( ‘ Tasmania ’ ) ;

23

Pipe l o c a t i o n U s e r S e n s o r C r e a t o r = c r e a t e E n t i t y P i p e ( ‘ b a s i c ’ , ‘ sensorCreator ’ , ‘ user ’ ) ;

24

TwitterLocationSensor . applyPipe ( locationUserSensorCreator ) ;

25

TwitterLocationSensor . s t a r t () ;

26 27

S e n s o r cosmSensor = c r e a t e L o c a t i o n S e n s o r ( ‘ cosm ’ ) ;

28

cosmSensor . t a s k ( ‘ Tasmania ’ ) ;

29

cosmSensor . s t a r t ( ) ;

30

launchDataExplorer ( ) ;

31

}

Listing 6.5: Implementation of the Tasmania Bush Fire Case Study using the client API The Java code required to deploy the Cyber Sensor Network (CSN) for this use case using the client API is provided in Listing 6.5. The CSN was left operational for a number of weeks but its behaviour was fine-tuned during this time through the refinement of what key phrases were used. For copyright reasons data was not stored. Figure 6.2 gives a brief collage of the kind of data that the CSN reported upon. The case study demonstrates the dynamic creation of Cyber Sensor Networks by leveraging a basic Pipe for creating Cyber Sensors. For example, the Cyber Sensor Pipeline in lines 14-18 creates a new User Cyber Sensor for every user that can be extracted from the entity data acquired by the Twitter Sensor. This example also demonstrates the use of the horizontal programming abstractions for the seamless orchestration of Cyber Sensor Pipelines. By using the createEntitySensor and the createEntityPipe methods, the Pipeline uses the Data Model to ensure the Pipe acts on the entity information, without any need for the developer to specify its location within the data Map.

6.4

Monitoring and Analysing Medical Epidemics

Social media environments such as Twitter are, by nature, a forum for users to self report on their mental and physical conditions and experiences. This makes such environments 122

Figure 6.2: A collage of the typologies of data sources harvested by the CSN in the Tasmania bush fire study. an ideal platform for real-time analysis of publicly available information with an aim to detect trends in medical conditions and to monitor and predict their spread in real-time. Such information can then be utilised by medical professionals or medical informatics applications, in order to disseminate relevant high quality information, ensure appropriate levels of medications are in stock and for the development of early warning systems that aim to prevent the spread of disease. Monitoring the interactions of social media users who claim a contagious illness may provide insights into contamination and spread of the ailment over time. It can be inferred from users’ interactions online who they may have interacted with offline. While user mentions do not necessarily indicate physical co-location of participants, their geographical information can give insight into the likelihood of offline interactions. Users who 123

adopt location based services such as Twitter geotagging or Facebook check-ins enable fine grained targeting of users based on geographical location and user network. Enabling the fusion of meta-information such as reputation and sentiment to users and text supports the provision of a richer context upon which applications can make decisions. This use case aims to illustrate the ease with which such complex context requirements can be met, through minimal programming, using the SIXTH supported Pipeline programming framework. Research related to the application of this case study include [73, 75, 100, 95]. This case study illustrates the use of Cyber Sensor Pipelines to (1) monitor the regional spread and global sentiment relating to medical conditions and (2) setup targeted monitoring of users found to be reporting a medical condition and users found to be interacting with those users. The complete implementation is split into these two distinct applications. The medical conditions were chosen due to their contagious nature and the potential for such a system to play a role in their detection and monitoring their spread, both locally and globally. These are flu, chicken pox, malaria, cholera and meningitis. This case study was developed before the occurrence of the Ebola epidemic in 2014. Before describing the implementation of the two applications, the SIXTH adaptors used in this case study are described as follows: Twitter Sensor Adaptor The Twitter environment is monitored using entity sensors, which target the defined list of medical conditions. The list is defined for simplicity purposes but could also be generated dynamically by other SIXTH components. Twitter will also be monitored using Location and User sensors which will be generated by the applications. Facebook Sensor Adaptor The Facebook environment is monitored using entity sensors, which target the defined list of medical conditions. User sensors are not generated for Facebook as, at the time of writing, it is not permitted without authentication by each individual user. Alchemy Pipe Adaptor The Alchemy environment is used to aggregate sentiment information relating to the medical conditions and the users found to be related to the medical conditions. Klout Pipe Adaptor The Klout environment is used to monitor the ‘reputation score’ of individual users found to be related to the medical conditions. The two applications implemented as part of this case study are the illness monitoring application and the Twitter user monitoring application. The first application is implemented as an extension of a generic entity monitoring application. This approach to developing new application types is in line with the ethos to maintain reusability of 124

components. EntityMonitoringApplication.java sets up the deployment to monitor any defined entities in both Facebook and Twitter environments. The full code is provided in Listing 6.6. Entity Sensors to monitor the illness keywords are created. Pipes are created and applied to the sensors to aggregate and summarise sentiment data acquired from the Alchemy environment. The sensor creater Pipe is also added to the Twitter Entity Sensor so that Twitter Users are generated for use by the next application. 1

@Override

2

protected void setup ( ) {

3

Sensor facebookSensor = createEntitySensor ( ‘ facebook ’ ) ;

4

Sensor TwitterSensor = createEntitySensor ( ‘ Twitter ’ ) ;

5

String keywordString = ‘ ’ ;

6

f o r ( S t r i n g keyword : keywords ) { k e y w o r d S t r i n g+=keyword + ‘; ’ ;

7 8

}

9

i f ( k e y w o r d S t r i n g . endsWith ( ‘ ; ’ ) ) { k e y w o r d S t r i n g . s u b s t r i n g ( 0 , k e y w o r d S t r i n g . l e n g t h ( ) −1) ;

10 11

}

12 13

f a c e b o o k S e n s o r . t a s k ( k e y w o r d S t r i n g ) ; // t a s k t h e s e n s o r t o monitor a l l o f

14

TwitterSensor . task ( keywordString ) ;

t h e keywords

15 16

// s e t up t h e P i p e l i n e f o r f a c e b o o k

17

Pipe f a c e b o o k S e n t i m e n t P i p e = c r e a t e E n t i t y P i p e ( ‘ Alchemy ’ , ‘ a g g r e g a t o r ’ , ‘

18

Pipe facebookSummaryPipe = c r e a t e E n t i t y P i p e ( ‘ b a s i c ’ , ‘ summary ’ , ‘ ’ ) ;

19

L i s t P i p e l i n e = new A r r a y L i s t () ;

20

P i p e l i n e . add ( f a c e b o o k S e n t i m e n t P i p e ) ;

21

P i p e l i n e . add ( facebookSummaryPipe ) ;

22

facebookSensor . applyPipeLine ( P i p e l i n e ) ;

sentiment ’ ) ;

23 24

// T w i t t e r a l l o w s c r e a t i o n o f u s e r s e n s o r s s o a n o t h e r P i p e l i n e i s required

25

Pipe T w i t t e r S e n t i m e n t P i p e = c r e a t e E n t i t y P i p e ( ‘ Alchemy ’ , ‘ a g g r e g a t o r ’ , ‘ sentiment ’ ) ;

26

Pipe TwitterSummaryPipe = c r e a t e E n t i t y P i p e ( ‘ b a s i c ’ , ‘ summary ’ , ‘ ’ ) ;

27

Pipe u s e r C r e a t o r P i p e = c r e a t e P i p e ( ‘ b a s i c ’ , ‘ s e n s o r c r e a t o r ’ , ‘ u s e r ’ ) ;

28

L i s t T w i t t e r P i p e l i n e = new A r r a y L i s t () ;

29

P i p e l i n e . add ( T w i t t e r S e n t i m e n t P i p e ) ;

30

P i p e l i n e . add ( TwitterSummaryPipe ) ;

31

P i p e l i n e . add ( u s e r C r e a t o r P i p e ) ;

32

TwitterSensor . applyPipeLine ( P i p e l i n e ) ;

33 34

facebookSensor . s t a r t () ;

35

TwitterSensor . s t a r t () ;

125

36

}

Listing 6.6: Implementation of the entity monitoring application using the client API EntityMonitoringApplication.java simply creates an initial deployment that is focussed on expanding context related to specific entities as they are represented on Twitter and Facebook. The second application, TwitterUserMonitoringApplication.java is implemented so that it sets up targeted monitoring of, either, a defined list of users or users who are detected at runtime. This case study leverages the Twitter User Sensors generated by the last application. As illustrated in Listing 6.7 the deployment is re-tasked at runtime as the application receives data from User Sensors. Pipes are created to aggregate reputation score information from the Klout environment and sentiment information from Alchemy. The reputation score is directly related to the user which the sensor targets while the sentiment information relates to individual entities (tweets) produced by the users. The summary Pipes provide a summarised view of the data which can then be leveraged by applications. 1 2

@Override

3

p u b l i c v o i d respondToData ( S e n s o r s e n s o r , I S e n s o r D a t a data ) {

4

i f ( s e n s o r . g et S e n s o r T y pe ( ) . e q u a l s I g n o r e C a s e ( UserCyberSensor .TYPE) ) {

5

// monitor and summarise k l o u t s c o r e s o f each u s e r

6

Pipe k l o u t P i p e = c r e a t e U s e r A g g r e g a t o r P i p e ( ‘ k l o u t ’ , ‘ k l o u t S c o r e ’ ) ;

7

Pipe s e n t i m e n t P i p e = c r e a t e E n t i t y A g g r e g a t o r P i p e ( ‘ Alchemy ’ , ‘ sentiment ’ ) ;

8

Pipe kloutSummaryPipe = createNumericSummaryPipe ( ‘ b a s i c ’ ) ;

9

kloutSummaryPipe . t a s k ( ‘ m o d a l i t y ’ , k l o u t P i p e . getDataModel ( ) . g e t ( ‘ kloutScore ’ ) ) ;

10

// monitor and summaruse t h e s e n t i m e n t o f each u s e r s u p d a t e s

11

Pipe sentimentSummaryPipe = createEntitySummaryPipe ( ‘ b a s i c ’ ) ;

12

sentimentSummaryPipe . t a s k ( ‘ m o d a l i t y ’ , s e n t i m e n t P i p e . getDataModel () . get ( ‘ sentiment ’ ) ) ;

13

// c r e a t e u s e r s e n s o r s f o r u s e r s who i n t e r a c t with t a r g e t u s e r s

14

Pipe u s e r C r e a t o r P i p e = c r e a t e U s e r P i p e ( ‘ b a s i c ’ , ‘ u s e r c r e a t o r ’ , ‘ userfromuser ’ ) ;

15

L i s t P i p e l i n e = new A r r a y L i s t () ;

16

P i p e l i n e . add ( k l o u t P i p e ) ;

17

P i p e l i n e . add ( kloutSummaryPipe ) ;

18

P i p e l i n e . add ( s e n t i m e n t P i p e ) ;

19

P i p e l i n e . add ( sentimentSummaryPipe ) ;

20

P i p e l i n e . add ( u s e r C r e a t o r P i p e ) ;

21 22

sensor . applyPipeLine ( P i p e l i n e ) ;

23

sensor . start () ;

24

126

}

25 26

}

27

Listing 6.7: Implementation of the Twitter user monitoring application using the client API

Figure 6.3: Full view of the real-time mapping application illustrating the distribution of illnesses identified on social media where location information was available. The colours each represent a different illness as follows: Blue=influenza; Orange=pox; Red=meningitis; Green=cholera; Magenta=malaria These applications were implemented and deployed using no database in order to demonstrate the platforms support for developing fully real-time applications to epidemic detection and monitoring using social media. Figure 6.3 illustrates the sensor data acquired by the first application using a mapping application. The distribution of detected illnesses can be seen and localized clusters of related illnesses are easily identifiable such as that of meningitis in Chile. The mapping application also facilitates closer analysis through zooming of the map, which is illustrated in Figure 6.5. Clusters of flu mentions are visible in Indonesia and surrounding areas. Figure 6.4 shows a real time analysis charting application where the number of illness related updates and created user sensors are shown in real-time. There is also a view into anonymised data as it is detected in real-time and the information acquired from the Alchemy API provides additional insight into the updates relating to influenza specifically. This case study introduced the use of Aggregator Pipes for the runtime acquisition of contextually relevant data from Web Services. The case study demonstrates the ease 127

Figure 6.4: Full view of the real-time analysis application which updates the values as new mentions or unique users are identified and illustrates the percentage of sentiment distribution for influenza related updates as provided by the Alchemy API with which these new context sources can be integrated in an application and the benefit of using the DataModel of the CyberSensorData object for accessing sensed data, rather than managing the diverse data formats manually. Like in earlier examples, this case study also demonstrates the power and flexibility of Cyber Sensor Pipelines for identifying and accessing new contextually relevant data sources. Focussing on the Alchemy data potential use cases are considered for the information aggregated through SIXTH Pipelines. Prior to any natural language processing an application would not have any understanding of the context in which an update was provided by a user. By aggregating sentiment information from Alchemy an application has richer context from which they can infer potential reasons for the update. A positive sentiment may indicate the end of an illness or that an illness is improving. A negative sentiment may indicate the beginning of an illness or that symptoms are worsening. However, an update that contains a URL is likely to be information sharing rather than self reporting, regardless of the sentiment attached to the update. The application of a single Pipe to the sensors has provided a much richer context upon which an application may reason about the data in real-time.

128

Figure 6.5: Map view zoomed in to illustrate illness distribution in Indonesia and surrounding islands

6.5

Autonomous Community Curation

For this case study an autonomous community curation application developed using SIXTH. It’s aim is to autonomously curate a community of experts relating to the Olympics, which is a global sporting event that took place in London in 2012 7 . The application aims to provide a curation service that autonomously discovers Twitter users who will provide public updates relating to a particular event, in this instance, the Olympics. Research related to this case study includes [55, 102, 76, 74]. For the purpose of implementing this use case, a definition of what is meant by ‘expert’ is required. For this use case, an expert is defined as a Twitter user that reliably provides accurate information relating to the Olympics in 2012. The expert is reliable as determined by the consistency with which they tweet (a measurement of the users tweeting frequency) and is reputable among their peers (as evidenced by a Klout score). This definition informs the SIXTH adaptors required for use in this case study which are described as follows: Twitter Sensor Adaptor Twitter will be monitored using Entity and Location Sensors that target Olympic related keywords and the location of London respectively. The resulting data will be monitored and the application will decide based on a Pipeline when potential experts have been found. In this case, Twitter User Sensors will be created to enable the monitoring of these potential experts. 7

This case study was developed prior to the London 2012 Olympics

129

Klout Adaptor Klout will be used to aggregate reputation scores for each of the users detected by the sensors monitoring Twitter. An additional service provided by Klout enables the aggregation of topics that the user is considered to be reputable in relation to. This will be leveraged to determine the relevance of an expert to the application domain. This application is composed of two sub-applications. The first focusses on production of user sensors that have the potential to meet the ‘expert’ requirements previously defined. The second application enables targeted monitoring of these users. The implementation of TwitterEntityExpertCurator is provided in Listing 6.8. A Twitter Entity Sensor is created and tasked to monitor the defined keywords. A Pipeline is created that starts by aggregating Klout topic information related to the user detected by the entity sensor. The next Pipe filters the data based on the relevance of the Klout topics, in this case only keeping data for which the topic ‘Olympics’ is included. Then a user sensor is created for users whose topics are of relevance. Lines 20 - 26 extend the Pipeline to focus on additional users whose Twitter names are found appearing in the filtered stream. If a connected user is detected their Klout topics are aggregated and if those topics are found to contain ‘Olympics’ an additional Twitter User Sensor is created to monitor that user in real-time. This extended Cyber Sensor Pipeline optimises the potential community of experts. 1 2

@Override

3

protected void setup ( ) {

4

String topicString = ‘ ’ ;

5

for ( String topic : topics ) { t o p i c S t r i n g+=t o p i c + ‘; ’ ;

6 7

}

8

i f ( t o p i c S t r i n g . endsWith ( ‘ ; ’ ) ) ;

9

t o p i c S t r i n g = t o p i c S t r i n g . s u b s t r i n g ( 0 , t o p i c S t r i n g . l e n g t h ( ) −1) ;

10 11

Sensor TwitterSensor = createEntitySensor ( ‘ Twitter ’ ) ;

12

TwitterSensor . task ( topicString ) ;

13

Pipe u s e r K l o u t T o p i c s = c r e a t e U s e r A g g r e g a t o r P i p e ( ‘ k l o u t ’ , ‘

14

Pipe t o p i c F i l t e r = c r e a t e E n t i t y F i l t e r P i p e ( ‘ b a s i c ’ ) ;

15

t o p i c F i l t e r . task ( ‘ modality ’ , ‘ k l o u t t o p i c s ’ ) ;

16

t o p i c F i l t e r . t a s k ( ‘ m o d a l i t y c o n t a i n s ’ , ‘ Olympics ’ ) ;

17

Pipe u s e r C r e a t o r P i p e = c r e a t e P i p e ( ‘ b a s i c ’ , ‘ s e n s o r c r e a t o r ’ , ‘ u s e r ’ ) ;

k l o u t t o p i c s ’ ) ; // w i l l g e t t h e u s e r from t h e datamodel

// f o r c r e a t i n g s e n s o r s t o monitor u s e r s with a p p r o p r i a t e k l o u t topics 18 19

// i f t h e r e a r e any i n t e r a c t i o n s between u s e r s check t h e i r k l o u t t o p i c s too

130

Pipe c o n n e c t e d U s e r K l o u t T o p i c s = c r e a t e U s e r A g g r e g a t o r P i p e ( ‘ k l o u t ’ , ‘

20

k l o u t t o p i c s ’ ) ; // w i l l g e t t h e u s e r from t h e datamodel c o n n e c t e d U s e r K l o u t T o p i c s . t a s k ( ‘ m o d a l i t y ’ , ‘ T w i t t e r . mentions .

21

screenname ’ ) ; 22

Pipe c o n n e c t e d U s e r T o p i c F i l t e r = c r e a t e E n t i t y F i l t e r P i p e ( ‘ b a s i c ’ ) ;

23

connectedUserTopicFilter . task ( ‘ modality ’ , ‘ k l o u t t o p i c s ’ ) ;

24

c o n n e c t e d U s e r T o p i c F i l t e r . t a s k ( ‘ m o d a l i t y c o n t a i n s ’ , ‘ Olympics ’ ) ;

25

Pipe c o n n e c t e d U s e r C r e a t o r P i p e = c r e a t e P i p e ( ‘ b a s i c ’ , ‘ s e n s o r c r e a t o r ’ , ‘ u s e r ’ ) ; // f o r c r e a t i n g s e n s o r s t o monitor u s e r s with appropriate klout topics c o n n e c t e d U s e r C r e a t o r P i p e . t a s k ( ‘ m o d a l i t y ’ , ‘ T w i t t e r . mentions . i d ’ ) ;

26 27 28

L i s t P i p e l i n e = new A r r a y L i s t () ;

29

P i p e l i n e . add ( u s e r K l o u t T o p i c s ) ;

30

P i p e l i n e . add ( t o p i c F i l t e r ) ;

31

P i p e l i n e . add ( u s e r C r e a t o r P i p e ) ;

32

P i p e l i n e . add ( c o n n e c t e d U s e r K l o u t T o p i c s ) ;

33

P i p e l i n e . add ( c o n n e c t e d U s e r T o p i c F i l t e r ) ;

34

P i p e l i n e . add ( c o n n e c t e d U s e r C r e a t o r P i p e ) ;

35

TwitterSensor . applyPipeLine ( P i p e l i n e ) ; TwitterSensor . s t a r t () ;

36 37

}

Listing 6.8: Implementation of the Twitter entity expert curation application The second application leverages the deployment that was set up in the curation application. As illustrated in Listing 6.9, User Sensors are detected at runtime and a Pipeline is applied that aggregates and summarises Klout score and topic and Alchemy sentiment information. 1

@Override

2

p u b l i c v o i d respondToData ( S e n s o r s e n s o r , I S e n s o r D a t a data ) {

3

i f ( s e n s o r . g et S e n s o r T y pe ( ) . e q u a l s I g n o r e C a s e ( UserCyberSensor .TYPE) ) {

4

// monitor and summarise k l o u t s c o r e s o f each u s e r

5

Pipe k l o u t S c o r e P i p e = c r e a t e U s e r A g g r e g a t o r P i p e ( ‘ k l o u t ’ , ‘ k l o u t S c o r e ’ );

6

Pipe k l o u t T o p i c P i p e = c r e a t e U s e r A g g r e g a t o r P i p e ( ‘ k l o u t ’ , ‘ k l o u t t o p i c ’ );

7

Pipe tweetSummaryPipe = createEntitySummaryPipe ( ‘ b a s i c ’ ) ;

8

Pipe kloutSummaryPipe = createNumericSummaryPipe ( ‘ b a s i c ’ ) ;

9

kloutSummaryPipe . t a s k ( ‘ m o d a l i t y ’ , k l o u t S c o r e P i p e . getDataModel ( ) . g e t ( ‘ kloutScore ’ ) ) ;

10 11

// f i l t e r out t h e t w e e t s t h a t a r e not r e l e v a n t i n terms o f e x p e r t i s e

12

Pipe t w e e t F i l t e r P i p e = c r e a t e E n t i t y F i l t e r P i p e ( ‘ b a s i c ’ ) ;

13

tweetFilterPipe . task ( ‘ modalitycontainedin ’ , kloutScorePipe . getDataModel ( ) . g e t ( ‘ k l o u t o p i c ’ ) ) ;

14

131

15

// monitor t h e s e n t i m e n t o f r e l e v a n t t w e e t s

16

Pipe s e n t i m e n t P i p e = c r e a t e E n t i t y A g g r e g a t o r P i p e ( ‘ Alchemy ’ , ‘

17

Pipe sentimentSummaryPipe = createEntitySummaryPipe ( ‘ b a s i c ’ ) ;

18

sentimentSummaryPipe . t a s k ( ‘ m o d a l i t y ’ , s e n t i m e n t P i p e . getDataModel ( ) .

sentiment ’ ) ;

get ( ‘ sentiment ’ ) ) ; 19

L i s t P i p e l i n e = new A r r a y L i s t () ;

20

P i p e l i n e . add ( k l o u t S c o r e P i p e ) ;

21

P i p e l i n e . add ( k l o u t T o p i c P i p e ) ;

22

P i p e l i n e . add ( tweetSummaryPipe ) ;

23

P i p e l i n e . add ( kloutSummaryPipe ) ;

24

P i p e l i n e . add ( t w e e t F i l t e r P i p e ) ;

25

P i p e l i n e . add ( s e n t i m e n t P i p e ) ;

26

P i p e l i n e . add ( sentimentSummaryPipe ) ;

27

sensor . applyPipeLine ( P i p e l i n e ) ;

28

sensor . start () ;

29

}

30 31

}

Listing 6.9: Implementation of the Twitter entity expert curation application This case study uses a similar feature set to the Epidemic Monitoring and Analysis case study, yet serves an entirely dissimilar use case. This case study demonstrates an additional feature in the use of a basic Pipe for the filtering out of contextually irrelevant (or undesirable information). Building on the experience of the previous case studies, the Cyber Sensor Pipeline paradigm remains a powerful and flexible approach to generating Cyber Sensor Networks for context acquisition.

6.6

Discussion

This chapter described a series of case studies designed to provide evidence for the contributions of this thesis and their alignment with the requirements of a supporting framework for CPS context-awareness. The features being demonstrated and their evidence within the individual case studies were summarised inTable 6.1. WWW Middleware Integration contributions include the Adaptors (CyberSensorAdaptor, PipeAdaptor and their extensions), the Cyber Sensor abstraction (and the horizontal location, entity and user abstractions), CyberSensorData and its basic data mode. The Pipe for dynamic re-tasking is part of the WWW Service Management category. WWW context-acquisition features include Pipes for the processing and aggregation of context. The Cyber Sensor Pipeline feature is evidenced through the seamless orchestration of the Cyber Sensor and Pipe components. Five case studies were described in this chapter providing evidence for SIXTH meet132

ing the requirements stated in Subsection 5.1.1, as well as for the flexibility of the Cyber Sensor Pipeline paradigm in solving a range of diverse problems. The first case study (Section 6.1) provides evidence that the extensibility of SIXTH’s core has been maintained and demonstrates the ease with which the middleware can be extended to monitor new WWW resources. Developers simply extend the default implementation, AbstractCyberAdaptor, and use (or extend and use) the Cyber Sensor and Cyber Sensor Data components in their development. The second case study, Ubiquitous Mapping (Section 6.2), comprised a simple mapping application that adds markers to a map for the location of every Cyber Sensor produced by the middleware. This was followed by a demonstration of the basic Pipes included in the default Pipe Adaptor in the third case study (Section 6.3). In this case study, new User Cyber Sensors were created to monitor Twitter users who were found contextually relevant to the bush fires in Tasmania. The horizontal abstractions were used in this case study to seamlessly orchestrate Cyber Sensor Pipelines. The final two case studies emphasised the use of Pipes for processing data in real-time as well as acquiring contextually relevant data from Web Services. The epidemic monitoring and analysis case study (Section 6.4) used Cyber Sensor Pipelines to monitor and analyse the reporting of medical epidemics on Twitter and Facebook in real-time. Cyber Sensor Pipelines were setup to acquire illness related information; monitor the users reporting the information; and acquire sentiment context using a Web Service. The Pipelines resulted in a collection of contextually relevant User Cyber Sensor Pipelines, acquiring sentiment and reputation context relating to the users and their textual updates. The autonomous community curation case study (Section 6.5) used the very same techniques as in the previous example, but for the purpose of identifying ‘Olympics experts’. These latter two case studies, in particular, demonstrated the full range of basic building blocks and provided evidence for their application to diverse application scenarios. The more complex case studies described in this chapter were chosen based on literature leveraging WWW data and are unique in their development using a sensor middleware, as opposed to directly using the APIs. While good practice in developing Service Oriented Architectures comprising modular and reusable components would reduce the lines of code required to develop such use cases. Even without SIXTH, the sensor middleware approach brings a number of benefits; using sensor middleware techniques for the development of WWW data driven applications simplifies the application of sensor network management techniques to the WWW. For example, the application of sensor re-tasking messages to Cyber Sensors resulted in a powerful method for creating new Cyber Sensor Pipelines on the fly; the application of sensor discovery techniques to the task of Web Service discovery has potential in future research; as does the application of uncertainty management techniques to CPS data.

133

While the case studies described in this chapter were developed as demonstrators of the functionality of SIXTH for CPS Computing, they also provide insight into additional features or challenges that need to be addressed (but are beyond the scope of this thesis). For example, the natural language processing (NLP) services used in the case studies worked well only when a relatively large amount of text was provided. NLP for real-time streams is required to work effectively with short bursts of texts. Alternatively, Cyber Sensor Pipelines could aggregate context into ‘single value’ upon which the NLP would act. Further, SIXTH has the potential to form a platform for the evaluation of such services. During the deployment of each case study it became clear that optimised WWW resource management is required. While a simple method is used in AbstractCyberAdaptor and PipeAdaptor that ensures API calls stay within the limits determined by the provider, the response used by each provider, and their consistency with respect to API call allocations, can result in the failure of this method. Further, the performance of Cyber Sensor Pipelines is dependent on each of its constituent components and the management of the inherent complexities warrants further research. It is evident in the running of an application that the timeliness of data is directly impacted by the slowest data source, or service with most limited API call allocation. While the features of the contributions of this thesis have been validated through these case studies, the usability of the programming abstractions from the perspective of developers must be evaluated. This is achieved through a usability study as described in Chapter 7.

134

Chapter 7

Evaluation Part 2: Usability Studies The core contributions of this thesis include the design and implementation of middleware and programming support for Cyber-Physical-Social (CPS) context gathering. These contributions include the development of adaptors for the integration of World Wide Web (WWW) resources as Cyber Sensors(Section 4.2 and Subsection 5.3.1) and Pipes (Section 4.3 and Subsection 5.3.3) in the SIXTH middleware. These basic building blocks enable the flexible Cyber Sensor Pipeline approach to CPS context acquisition and satisfy the unified object abstraction and WWW service management requirements defined in Section 3.1. The functionality of these components is validated in Chapter 6 through a series of case studies. Each case study provides evidence that the features summarised in Table 5.1, are useful for acquiring context in practical CPS context-aware application development. These case studies formed the first part of the evaluation of the contributions of this thesis. This chapter extends the case study evaluation to include the higher-level programming abstractions (Section 4.4 and Subsection 5.3.2) which aim to provide an intuitive approach to comprehensive WWW context gathering using a Wireless sensor network (WSN) programming abstraction approach. Specifically, this chapter will evaluate the usability of the following contributions for the purpose of specifying and acquiring context from WWW resources: Cyber Sensor Pipelines: These basic building blocks form the lowest level (nodelevel) programming abstraction for acquiring CPS context. While the flexibility of these components in meeting the complex and adaptive contextual requirements of applications was evidenced in Chapter 6, in this chapter they are evaluated from a usability perspective. Specifically, the evaluation aims to determine if developers find the abstraction an intuitive means to acquiring WWW context. 135

Cyber sensor network and Infrastructure Programming Abstractions: These higher-level programming abstractions aim to provide a means for acquiring comprehensive context based on complex application requirements, in an intuitive way and with minimal effort. The network-level abstraction supports acquisition of context from a single data source (e.g. Twitter) using multiple Cyber Sensors, while the infrastructure-level abstraction enables monitoring of heterogeneous data sources (e.g. Twitter and a Weather Application Programming Interface (API)). The goal of this evaluation is to determine if developers are indeed able to acquire comprehensive context using these abstractions and if the approach is found to be an intuitive means for doing so. Evaluating whether these goals have been achieved by the provision of these contributions requires a usability study that provides insights into the perception of independent developers regarding their experience using the provided functionality. User trials are a common approach to testing the efficacy of programming abstractions. For example, Guo et al. [65] performed user trials to evaluate OPEN; their ontology-based programming framework for rapid prototyping of context-aware applications. Their main goal was to validate whether users can correctly program context-aware applications using the three different programming modes available in OPEN. Fifteen participants, varying in programming ability, were invited to complete three tasks: to evaluate each of the programming modes, answer a questionnaire and to give feedback about the system at the end of each session. Similarly, to test the usability of the WASP programming language for sensor network application development, Bai et al. [14] conducted a user study that tested 28 novice programmers using two sensor network programming tasks;to evaluate a node-level and the other a network-level programming abstraction. The authors note some lessons learned from executing the usability study including: the need to eliminate factors that may affect the study but are not factors to be evaluated; the need to keep the tasks short enough for users to tolerate; and the need to permit user questions in a manner that will not affect the factors to be evaluated. The approach adopted to evaluate the usability of the contributions of this thesis follows a similar procedure to that described in [14]. Participants were given an introduction to the concepts; descriptions of tasks to be completed; and a survey to provide feedback on the overall system and the contributions being evaluated. For purposes of consistency, comparability and evaluation quality the evaluation methodology is developed further based on the product quality model standard; ISO 250101 . The product quality model comprises eight quality characteristics and their subcharacteristics. The ‘usability’ characteristic is chosen for the basis of this evaluation and is described in detail in Section 7.1. The chosen survey tool ‘System Usability Survey’ 1

http://iso25000.com/index.php/en/iso-25000-standards/iso-25010

136

is introduced in Section 7.2. The design of the usability study procedure as informed by ISO25010 is described in Section 7.3. The concluding section of this chapter outlines the results of the evaluation (Section 7.4).

7.1

ISO 25010 Usability Characteristic

The product quality model provided in the ISO 25010 standard offers a basis from which the evaluation methodology for this thesis is developed. The product quality model comprises eight quality characteristics and their sub-characteristics. The ‘Usability’ characteristic is chosen as the basis for this evaluation. A description of the usability characteristic and its sub-characteristics from the perspective of the contributions to be evaluated follows. This discussion informs the design of the usability study which is described in Section 7.3. Definition: “Degree to which [the software] can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. ” The above definition of usability from ISO 25010 can be decomposed into key concepts that need to be defined prior to the design of the usability study. These concepts are the specified users, specified goals and the specified context of use. A description of each is as follows: Users: The users of the software to be evaluated in this case are any people with the ability to program using the Java programming language. No prior knowledge of Web Services, sensor networks, network engineering or middleware is required to use the software. The Java requirement is enforced only because the client API requires development in Java. The node-level, network-level and infrastructure-level abstractions to be evaluated are each targeted at different user types. The intention of the node-level abstraction is to mask the heterogeneity and communication challenges while maintaining flexibility in development. The users of the node-level components are expected to have a basic understanding of sensors and services. However, this is more about understanding the semantic relationship between sensors and services. Such an understanding will allow a developer to leverage the suite of functions exposed by the API to achieve complex and adaptive deployments that the developer is fully responsible for and has complete control over. The network and infrastructure-level abstractions aim to allow developers to achieve more with less effort. The rules for dynamic sensor creation serves as a guidance for developers to design the dynamic production of their Cyber Sensor deployment,

137

based on complex context requirements. A key challenge in designing these abstractions is in achieving the appropriate balance between ease-of-use and flexibility. Goals: The goals for users to achieve are specified in accordance with the components that require evaluation. These are the node, network and infrastructure -level abstractions. As such, the usability study will include tasks that require the use of these components. Context of use: The context of use is prescribed by the choice of technologies used in the development of SIXTH, its extensions and the client API. The eclipse integrated development environment (IDE), OSGi, and Java client API are the key technologies required for performing tasks with the software. As advised in [14], effort is required to minimise the impact of these technologies on the user, with an aim to keeping focus on evaluating the core concepts. Participants will be provided with a template to get started, removing any need to understand the development environment. They will be provided with code samples rather than the API to eliminate the need to understand or navigate all the available functions, including those that are not factors to be evaluated. Besides the technologies, the context of use also refers to the environment in which the software is used. Some usability studies are performed in a dedicated laboratory while others explicitly choose more informal settings to keep users at ease. Since the software can be used on any computer, with the appropriate eclipse installation, this evaluation will follow the latter approach and allow the software to be used in any setting the user feels comfortable. Having defined the core concepts of importance to the evaluation methodology, the next step is define what it means to achieve goals with effectiveness, efficiency and satisfaction - in the context of the software in question. Effectiveness: How well does the software support the user in completing assigned tasks? Efficiency: How much or little resources do users need to be provided with to complete assigned tasks? Satisfaction: How does the user perceive the usefulness of the software? Does the user trust the software? Does the user enjoy using the software? The usability characteristic has six sub-characteristics through analysis of which the design of the evaluation methodology is refined further:

138

Appropriateness recognizability

definition: “Degree to which users can recognize

whether [the software] is appropriate for their needs.” This characteristic requires the user to first understand the core concepts in the software, then understand the applicability of those concepts to solve specified problems and any other potential problems the user might think of. It can be considered an attribute of how intuitive the concepts are. This is difficult to asses and efforts need to be made to prompt the user to consider the concepts before considering the options available to them in the client API. In support of this effort, the usability study documentation includes detailed descriptions of the concepts including code examples. The users, then, can use the code examples as a basis for completing the specified tasks, rather than the API documentation. Learnability

definition: “Degree to which [the software] can be used by specified users

to achieve specified goals of learning to use [the software] with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use.” Learnability is also a characteristic that can reveal whether the concepts underpinning the software are intuitive. It is defined by the ease with which a developer who is new to the software can accomplish tasks. Assessing learnability of the concepts, as opposed to the API, is challenging and efforts need to be made to manage this in the usability study procedure and documentation. Users will be encouraged to focus on the conceptual documentation rather than exploring the methods available in the API. Operability

definition: “Degree to which [the software] has attributes that make it easy

to operate and control.” The specific attributes provided in the software that aim to achieve operability are the vertical and horizontal abstractions. The vertical abstractions focus on operability for specific user groups. The horizontal abstractions focus on interoperability of software components and operability for all user groups. This evaluation includes tasks designed to incorporate both aspects of operability. User error protection

definition: “Degree to which [the software] protects users against

making errors.” While SIXTH’s support of notifications provides an ideal format for delivering error messages from the core middleware, this feature was not emphasised or extended in the contributions of this thesis. The error protection that was implemented was in the management of WWW environment interactions. However, users should not be aware of this feature, assuming it is working correctly. The client API, although not a complete product, does provide limited error protection features in the form of error messages for when users to not configure components correctly. Assessing this characteristic is not a core 139

focus of the evaluation but user feedback will be considered in this regard to inform future versions of both the API and the middleware. User interface aesthetics

definition: “Degree to which a user interface enables pleasing

and satisfying interaction for the user.” The API itself forms the interface between the user and the software. This is not a core focus of the evaluation but user feedback will be useful in informing optimisations and the design of future versions of the API. There is no graphical user interface involved in this evaluation besides that of the programming environment - which is not of concern to this evaluation. Accessibility

definition: “Degree to which [the software] can be used by people with the

widest range of characteristics and capabilities to achieve a specified goal in a specified context of use.” This characteristic is directly related to the definition of users and the components that were designed for each user type. The evaluation will be all inclusive of user types and tasks requiring use of components. As such, each users’ level of expertise with respect to Web Service APIs, middleware, sensor networks and Java will be recorded. The aim of this approach is to gain insight into whether each vertical abstraction is accessible to the target users, and/or other user types. Further, is the client API an accessible vehicle for each user type to access the functions of the software?

7.2

Usability Survey

The System Usability Scale (SUS) is a ten-item Likert scale that is often used to acquire feedback from usability study participants. The technique the SUS authors used in the selection of each item identified examples that lead to ‘extreme expressions of the attitude being captured’ [30]. This resulted in the pool of ten items being capable of giving a global overview of subjective assessments of usability. The included items cover a five system usability aspects, each of which are covered by a negative and a positive statement. The full list of statements is as follows: • I think that I would like to use this system frequently • I found the system unnecessarily complex • I thought the system was easy to use • I think that I would need the support of a technical person to be able to use the system

140

• I found the various functions in this system were well integrated • I thought there was too much inconsistency in this system • I would imagine that most people would learn to use this system very quickly • I found the system very cumbersome to use • I felt very confident using the system • I needed to learn a lot of things before I could get going with this system

Figure 7.1: Comparison of SUS scores to adjective ratings, acceptability scores, grading scales. [17] SUS is a very mature survey tool that is considered to be as reliable as industry standards which are longer and take more time to complete[30]. It has been shown to produce reasonably reliable results from sample sizes as small as 8-12 [167]. While SUS scores are values that fall between 0 and 100, grade rankings have been proposed based on a pool of over 3,500 SUS results [18, 17, 30]. The average SUS score, derived from 500 studies, is said to be 68 [144]. Thus, when reviewing the results for the software, a score below 68 is below average and above 68 is above average. This is consistent with the grade rankings proposed in [17] (Figure 7.1) which are defined by the authors using ‘acceptability ranges’. These features of SUS make it ideal for use in the evaluation of the contributions of this thesis and provides a basis for comparison to the global average. The SUS survey will be used to assess the perceived usability of the overall contributions. It is also necessary to gain feedback for each of the individual vertical abstractions. To achieve this, the usability study will be divided into three parts. After each part the participant will be asked to complete a brief survey. Then, before any debriefing takes place, the participant will be asked to complete the SUS survey. The additional surveys are customised for the component being evaluated (e.g., the network survey is illustrated in Figure 7.2) and is designed to evaluate the participants’ perception of the individual concepts. 141

Figure 7.2: Survey provided to participants after completing part 2 of the evaluation that focuses on the network-level abstraction

7.3

Usability Study Procedure

As described in Section 7.2, the SUS survey tool has been chosen to assess the over perceived usability of the contributions and a customised survey was designed to assess the perceived usability of the individual components. In Section 7.1, it was decided that detailed documentation, including code examples, will guide participants through assigned tasks without the aid of API documentation. The remaining preparation for the usability study is to define the tasks and the documentation that will be used to guide the participant through the stages of the study. 7.3.0.2

User Documentation and Tasks

Having decided that the evaluation is best performed without the aid of the API documentation, detailed documentation is required to guide participants through the usability study. In accordance with the evaluation methodology, the documentation is split into four parts. The complete documentation is provided in Appendix E. A description of each part of the documentation and the tasks the participants are asked to complete follows: Introduction The introductory document serves as a basic introduction to the usability study without reference to any of the concepts or components being evaluated. The 142

list of available adaptors is presented as a reference to be used when the participants are completing tasks later on. Part 1: Introductions to Sensors and Pipelines This document introduces the participant to the concepts of sensors and Pipelines. Definitions are provided for ‘Location Sensor’, ‘Entity Sensor’, and ‘User Sensor’ and ‘Pipe’. Code snippets are provided to illustrate how they are created, configured and started using the client API. Sample outputs are provided to illustrate to the participant how to interpret a successful result. The participant is then presented with the following tasks: 1. Create a ‘World Weather’ location sensor to monitor Ireland 2. Create a ‘Twitter’ entity sensor to monitor Ebola 3. Create an ‘Alchemy’ Aggregator Pipe to monitor the sentiment of Ebola data Part 2: Introduction to sensor networks This document introduces the participant to the concepts of a sensor network. Again, definitions are provided for each of the horizontal abstractions implemented for the network-level abstraction and code snippets are provided as an example of how networks are created, configured and started using the API. 1. Create a ‘Twitter’ network to produce user and location sensors related to the entity ‘Ebola’ 2. Create a ‘World Weather’ network to produce location sensors for all locations related to the entity ‘rain’ Part 3: Introduction to Sensor Infrastructures Using a consistent approach to the previous parts, this document introduces the participant to the concept of a sensor infrastructure and provides the appropriate conceptual definitions and code snippets to allow the participant to complete tasks using the API. 1. Create an infrastructure to produce ‘World Weather’ location sensors based on a ‘Twitter’ entity sensor monitoring ‘flu’ 2. Create an infrastructure to monitor all users in countries where it is raining. You will need to monitor ‘rain’ from ‘World Weather’ and produce ‘Twitter’ location sensors accordingly. From this you can generate user sensors

7.4

Usability Study Results

The sample size for the usability study was 12 participants which is a suitable number for producing reliable results using the System Usability Scale [167]. The participants were asked to rate their expertise for Java, web APIs, sensor networks and middleware. All 143

Figure 7.3: SUS Score Analysis participants scored 4 or 5 out of 5 in Java while there was more diversity in experience across the other three technologies. Each participant was provided with the documentation and instructed that they were permitted to ask questions. When participants asked a question that was related to one of the specified tasks they were referred back to the appropriate documentation. This decision was made in an effort to maintain focus on the concepts underlying the API, rather than the API itself. Participants were observed when completing the tasks, encouraged to ‘think aloud’ and a debriefing session took place after the participant completed the SUS survey at the end of the usability study. Full results are provided in Figure 7.3. Before analysing the results of the questionnaire, its reliability was assessed. The Cronbach’s alpha coefficient was calculated as 0.78. Using the guidelines of [54], the reliability is thus classified as acceptable. The average SUS score resulting from the usability study is 80.208. This is above average compared to other systems evaluated using this survey. The score puts the software in the acceptable range with a grade of B and an adjective rating of ‘good’. Figure 7.4 illustrates the individual SUS scores compared to the average global score of 68. Two scores falls below average at 52.5 and 67.5. The latter participant completed all tasks successfully but noted that better - and preferably graphical - documentation would make the concepts easier to understand in a short time. This participant also questioned the relationship between the concepts and wondered can the same problem be solved using any of the three techniques. During the debriefing session this was confirmed and the participant stated that documentation to that effect would make the concepts easier to grasp. The former participant was able to complete each task in the end but struggled 144

Figure 7.4: Comparison of the individual SUS Scores to the Global Average of 68 to select the appropriate invariant for adaptor names and variable names according to the reference sheet.

Figure 7.5: Stated Expertise for each SUS result, ordered by SUS Score in ascending order Feedback from two other participants indicated it would be better to use static variables than ‘free text’. However, this is a feature of Service Oriented Architectures that is 145

essential for maintaining the loose coupling between services that ensures a system remains extensible and maximises reusability of individual components. It is also a feature that developers using Web Services will be accustomed to. The participants who provided this feedback gave themselves 3 out of 5, and 1 out of 5 for web API experience. Prior experience of technology has the potential to influence how it may be perceived, experienced and utilised. Four categories of prior experience were identified: Java programming expertise, exposure to web APIs, familiarity with sensor networking, and utilisation of middleware. Each was assessed using a six-point scale ranging from no experience to expert. For analysis purposes, subjects were categorised as being novices (scoring 0 to 3 on the scale) or expert (scoring 4 to 5 on the scale). A non-parametric approach to data analysis is adopted as the data is ordinal in nature. In many usability studies, a parametric approach to analysis is adopted. However, this consensus has been challenged in recent years, most noticeably by [137]. All subjects professed significant experience in Java, thus this could not be harnessed as a potential determinant of the user experience. Familiarity with the sensor construct was found to influence perceptions. Those classifying themselves as novices in sensor technologies perceived the functionality of SIXTH as being well integrated - U = 5.00, p < 0.05. This positive perception suggests that such subjects found the notion of a sensor intuitive and found it easy to harness the SIXTH APIs to fulfil the requested tasks. Those with sensor experience were more circumspect; perhaps expecting additional features including some that they may have had exposure to in the past. Novice users of middleware found the notion of a Pipe somewhat difficult to comprehend; those with prior experience in working with middleware platforms were more confident they could apply the Pipe concept to other problems - U= 3.00, p < 0.05. Figure 7.5 provides the experience scores in relation to the SUS scores from the feedback of each participant. It is interesting to note that the highest scores are a result of the participants with the least experience in web APIs, sensor networks and, largely, in middleware also. Four of the participants with experience in sensor networks noted that the network and infrastructure-level abstractions made them feel they were ‘losing control’ over what sensors were being generated in the background. This feedback is expected and is in support of the aim to provide higher-level abstractions for those with less experience. During the debriefing session these participants were talked through how the more complex challenges can be solved using the node-level abstraction alone. Overall, the participants were satisfied with this as the appropriate approach for their levels of expertise. However, three participants noted that they would like to use the higher-level abstraction but to have access to the ‘raw’ rules rather than the abstracted methods in the API. Also related to the higher-level abstractions, the participants were observed to use methods that had the desired end result in the method name. For example, the method cre-

146

ateEntityFocusedNetwork(“Twitter”, “Ebola”) creates a Twitter Entity Sensor and tasks it to monitor ‘Ebola’. The developer then calls methods such as network.generateLocations() to set the rule for generating the locations from which Ebola is being reported. What many participants started with in their code was createLocationFocussedNetwork(“Twitter”, “Ebola”). This feedback is particularly interesting as it provides insight into what the participants expect of the API, versus what is actually provided. While this feedback seems related to the API, as opposed to the concepts, it can be derived that higher-level abstractions could be abstracted further to support such method calls. For example, if the API provided the method that the participants were attempting to use, the abstraction would type-check the second parameter to decide what base sensor to create, then use logic to decide what sensors need to be generated to get from the base sensor to the ‘focus’ sensor. When participants were asked what they felt was missing from the concepts at the focus of the evaluation they largely struggled to think of anything. Some commented that this was due to limited exposure to the API, while others commented that anything they initially thought was missing, they quickly realised could be achieved by thinking it through. Interestingly, some participants mentioned that the sensor concept was missing the ability to generate new sensors based on the data. They noted during the debriefing session that they would not have made that comment had they already known about the higher-level abstractions. Other comments made by participants include additional types of Pipes such as filters and branches. The remaining comments from the participants focus on API specific challenges. The client API certainly requires completion and optimisation and this must be supported by additional usability studies that focus on how the API should deliver the concepts to the different target users.

7.5

Discussion

This chapter described usability studies designed to evaluate the effectiveness of the Cyber Sensor Pipelines, Networks and Infrastructues as programming abstraction for acquisition of WWW data. The usability studies involved 12 participants familiar with Java and with varying experience in relation to web APIs, sensor networks and middleware. While the software earned an above average SUS score (80.208), the surveys and debriefing sessions revealed some interesting points regarding the usability of the abstractions for different target groups. Users inexperienced with sensor technologies perceived the sensor approach to WWW data acquisition intuitive and found it easy to complete the tasks. This feedback validates the sensor paradigm as being a useful approach to unified CPS acquisition. While the high-level abstractions were expected to be most suited to novice sensor and middleware

147

users, those with experience in that regard appreciated their power but were frustrated with the lack of control over Cyber Sensor creation. The network-level and infrastructure-level abstractions can now be extended to allow further abstraction for novice developers, while maintaining a flexible interface for expert developers who wish to leverage these abstractions. Different types of interface should be explored for novice developers, including graphical programming interfaces. The horizontal abstractions did not cause concern for the participants with the node-level abstraction but it needs to be considered more thoroughly how best to deliver these abstractions in the network-level and infrastructure-level components. The combined evaluation results and feedback also informs future versions of the API as follows: • The higher-level abstractions aimed at developers with little to no experience of web APIs or sensor networks should be abstracted further to support the interactions a developer expects to make given the concepts they have been introduced to. This will require an iterative development process with detailed usability studies, preferably with domain experts who have the motivation to solve real problems. • The higher-level abstractions are also suited to developers who have experience with web APIs and sensor networks but they should be delivered to this audience in a manner that maintains flexibility. These developers are not concerned with choosing from a list of static rule types and prefer manual rule creation as it gives them a feeling of control. These users will also benefit from a more developed version of the abstractions that allows network-level and infrastructure-level abstractions to generate networks and infrastructures as well as sensors. The above results validate the usability of the Cyber Sensor Pipelines, Network-level abstractions and Infrastructure-level abstractions for the task of comprehensive WWW context acquisition. Combined with the case study evaluations in Chapter 6, the evaluation forms the final contribution of this thesis. Chapter 8 follows with the conclusions of this thesis and a description of opportunities for future research.

148

Chapter 8

Conclusions and Future Work The research presented in this thesis has culminated in the successful delivery of middleware and programming support which, by bridging the Cyber-Physical-Social (CPS) divide, enables CPS context-awareness. Each chapter in this thesis has served to meet the objectives defined in Section 1.5 and describe the core contributions made by this research as introduced in Section 1.6. CPS Computing, as an emerging paradigm, is limited in terms of available research literature that defines the problem space from a context-awareness perspective. The first objective of this thesis - aiming to bridge that gap - was to explore existing research towards CPS Computing and identify the research areas that, when unified, will enable CPS context-awareness. The background research conducted was described in Section 2.1. CPS Computing, from the perspective of context-awareness, was defined and the technical infrastructures it comprises - from World Wide Web (WWW) and Ubiquitous Computing research - were identified. To better understand the CPS Computing problem space from a context-awareness perspective, the second objective was to conduct a deeper investigation into these infrastructures. This background research is described in Section 2.2 and was critically analysed with respect to the benefits and challenges of CPS contextawareness in Subsection 2.4.1. The outcome of these research efforts was the design of a middleware-enabled unifying CPS infrastructure (Subsection 2.4.2) and the definition of the requirements of a middleware to enable CPS context-awareness (Subsection 2.4.2). The unifying CPS infrastructure is the first contribution of this thesis and the defined requirements inform the remaining objectives and contributions. Having designed the unifying CPS infrastructure, the third objective sought to identify and analyse existing supporting frameworks towards the enablement of CPS contextawareness. Three approaches were identified (Section 3.2) of which sensor middleware integration was determined the most suitable, based on the previously identified challenges. Evaluation of existing middleware frameworks failed to find a system that meets all of the previously defined requirements (Subsection 3.2.4). The evaluation found that 149

existing systems fail to exploit the unique characteristics of WWW resources that, if optimised sensor creation and programming abstractions are provided, will enable flexible and rapid development of comprehensive context acquisition applications. This outcome leads to the fourth objective that, aiming to discover techniques that will support unified CPS context-acquisition, was to review supporting technologies from both WWW and Ubiquitous Computing research. This review of WWW mashup techniques and Wireless Sensor Network (WSN) programming abstractions (Section 3.3), combined with the findings of the earlier research, informed the design of WWW context-source and processing taxonomies (Section 4.1). This is the second core contribution of this thesis and was key to the design decisions required to achieve the fifth objective, to design and implement a supporting framework for CPS context-awareness. Focussing specifically on the integration of the existing WWW infrastructure with a sensor middleware, the fifth objective is achieved in Chapter 4 and Chapter 5, in which three further contributions are delivered. The first is the design and implementation of support for the dynamic creation and configuration of Cyber Sensors as equal citizens within the SIXTH sensor middleware (Section 4.2 and Subsection 5.3.1). This is complemented by the contribution of support for the dynamic creation and configuration as Pipes for the realisation of Cyber Sensor Pipelines (Section 4.3 and Subsection 5.3.3). The penultimate contribution of this thesis builds on those previous basic building blocks to deliver high-level programming abstractions for CPS context acquisition (Section 4.4 and Subsection 5.3.2). Extending the Cyber Sensor paradigm, the high-level programming abstractions are inspired by WSN programming and provide a collection of vertical and horizontal abstractions. The vertical abstractions include node-level, networklevel and infrastructure-level. Cyber Sensors are considered a node-level abstraction; network-level abstractions provide a single interface to producing a collection of Cyber Sensors that monitor a single WWW resource (e.g., Twitter); and infrastructure-level abstractions provide a single interface to creating a collection of Cyber Sensors that monitor heterogeneous WWW resources (e.g., Twitter and a Weather Application Programming Interface (API)). The horizontal abstractions then add a semantic element to the abstractions enabling the creation and configuration of each vertical abstraction (and Pipes) interested in the context of specified entities, locations and users. As highlighted in Subsection 5.1.1, the above contributions successfully unify WWW resources with the SIXTH middleware and provide functions to facilitate comprehensive context acquisition and its real-time processing. While this evaluation indicates the contributions of this thesis meet the requirements of a supporting framework for CPS contextawareness, the final objective of this thesis was to formally assess the efficacy of these functions. The final contributions of this thesis achieve this objective by (1) providing evidence that the contributions meet the stated requirements through a series of case studies

150

demonstrating the flexibility of the basic building blocks to enabling context-awareness in a variety of use cases (Chapter 6) and (2) validating the usability of the sensor programming abstractions as an approach to WWW context acquisition by conducting a usability study (Chapter 7). This thesis has contributed to the emerging field of CPS Computing by identifying the challenges of integrating the WWW with Ubiquitous Computing infrastructures; designing a middleware enabled unifying CPS infrastructure; defining the requirements of a supporting framework to enable CPS context-awareness. Further, the results of the evaluations conducted verify that this research was successful in providing a functional and usable CPS middleware and programming framework for CPS context-awareness. Finally, the research conducted as part of this thesis will inform and support future research in this field. This thesis concludes with the description of a number of future directions identified as part of this research.

8.1

Future Work

Throughout the design, implementation and evaluation of the contributions of this thesis, a number of opportunities and challenges have been identified that have potential to inform future work in this field. The following list provides a summary of these findings: • The basic data mapping and horizontal programming abstractions provided in the implemented solution enables seamless interactions between sensors and services that focus on locations, users or entities. Future work is required to research additional abstractions that are relevant to CPS Computing. It should be determined whether the semantics are globally relevant or application-specific. The data mapping is implemented using a light weight approach that is only concerned with the horizontal abstractions defined in Chapter 4. Developers customising a SIXTH deployment can replace this component with a comprehensive ontology that unifies data sources into a single formal ontology. Alternatively, a service could be provided for the dynamic formation of folksonomies based on the interactions between components. In such an approach the composition of components by developers/users would also implicitly inform the formation of the folksonomy. • While the programming abstractions at the core of this thesis have been demonstrated to be effective in use, further research is required, from a human computer interaction (HCI) perspective, to determine the best means for delivering these functions to developer types. Ease of use is paramount for non-developers, while maintaining control and flexibility is important to developers and network engineers. There is potential here for domain-specific APIs that support the de-

151

velopment of complex context-gathering applications through an intuitive interface that uses domain-specific language. Technically, implementation of such a solution will not be challenging given the extensible nature of SIXTH. However, the design of domain-specific APIs requires collaboration with domain experts and iterative usability studies which will inform the evolution of the API into a robust tool for domain experts. RESTful and visual interfaces to SIXTH also pose interesting avenues for exploration in future work. • The programming abstractions implemented for this thesis leverage the research of WSN programming techniques and apply it to web resources. To fully bridge the CPS divide it is required that WSN programming abstractions also be implemented. SIXTH has been enhanced with a query language which may facilitate a starting point for such a project. It is essential for the WSN programming abstractions to maintain uniformity with the WWW programming abstractions to truly facilitate rapid development of CPS applications. This was the motivation behind choosing the WSN programming abstraction approach to WWW sensing. • Implementation of additional WWW and WSN network types and additional Pipe types to expand on the offering developed in this thesis. This should include the provision of Pipes for converting to and from URIs in support of Linked Data and the Semantic Web. • Performance testing is required to fully understand the limits of SIXTH deployments on various device types. In collaboration with target users, the roles of SIXTH instances must be defined in a manner that ensures each deployment will perform within the limits of the host device. Once these roles are defined, further optimisations may be designed that sees SIXTH instances interact with an aim to optimising performance and quality of service for all applications. Additional areas for exploration in future work include (1) algorithms for automated Cyber Sensor Network deployment and management, (2) extension of the SIXTH discovery service to facilitate semantic search and autonomous composition of Pipelines, (3) collaboration with domain experts to explore further the potential of CPS applications and identification of QoS requirements, (4) uncertainty management for Cyber Sensors, and (5) testing and debugging tools for CPS Computing.

152

Appendix A

Programming Abstraction Properties

153

A.1

Network Abstraction Properties

Generate Users Property accepts values ‘on’ and ‘off’. If the value is set to on, user sensors will be produced as a result of user data being extracted from the base sensor data. Generate Entities Property accepts values ‘on’ and ‘off’. If the value is set to on, entity sensors will be produced as a result of entity data being extracted from the base sensor data. Generate Locations Property accepts values ‘on’ and ‘off’. If the value is set to on, location sensors will be produced as a result of location data being extracted from the base sensor data. Pipe For User Property accepts

values

in

the

format

‘Pipeadaptor-

name.servicename’. If the value is set, a new Pipe will be created and applied each new UserCyberSensor as they are generated. Pipe For Entity Property accepts

values

in

the

format

‘Pipeadaptor-

name.servicename’ If the value is set, a new Pipe will be created and applied each new EntityCyberSensor as they are generated. Pipe For Location Property accepts

values

in

the

format

‘Pipeadaptor-

name.servicename’. If the value is set, a new Pipe will be created and applied each new LocationCyberSensor as they are generated. Pipe For Base Sensor Property accepts

values

in

the

format

‘Pipeadaptor-

name.servicename’. If the value is set, a new Pipe will be created and applied to the base sensor. Invariant Property accepts a String value that is used to configure the base sensor. For example, in the case of a NetworkAbstraction with an EntityCyberSensor as its base, the invariant is a keyword String. State Property accepts values ‘on’ and ‘off’. If the value is set to on, the base sensor is tasked and SensorCreatorPipes configured and applied to the sensor according to the first three properties. The SensorCreatorPipes are also configured to apply Pipes to the generated sensors, according to the next three properties.

A.2

Infrastructure Abstraction Properties

Generate Users Property accepts a String value that is the name of the adaptor from which to generate UserCyberSensors or the value ‘all’ if all possible UserCyberSen154

sors are to be generated. If the value is set, user sensors will be produced from the specified network, or all networks, as a result of user data being extracted from the base sensor data. Generate Entities Property accepts a String value that is the name of the adaptor from which to generate EntityCyberSensors or the value ‘all’ if all possible EntityCyberSensors are to be generated. If the value is set, entity sensors will be produced from the specified network, or all networks, as a result of entity data being extracted from the base sensor data. Generate Locations Property accepts a String value that is the name of the adaptor from which to generate LocationCyberSensors or the value ‘all’ if all possible LocationCyberSensors are to be generated. If the value is set, LocationCyberSensors will be produced from the specified network, or all networks, as a result of location data being extracted from the base sensor data. Generate Users from Entities Property accepts a String value that is the name of the adaptor from which to generate UserCyberSensors or the value ‘all’ if all possible UserCyberSensors are to be generated. If the value is set, UserCyberSensors will be produced from the specified network, or all networks, as a result of user data being extracted from all generated EntityCyberSensors data. Generate Users from Locations Property accepts a String value that is the name of the adaptor from which to generate UserCyberSensors or the value ‘all’ if all possible UserCyberSensors are to be generated. If the value is set, UserCyberSensors will be produced from the specified network, or all networks, as a result of user data being extracted from all generated LocationCyberSensors data. Generate Users from Users Property accepts a String value that is the name of the adaptor from which to generate UserCyberSensors or the value ‘all’ if all possible UserCyberSensors are to be generated. If the value is set, UserCyberSensors will be produced from the specified network, or all networks, as a result of user data being extracted from all generated UserCyberSensors data. Generate Entities from Entities Property accepts a String value that is the name of the adaptor from which to generate EntityCyberSensors or the value ‘all’ if all possible EntityCyberSensors are to be generated. If the value is set, EntityCyberSensors will be produced from the specified network, or all networks, as a result of entity data being extracted from all generated EntityCyberSensors data. Generate Entities from Locations Property accepts a String value that is the name of the adaptor from which to generate EntityCyberSensors or the value ‘all’ if all pos155

sible EntityCyberSensors are to be generated. If the value is set, EntityCyberSensors will be produced from the specified network, or all networks, as a result of entity data being extracted from all generated LocationCyberSensors data. Generate Entities from Users Property accepts a String value that is the name of the adaptor from which to generate EntityCyberSensors or the value ‘all’ if all possible EntityCyberSensors are to be generated. If the value is set, EntityCyberSensors will be produced from the specified network, or all networks, as a result of entity data being extracted from all generated UserCyberSensors data. Generate Locations from Entities Property accepts a String value that is the name of the adaptor from which to generate LocationCyberSensors or the value ‘all’ if all possible LocationCyberSensors are to be generated. If the value is set, LocationCyberSensors will be produced from the specified network, or all networks, as a result of location data being extracted from all generated EntityCyberSensors data. Generate Locations from Locations Property accepts a String value that is the name of the adaptor from which to generate LocationCyberSensors or the value ‘all’ if all possible LocationCyberSensors are to be generated. If the value is set, LocationCyberSensors will be produced from the specified network, or all networks, as a result of location data being extracted from all generated LocationCyberSensors data. Generate Locations from Users Property accepts a String value that is the name of the adaptor from which to generate LocationCyberSensors or the value ‘all’ if all possible LocationCyberSensors are to be generated. If the value is set, LocationCyberSensors will be produced from the specified network, or all networks, as a result of location data being extracted from all generated UserCyberSensors data. Pipe For User Property accepts

values

in

the

format

‘Pipeadaptor-

name.servicename’. If the value is set, a new Pipe will be created and applied each new UserCyberSensor as they are generated. Pipe For Entity Property accepts

values

in

the

format

‘Pipeadaptor-

name.servicename’ If the value is set, a new Pipe will be created and applied each new EntityCyberSensor as they are generated. Pipe For Location Property accepts

values

in

the

format

‘Pipeadaptor-

name.servicename’. If the value is set, a new Pipe will be created and applied each new LocationCyberSensor as they are generated.

156

Pipe For Base Sensor Property accepts

values

in

the

format

‘Pipeadaptor-

name.servicename’. If the value is set, a new Pipe will be created and applied to the base sensor. Invariant Property accepts a String value that is used to configure the base sensor. For example, in the case of a NetworkAbstraction with an EntityCyberSensor as its base, the invariant is a keyword String. State Property accepts values ‘on’ and ‘off’. If the value is set to on, the base sensor is tasked and SensorCreatorPipes configured and applied to the sensor according to the first three properties. The SensorCreatorPipes are also configured to apply Pipes to the generated sensors, according to the next three properties.

157

Appendix B

Pipe Types Available in SIXTH

158

AggregatorPipe With support of a PipeAdaptor implementation for the WWW resource from which new data will be acquired, fuses related data with a sensor stream, based on the value of the modality the Pipe is configured to act on • UserAggregatorPipe: extends AggregatorPipe to facilitate the fusion of user related data without the requirement to configure the modality path • LocationAggregatorPipe: extends AggregatorPipe to facilitate the fusion of location related data without the requirement to configure the modality path • EntityAggregatorPipe: extends AggregatorPipe to facilitate the fusion of entity related data without the requirement to configure the modality path FilterPipe Filters sensor data according to configurable rules which apply to the value of the modality for which a Pipe is configured to act on • NumericModalityFilterPipe: extends FilterPipe to support numerical rules which the value of the configured modality path must meet so as not to be filtered out • UserFilterPipe: extends FilterPipe to facilitate filtering of user related data without the requirement to configure the modality path • LocationFilterPipe: extends FilterPipe to facilitate filtering of location related data without the requirement to configure the modality path • EntityFilterPipe: extends FilterPipe to facilitate filtering of entity related data without the requirement to configure the modality path SummaryPipe Provides summarised value representing the value of the modality for which a path has been configured, in the context of the values that came before it. • NumericSummaryPipe: extends SummaryPipe to include support for summary of numeric values • UserSummaryPipe: extends SummaryPipe to facilitaty summary of user related data without the requirement to configure the modality path • LocationSummaryPipe: extends SummaryPipe to facilitaty summary of location related data without the requirement to configure the modality path • EntitysummaryPipe: extends SummaryPipe to facilitaty summary of entity related data without the requirement to configure the modality path CreatorPipe Creates and configures a new SIXTH component according to the value of the modality for which a path has been configured. Entity, Location and User variations are also implemented but for simplicity the different types are described as follows: 159

• PipeCreator creates and configures a new Pipe based on the value of the modality which the Pipe is configured to act on • SensorCreator creates and configures a new Sensor based on the value of the modality which the Pipe is configured to act on • NetworkCreator creates and configures a new Network based on the value of the modality which the Pipe is configured to act on • InfrastructureCreator creates and configures a new Infrastructure based on the value of the modality which the Pipe is configured to act on

160

Appendix C

SIXTH XML Tasking Messages

161

Configuring SIXTH Components       Create a new sensor:      adaptor name  sensor type        Configure a sensor:      property value  .  .            Create a new Pipe:      adaptor name   pipe type   pipe name       Configure a Pipe:         property value        

162

      Create a new Network:      adaptorName  invariant type      Configure a Network:        property value  .  .      Create a new Infrastructure:      adaptorName  invariantType      Configure an Infrastructure:        property value.                 

163

Appendix D

SIXTH client API Javadocs

164

2/24/2015

AbstractApplication

Package   Class  Use  Tree  Index  Help   PREV CLASS   NEXT CLASS SUMMARY: NESTED | FIELD | CONSTR | METHOD

FRAMES    NO FRAMES    All Classes DETAIL: FIELD | CONSTR | METHOD

  Class AbstractApplication buildingblocks

java.lang.Object    buildingblocks.AbstractApplication

All Implemented Interfaces: ie.ucd.sixth.core.application.IApplication, ie.ucd.sixth.core.receiver.ICredentialedReceiver, ie.ucd.sixth.core.receiver.IDataReceiver public abstract class AbstractApplication extends java.lang.Object implements ie.ucd.sixth.core.application.IApplication, ie.ucd.sixth.core.receiver.IDataReceiver

Abstract Application provides default implementation for App.java that takes care of all interactions with the SIXTH discoveryReceivor It provides useful helper methods for an application to create and task sensors/pipes/networks/infrastractures App.java must extend AbstractApplication The setup(); method must be completed using the helper methods to generate sensors/pipes/networks/infrastructures as required The respondToData method gives the application access to the sensor data from the generated sensors Author: Olga

Constructor Summary AbstractApplication(org.osgi.framework.BundleContext context) 

             

Method Summary static buildingblocks.Infrastructure createEntityFocusedInfrastructure(java.lang.String baseAdaptorName,

java.lang.String invariant) 

          An Infrastructure is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data Advanced Rules allow sensors to be recursively generated from generated sensor data While the network type of the base sensor is fixed, the generated sensors can be from any network type Factory method for creating a new sensing infrastructure that is driven by an entity sensor static buildingblocks.Network createEntityFocusedNetwork(java.lang.String adaptorName,

java.lang.String invariant) 

          A network is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data While the network type of the base sensor is fixed, and the generated sensors also belong to that network type Factory method for creating a new sensing network that is driven by an entity sensor static buildingblocks.Sensor createEntitySensor(java.lang.String adaptorName) 

          A sensor is generated from a network adaptor and is therefore always of that particular network type (eg., twitter) Individual Sensors are of a specific type which is determined by the modality of interest to the developer, i.e., location entity or user Factory method for creating a new sensor of type entity static buildingblocks.Infrastructure createLocationFocusedInfrastructure(java.lang.String baseAdaptorName,

java.lang.String invariant) 

          An Infrastructure is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data Advanced Rules allow sensors to be recursively generated from generated sensor data While the network type of the base sensor is fixed, the generated sensors can be from any network type Factory method for creating a new sensing infrastructure that is driven by an Location sensor static buildingblocks.Network createLocationFocusedNetwork(java.lang.String adaptorName,

java.lang.String invariant) 

          A network is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data While the network type of the base sensor is fixed, and the generated sensors also belong to that network type Factory method for creating a new sensing 165 network that is driven by a location sensor static buildingblocks.Sensor createLocationSensor(java.lang.String adaptorName) 

file:///C:/Users/Olga/Dropbox/thesis/ThesisWorkspace_Final/ie.ucd.sixth.receiver.evaluation/doc/buildingblocks/AbstractApplication.html

1/5

2/24/2015

AbstractApplication

          A sensor is generated from a network adaptor and is therefore always of that particular network type (eg., twitter) Individual Sensors are of a specific type which is determined by the modality of interest to the developer, i.e., location entity or user Factory method for creating a new sensor of type location static buildingblocks.Pipe createPipe(java.lang.String adaptorName, java.lang.String pipeType,

java.lang.String serviceName) 

          Pipes are tied to a particular adaptor type depending on the web resource the pipe is interacting with (e.g., klout or alchemy) Pipes that do not interact with an external resource are managed by the 'basic' pipe adaptor (e.g., sensorcreator, filter) Pipes have a specified type according to their functionality (e.g., fuser, filter, sensorcreator) Pipes have a specified name according to their semantic functionality (e.g., sentiment, locationsensorcreator) static buildingblocks.Sensor createSensor(java.lang.String adaptorName,

java.lang.String sensorType) 

          Helper method that allows an application to create sensor objects that abstract interactions with SIXTH via DiscoveryReceiver static buildingblocks.Infrastructure createUserFocusedInfrastructure(java.lang.String baseAdaptorName,

java.lang.String invariant) 

          An Infrastructure is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data Advanced Rules allow sensors to be recursively generated from generated sensor data While the network type of the base sensor is fixed, the generated sensors can be from any network type Factory method for creating a new sensing infrastructure that is driven by a user sensor static buildingblocks.Network createUserFocusedNetwork(java.lang.String adaptorName,

java.lang.String invariant) 

          A network is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data While the network type of the base sensor is fixed, and the generated sensors also belong to that network type Factory method for creating a new sensing network that is driven by a user sensor static buildingblocks.Sensor createUserSensor(java.lang.String adaptorName) 

          A sensor is generated from a network adaptor and is therefore always of that particular network type (eg., twitter) Individual Sensors are of a specific type which is determined by the modality of interest to the developer, i.e., location entity or user Factory method for creating a new sensor of type user  java.lang.String getCredentials() 

             java.util.Map getRequirements(ie.ucd.sixth.core.sensor.NodeDescription desc) 

             void printAvailableInfrastructure() 

          helper method for application to print the details of all adaptors currently available in the SIXTH deployment  void receive(ie.ucd.sixth.core.sensor.data.ISensorData data) 

            abstract  void respondToData(buildingblocks.Sensor sensor,

ie.ucd.sixth.core.sensor.data.ISensorData data) 

             void unregister() 

              Methods inherited from class java.lang.Object equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

 

Constructor Detail AbstractApplication public AbstractApplication(org.osgi.framework.BundleContext context)

Method Detail 166 createEntityFocusedInfrastructure file:///C:/Users/Olga/Dropbox/thesis/ThesisWorkspace_Final/ie.ucd.sixth.receiver.evaluation/doc/buildingblocks/AbstractApplication.html

2/5

2/24/2015

AbstractApplication

public static buildingblocks.Infrastructure createEntityFocusedInfrastructure(java.lang.String baseAdaptorName,                                                                               java.lang.String invariant)

An Infrastructure is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data Advanced Rules allow sensors to be recursively generated from generated sensor data While the network type of the base sensor is fixed, the generated sensors can be from any network type Factory method for creating a new sensing infrastructure that is driven by an entity sensor Parameters: baseAdaptorName ­ ­ the network type for the base entity sensor (e.g. "twitter" or "worldweather") invariant ­ ­ the entity keyword value that will drive the dynamic production of sensors from the base sensor (e.g., "rain",

"ebola") Returns: Infrastructure object for tasking rules for the production of additional sensors based on base sensor data, and advanced rules for recursive sensor generation based on generated sensors

createEntityFocusedNetwork public static buildingblocks.Network createEntityFocusedNetwork(java.lang.String adaptorName,                                                                 java.lang.String invariant)

A network is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data While the network type of the base sensor is fixed, and the generated sensors also belong to that network type Factory method for creating a new sensing network that is driven by an entity sensor Parameters: adaptorName ­ ­ the network type for the base entity sensor (e.g. "twitter") invariant ­ ­ the entity keyword value that will drive the dynamic production of sensors from the base sensor (e.g., "ebola")

Returns: Network object for tasking rules for the production of additional sensors based on base sensor data

createEntitySensor public static buildingblocks.Sensor createEntitySensor(java.lang.String adaptorName)

A sensor is generated from a network adaptor and is therefore always of that particular network type (eg., twitter) Individual Sensors are of a specific type which is determined by the modality of interest to the developer, i.e., location entity or user Factory method for creating a new sensor of type entity Parameters: adaptorName ­ ­ the network type for this sensor (e.g. "twitter") Returns: Sensor object that should be used for tasking of entity keyword value that the sensor should monitor (e.g., "ebola")

createLocationFocusedInfrastructure public static buildingblocks.Infrastructure createLocationFocusedInfrastructure(java.lang.String baseAdaptorName,                                                                                 java.lang.String invariant)

An Infrastructure is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data Advanced Rules allow sensors to be recursively generated from generated sensor data While the network type of the base sensor is fixed, the generated sensors can be from any network type Factory method for creating a new sensing infrastructure that is driven by an Location sensor Parameters: baseAdaptorName ­ ­ the network type for the base Location sensor (e.g. "twitter" or "worldweather") invariant ­ ­ the locationname value that will drive the dynamic production of sensors from the base sensor (e.g., "ireland")

Returns: Infrastructure object for tasking rules for the production of additional sensors based on base sensor data, and advanced rules for recursive sensor generation based on generated sensors

createLocationFocusedNetwork public static buildingblocks.Network createLocationFocusedNetwork(java.lang.String adaptorName,                                                                   java.lang.String invariant)

A network is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data While the network type of the base sensor is fixed, and the generated sensors also belong to that network type Factory method for creating a new sensing network that is driven by a location sensor Parameters:

167

adaptorName ­ ­ the network type for the base entity sensor (e.g. "twitter")

file:///C:/Users/Olga/Dropbox/thesis/ThesisWorkspace_Final/ie.ucd.sixth.receiver.evaluation/doc/buildingblocks/AbstractApplication.html

3/5

2/24/2015

AbstractApplication

invariant ­ ­ the location name value that will drive the dynamic production of sensors from the base sensor (e.g., "uk") Returns: Network object for tasking rules for the production of additional sensors based on base sensor data

createLocationSensor public static buildingblocks.Sensor createLocationSensor(java.lang.String adaptorName)

A sensor is generated from a network adaptor and is therefore always of that particular network type (eg., twitter) Individual Sensors are of a specific type which is determined by the modality of interest to the developer, i.e., location entity or user Factory method for creating a new sensor of type location Parameters: adaptorName ­ ­ the network type for this sensor (e.g. "weather") Returns: Sensor object that should be used for tasking of location name value that the sensor should monitor (e.g., "ireland")

createPipe public static buildingblocks.Pipe createPipe(java.lang.String adaptorName,                                              java.lang.String pipeType,                                              java.lang.String serviceName)

Pipes are tied to a particular adaptor type depending on the web resource the pipe is interacting with (e.g., klout or alchemy) Pipes that do not interact with an external resource are managed by the 'basic' pipe adaptor (e.g., sensorcreator, filter) Pipes have a specified type according to their functionality (e.g., fuser, filter, sensorcreator) Pipes have a specified name according to their semantic functionality (e.g., sentiment, locationsensorcreator) Parameters: adaptorName ­ eg (klout, alchemy, basic) pipeType ­ eg (fuser, filter, sensorcreator) serviceName ­ eg (locationsensorcreator, sentiment, kloutscore)

Returns: The Pipe object which should be used to configure the pipe settings and to be applied to a sensor

createSensor public static buildingblocks.Sensor createSensor(java.lang.String adaptorName,                                                  java.lang.String sensorType)

Helper method that allows an application to create sensor objects that abstract interactions with SIXTH via DiscoveryReceiver Parameters: adaptorName ­ sensorType ­

Returns: Sensor that should be used for retasking and starting

createUserFocusedInfrastructure public static buildingblocks.Infrastructure createUserFocusedInfrastructure(java.lang.String baseAdaptorName,                                                                             java.lang.String invariant)

An Infrastructure is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base sensor data Advanced Rules allow sensors to be recursively generated from generated sensor data While the network type of the base sensor is fixed, the generated sensors can be from any network type Factory method for creating a new sensing infrastructure that is driven by a user sensor Parameters: baseAdaptorName ­ ­ the network type for the base User sensor (e.g. "twitter") invariant ­ ­ the user id value that will drive the dynamic production of sensors from the base sensor (e.g., "123456")

Returns: Infrastructure object for tasking rules for the production of additional sensors based on base sensor data, and advanced rules for recursive sensor generation based on generated sensors

createUserFocusedNetwork public static buildingblocks.Network createUserFocusedNetwork(java.lang.String adaptorName, 168                                                               java.lang.String invariant)

A network is generated on the basis of a single base sensor and a set of rules that determine which sensors are generated from the base file:///C:/Users/Olga/Dropbox/thesis/ThesisWorkspace_Final/ie.ucd.sixth.receiver.evaluation/doc/buildingblocks/AbstractApplication.html

4/5

2/24/2015

AbstractApplication

sensor data While the network type of the base sensor is fixed, and the generated sensors also belong to that network type Factory method for creating a new sensing network that is driven by a user sensor Parameters: adaptorName ­ ­ the network type for the base entity sensor (e.g. "twitter") invariant ­ ­ the user id value that will drive the dynamic production of sensors from the base sensor (e.g., "1234456")

Returns: Network object for tasking rules for the production of additional sensors based on base sensor data

createUserSensor public static buildingblocks.Sensor createUserSensor(java.lang.String adaptorName)

A sensor is generated from a network adaptor and is therefore always of that particular network type (eg., twitter) Individual Sensors are of a specific type which is determined by the modality of interest to the developer, i.e., location entity or user Factory method for creating a new sensor of type user Parameters: adaptorName ­ ­ the network type for this sensor (e.g. "twitter") Returns: Sensor object that should be used for tasking of user id value that the sensor should monitor (e.g., "123456")

getCredentials public java.lang.String getCredentials()

Specified by: getCredentials in interface ie.ucd.sixth.core.receiver.ICredentialedReceiver

getRequirements public java.util.Map getRequirements(ie.ucd.sixth.core.sensor.NodeDescription desc)

Specified by: getRequirements in interface ie.ucd.sixth.core.application.IApplication

printAvailableInfrastructure public void printAvailableInfrastructure()

helper method for application to print the details of all adaptors currently available in the SIXTH deployment

receive public void receive(ie.ucd.sixth.core.sensor.data.ISensorData data)

Specified by: receive in interface ie.ucd.sixth.core.receiver.IDataReceiver

Parameters: data ­ the data received by the application

respondToData public abstract void respondToData(buildingblocks.Sensor sensor,                                    ie.ucd.sixth.core.sensor.data.ISensorData data)

unregister public void unregister()

Specified by: unregister in interface ie.ucd.sixth.core.receiver.ICredentialedReceiver

Package   Class  Use  Tree  Index  Help   PREV CLASS   NEXT CLASS SUMMARY: NESTED | FIELD | CONSTR | METHOD

169

FRAMES    NO FRAMES    All Classes DETAIL: FIELD | CONSTR | METHOD

file:///C:/Users/Olga/Dropbox/thesis/ThesisWorkspace_Final/ie.ucd.sixth.receiver.evaluation/doc/buildingblocks/AbstractApplication.html

5/5

Appendix E

Evaluation Documentation

170

Introduction  Thank you for participating in this experiment. The purpose of this experiment is to evaluate an  API for creating data driven applications. The API is an interface to a sensor middleware called  SIXTH. SIXTH provides a collection of adaptors that enable access to data sources. The API  enables a developer to access data from these adaptors using three different mechanisms  which you will be asked to use. As such, this experiment is split into three parts. Each part  comprises an introduction to the concept behind a data access mechanism and two tasks which  you will complete using the mechanism. After each part there is a very short survey to complete  and at the end of the experiment there is one last survey in which you will provide feedback for  the overall approach.    The adaptors available in this experiment are listed below. Don’t worry about what they mean for  now, but use this as a reference for completing your tasks.      Social Network:  Adaptor Name: twitter  Sensor Types and parameters: entity (keyword string), location (placename string, user (user id  numeric string)    Weather Service:  Network Adaptor: worldweather  Sensor Types: entity (keyword string), location (placename string)    Reputation Service:  Adaptor Name: klout  type=fuser.  service = Twitter_KloutInfluence  service = Twitter_KloutScore  service = Twitter_KloutTopics    Sentiment Service:  Adaptor Name:: alchemy  type=fuser  service=EntityExtraction  service=keywordExtraction  service=Sentiment             

171

Introduction to Sensors and Pipelines    A Sensor is a configurable software component that monitors the environment it is  programmatically connected to, acquiring context related to users, entities and locations of  interest to an application.    There are three types of sensors that an application can create depending on the context of  interest:    A Location Sensor monitors all information relating to or originating from a geographical  location, from the perspective of the environment which the sensor is programmatically  connected to. For example, a Twitter Location Sensor monitoring “ireland” will produce data  originating from the location “ireland” (the user is in Ireland when they tweet) or related to  “ireland” (perhaps the user is located elsewhere but they are from Ireland). Similarly, a  Weather  Location Sensor can monitor data originating from the location “ireland”.    An Entity Sensor produces information relating to an object, event, or concept, regardless of the  geographical location of the entity. For example, a Twitter Entity Sensor monitoring “ebola” will  produce data relating to ebola from anywhere in the world. A Weather Entity Sensor monitoring  “rain” will produce data relating to “rain” from anywhere in the world.    A User Sensor monitors all information relating to a user, regardless of the users location. For  example, a Twitter User Sensor monitoring you, will produce data relating to all of your  accessible twitter updates including information regarding locations, entities, and other users  relevant to you.    The API at the focus of this experiment enables you to create any of these types of sensors  using the ‘environments’ for which ‘sensor adaptors’ are currently available. For this experiment,  sensor adaptors are available only for Twitter and WorldWeather. The code for creating a Twitter  Entity Sensor monitoring “flu” is as follows:    Sensor mySensor = createEntitySensor(“twitter”);  mySensor.task(“flu”);    A Pipe is a configurable software component that performs some function based on data  produced by a sensor. A Fuser Pipe interacts with a Web Service to produce data that provides  added value to the sensor data. For example, the below code creates a Fuser Pipe using the  ‘Klout’ adaptor. “Twitter_KloutScore” is a specific service offered by Klout that calculates a  reputation metric for a Twitter user. The pipe fuses this reputation information with the sensor  data stream in real time.    Pipe myPipe = createFuserPipe(“Klout”, “Twitter_KloutScore”);   

172

A Pipe is nothing without a sensor stream to act on. The API allows Pipes to be applied to  Sensors as follows:  mySensor.applyPipe(myPipe);    Now that the Sensor is configured it needs to be told to start. The following code starts the  Sensor:    mySensor.start();    For the purposes of this evaluation, you just need to check the console output to see if your  Sensor is working properly. The following message will appear once your sensor is created:    Application received data from [network type: twitter ] [sensor:  ­56288832:sixth.notification]  [values: {sensorType=entity, sensorId=­56288832}]    ­ you will see similar for a pipe being created.    Once your sensor starts sending data to the application you will see something like the following:      Application received data from [network type: twitter ] [sensor:  ­56288832:entity:ebola:Twitter_KloutScore]  [values: {twitter.createdAt=Fri Aug 29  14:59:35 IST 2014, ……      The network type is as you defined when creating the sensor:  [network type: twitter ]   The sensor information [sensor: ­56288832:entity:ebola:Twitter_KloutScore]   can be broken  down as follows:     ­56288832 ­ every sensor has a unique ID  entity ­ the sensor type you created  ebola ­ the ‘invariant’ you configured the sensor to monitor  Twitter_KloutScore ­ a pipe that added information based on the sensor data    The rest of the output details the values of individual values reported by the sensor.    Task 1. Create a WorldWeather Location Sensor to monitor Ireland  Task 2. Create a Twitter Entity Sensor to monitor Ebola on Twitter  Task 3. Create an ‘alchemy’ Fuser Pipe to monitor Sentiment of Ebola data    Start your sensors and review the output. When you are finished please complete the short  survey that follows.    

173

  Sensor and Pipeline Survey:        Q1. To what degree do you feel you completed the tasks?    (not at all) 1 2 3 4 5  (completely)      Q2. To what degree do you feel you understand the Sensor concept?     (not at all) 1 2 3 4 5  (completely)      Q3. To what degree do you feel you understand the Pipe concept?    (not at all) 1 2 3 4 5  (completely)    Q4. To what degree do you feel you could apply the Sensor Pipeline concept to other problems?    (not at all) 1 2 3 4 5  (completely)      Q.5 What, if anything, do you feel is missing from the sensor concept?     ____________________________________________________________________      ____________________________________________________________________    Q.6 What, if anything, do you feel is missing from the pipe concept?    ____________________________________________________________________      ____________________________________________________________________             

174

Introduction to Sensor Networks    A Network is a configurable software component that generates a series of sensors based on  data produced by a base sensor. A Network is grounded by a particular Sensor type depending  on the interests of the application. The produced sensors all belong to the same network type.  For example, a Twitter Network can only produce Twitter Entity, Twitter Location and Twitter  User Sensors.     An Entity Focused Network generates sensors based on the sensor data produced by its  Base Entity Sensor.      A Location Focused Network generates sensors based on the sensor data produced by its  Base Location Sensor.     A User Focused Network generates sensors based on the sensor data produced by its Base  User Sensor.    Each of these Networks can be configured to generate all Sensor types belonging to the same  Network. For example. The below example using the API generates User, Entity and Location  Sensors from a base user sensor:    Network sampleNetwork = createUserFocusedNetwork("twitter", "19877440");  sampleNetwork.generateEntities();  sampleNetwork.generateLocations();  sampleNetwork.start();    This example creates a base Twitter User Sensor to monitor the Twitter User with id 19877440.  The network is tasked to create Entity and Location Sensors based on the original User Sensor.  This means, the network will produce sensors that monitor all entities of interest to the user, all  locations of relevance to the user, while the sensor is active.    As with Sensors in the earlier task, a  network is activated by calling the start() method.    Task 4. Create a Twitter Network to produce user and location sensors related to the entity  ‘ebola’    Task 5. Create a WorldWeather Network to produce location sensors for all locations related to  the entity ‘rain’     Start your sensors and review the output. When you are finished please complete the short  survey that follows.      

175

  Network Survey:    Q1. To what degree do you feel you completed the tasks?    (not at all) 1 2 3 4 5  (completely)      Q2. To what degree do you feel you understand the Network concept?     (not at all) 1 2 3 4 5  (completely)      Q3. To what degree do you feel you could apply the Network concept to other problems?    (not at all) 1 2 3 4 5  (completely)      Q.5 What, if anything, do you feel is missing from the Network concept?     ____________________________________________________________________      ____________________________________________________________________                                  Introduction to Infrastructures   

176

An Infrastructure is a configurable software component that generates a series of sensors  based on data produced by a base sensor. An Infrastructure is grounded by a particular Sensor  type, from a defined Network type, depending on the interests of the application. Infrastructures  differ from Networks in that the produced sensors may belong to any available network type.  For  example, an Infrastructure with a Twitter Entity Sensor as its base, can produce Twitter  Sensors, WorldWeather Sensors, and Sensors from any other Network type available.     An Entity Focussed Infrastructure generates sensors based on the sensor data produced by  its Base Entity Sensor.      A Location Focussed Infrastructure generates sensors based on the sensor data produced  by its Base Location Sensor.     A User Focussed Infrastructure generates sensors based on the sensor data produced by its  Base User Sensor.    The sensor production rules for an Infrastructure are the same as for a Network, except, the  network type can be defined. When a network type is not defined, the Infrastructure will ensure  the Sensor Type from all available networks is generated. For example, the below code using the  API creates an Entity Focussed Infrastructure for which the base sensor is a Twitter Entity  Sensor monitoring rain. From this base sensor, Location Sensors from all available adaptors are  created. In this case, these are WorldWeather Location and Twitter Location Sensors:    Infrastructure sampleEntityInfrastructure = createEntityFocusedInfrastructure("twitter", "rain");   sampleEntityInfrastructure.generateLocations();    sampleEntityInfrastructure.start();    Alternatively, the next example has been configured to only generate WorldWeather Location  Sensors based on the Twitter Entity Sensor monitoring rain. This is acheived by passing the  name of the WorldWeather network as an argument to the generateLocations rule:    Infrastructure sampleEntityInfrastructure = createEntityFocusedInfrastructure("twitter", "rain");   sampleEntityInfrastructure.generateLocations(“worldweather”);    sampleEntityInfrastructure.start();    Infrastructures have an advanced feature to enable recursive sensor generation rules. These  methods allow you to generate sensor types based on generated sensor types as in the  example below:    Infrastructure sampleEntityInfrastructure = createEntityFocusedInfrastructure("worldweather",  "snow");  sampleEntityInfrastructure.generateLocations("twitter");   sampleEntityInfrastructure.generateEntitiesFromLocations(“twitter”);  

177

sampleEntityInfrastructure.start();        Task 6. Create an Infrastructure to produce WorldWeather Location sensors based on a Twitter  Entity Sensor monitoring ‘flu’.    Task 7. Create an Infrastructure to monitor all users in countries where it is raining. You will  need to monitor ‘rain’ from worldweather and produce Twitter location sensors accordingly.  From this you can generate user sensors.    Start your Infrastructures and review the output. When you are finished please complete the  short survey that follows.                                                     Infrastructure Survey:        Q1. To what degree do you feel you completed the tasks? 

178

  (not at all) 1 2 3 4 5  (completely)      Q2. To what degree do you feel you understand the Infrastructure concept?     (not at all) 1 2 3 4 5  (completely)      Q3. To what degree do you feel you could apply the Infrastructure concept to other problems?    (not at all) 1 2 3 4 5  (completely)      Q.5 What, if anything, do you feel is missing from the Infrastructure concept?     ____________________________________________________________________      ____________________________________________________________________                               

System Usability Scale         © Digital Equipment Corporation, 1986.   

179

   1. I think that I would like to use this system frequently    (strongly disagree) 1 2 3 4 5  (strongly agree)      2. I found the system unnecessarily complex  (strongly disagree) 1 2 3 4 5  (strongly agree)      3. I thought the system was easy  to use     (strongly disagree) 1 2 3 4 5  (strongly agree)      4. I think that I would need the support of a technical person to be able to use this  system     (strongly disagree) 1 2 3 4 5  (strongly agree)     5. I found the various functions in  this system were well integrated  (strongly disagree) 1 2 3 4 5  (strongly agree)       6. I thought there was too much inconsistency in this system  (strongly disagree) 1 2 3 4 5  (strongly agree)      7. I would imagine that most people would learn to use this system     very quickly    (strongly disagree) 1 2 3 4 5  (strongly agree)     8. I found the system very cumbersome to use  (strongly disagree) 1 2 3 4 5  (strongly agree)     9. I felt very confident using the  system  (strongly disagree) 1 2 3 4 5  (strongly agree)       10. I needed to learn a lot of things before I could get going with this system   (strongly disagree) 1 2 3 4 5  (strongly agree)         EXPERTISE:    JAVA: (never used) 0  1 2 3 4 5  (expert)    WEB APIS: (never used) 0  1 2 3 4 5  (expert)    SENSOR NETWORKS: (never used) 0  1 2 3 4 5 (expert) 

180

 

 

  MIDDLEWARE:(never used)0        Additional comments:   

1

2

3

181

4

5 

(expert) 

Bibliography [1]

Abdelzaher, Tarek et al. “Envirotrack: Towards an environmental computing paradigm for distributed sensor networks”. In: Distributed computing systems, 2004. proceedings. 24th international conference on. IEEE. 2004, pp. 582–589.

[2]

Aberer, Karl, Hauswirth, Manfred, and Salehi, Ali. “A middleware for fast and flexible sensor network deployment”. In: Proceedings of the 32nd international conference on Very large data bases. VLDB Endowment. 2006, pp. 1199–1202.

[3]

Aberer, Karl, Hauswirth, Manfred, and Salehi, Ali. “Infrastructure for dataprocessing in large-scale interconnected sensor networks”. In: Mobile Data Management, 2007 International Conference on. IEEE. 2007, pp. 198–205.

[4]

Aberer, Karl, Hauswirth, Manfred, and Salehi, Ali. The Global Sensor Networks middleware for efficient and flexible deployment and interconnection of sensor networks. Tech. rep. 2006.

[5]

Abowd, Gregory D and Mynatt, Elizabeth D. “Charting past, present, and future research in Ubiquitous Computing”. In: ACM Transactions on Computer-Human Interaction (TOCHI) 7.1 (2000), pp. 29–58.

[6]

Abowd, Gregory D et al. “Towards a better understanding of context and contextawareness”. In: Handheld and Ubiquitous Computing. Springer. 1999, pp. 304–307.

[7]

Aghaei, Sareh, Nematbakhsh, Mohammad Ali, and Farsani, Hadi Khosravi. “Evolution of the world wide web: From WEB 1.0 TO WEB 4.0”. In: International Journal of Web and Semantic Technology 3.1 (2012), pp. 1–10.

[8]

Akribopoulos, Orestis et al. “A web services-oriented architecture for integrating small programmable objects in the web of things”. In: Developments in E-systems Engineering (DESE), 2010. IEEE. 2010, pp. 70–75.

[9]

Akyildiz, Ian F et al. “A survey on sensor networks”. In: Communications magazine, IEEE 40.8 (2002), pp. 102–114.

182

[10]

Albakour, M, Macdonald, Craig, Ounis, Iadh, et al. “Identifying local events by using microblogs as social sensors”. In: Proceedings of the 10th Conference on Open Research Areas in Information Retrieval. Le Centre De Hautes Etudes Internationales D’Informatique Documantaire. 2013, pp. 173–180.

[11]

Altinel, Mehmet et al. “Damia: a data mashup fabric for intranet applications”. In: Proceedings of the 33rd international conference on Very large data bases. VLDB Endowment. 2007, pp. 1370–1373.

[12]

Ashton, Kevin. “That internet of things thing”. In: RFiD Journal 22.7 (2009), pp. 97–114.

[13]

Atzori, Luigi, Iera, Antonio, and Morabito, Giacomo. “The internet of things: A survey”. In: Computer networks 54.15 (2010), pp. 2787–2805.

[14]

Bai, Lan S, Dick, Robert P, and Dinda, Peter A. “Archetype-based design: Sensor network programming for application experts, not just programming experts”. In: Proceedings of the 2009 International Conference on Information Processing in Sensor Networks. IEEE Computer Society. 2009, pp. 85–96.

[15]

Balazinska, Magdalena et al. “Data management in the worldwide sensor web”. In: IEEE Pervasive Computing 6.2 (2007), pp. 30–40.

[16]

Baldauf, Matthias, Dustdar, Schahram, and Rosenberg, Florian. “A survey on context-aware systems”. In: International Journal of Ad Hoc and Ubiquitous Computing 2.4 (2007), pp. 263–277.

[17]

Bangor, Aaron, Kortum, Philip T, and Miller, James T. “An empirical evaluation of the system usability scale”. In: Intl. Journal of Human–Computer Interaction 24.6 (2008), pp. 574–594.

[18]

Bangor, Aaron, Kortum, Philip, and Miller, James. “Determining what individual SUS scores mean: Adding an adjective rating scale”. In: Journal of usability studies 4.3 (2009), pp. 114–123.

[19]

Baqer, M and Kamal, Adel. “S-sensors: Integrating physical world inputs with social networks using Wireless Sensor Networks”. In: Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), 2009 5th International Conference on. IEEE. 2009, pp. 213–218.

[20]

Beach, Aaron et al. “Fusing mobile, sensor, and social data to fully enable ContextAware Computing”. In: Proceedings of the Eleventh Workshop on Mobile Computing Systems and Applications. ACM. 2010, pp. 60–65.

[21]

Belanger, F et al. “Taxonomy and success metrics of online websites”. In: Communications of the ACM 49.12 (2006), pp. 114–116.

183

[22]

Benatallah, Boualem et al. “Declarative composition and peer-to-peer provisioning of dynamic web services”. In: Data Engineering, 2002. Proceedings. 18th International Conference on. IEEE. 2002, pp. 297–308.

[23]

Benevenuto, Fabricio et al. “Characterizing user behavior in online social networks”. In: Proceedings of the 9th ACM SIGCOMM conference on Internet measurement conference. ACM. 2009, pp. 49–62.

[24]

Bernardi, Ansgar et al. “The social semantic desktop: a new paradigm towards deploying the semantic web on the desktop”. In: Semantic Web Engineering in the Knowledge Society. IGI Global (2008).

[25]

Berners-Lee, Tim, Hendler, James, Lassila, Ora, et al. “The semantic web”. In: Scientific american 284.5 (2001), pp. 28–37.

[26]

Bosak, Jon, Bray, Tim, et al. “XML and the second-generation Web”. In: Scientific American 280.5 (1999), pp. 89–93.

[27]

Botts, Mike et al. “OGC sensor web enablement: Overview and high level architecture”. In: GeoSensor networks. Springer, 2008, pp. 175–190.

[28]

Boyd, Danah and Crawford, Kate. “Six Provocations for Big Data”. In: Computer 123.1 (2011), pp. 1–17. issn: 15565068.

[29]

Breslin, John et al. “Integrating Social Networks and Sensor Networks”. In: Social Networks January (2009).

[30]

Brooke, John. “SUS: a retrospective”. In: Journal of Usability Studies 8.2 (2013), pp. 29–40.

[31]

Broring, Arne et al. “New generation sensor web enablement”. In: Sensors 11.3 (2011), pp. 2652–2699.

[32]

Buchegger, Sonja et al. “PeerSoN: P2P social networking: early experiences and insights”. In: Proceedings of the Second ACM EuroSys Workshop on Social Network Systems. ACM. 2009, pp. 46–52.

[33]

Burstein, Mark et al. “A semantic web services architecture”. In: IEEE Internet Computing 9.5 (2005), pp. 72–81.

[34]

Castillo-Effer, M et al. “Wireless Sensor Networks for flash-flood alerting”. In: Devices, Circuits and Systems, 2004. Proceedings of the Fifth IEEE International Caracas Conference on. Vol. 1. IEEE. 2004, pp. 142–146.

[35]

Chi, Ed H. “The social web: Research and opportunities”. In: Computer 9 (2008), pp. 88–91.

[36]

Chui, Michael, Loffler, Markus, and Roberts, Roger. “The internet of things”. In: McKinsey Quarterly 2.2010 (2010), pp. 1–9. 184

[37]

Conti, Marco et al. “Looking ahead in pervasive computing: Challenges and opportunities in the era of cyber-physical convergence”. In: Pervasive and Mobile Computing 8.1 (2012), pp. 2–21.

[38]

Correia, Nuno, Meneses, Filipe, and Jose, Rui. “SPOTSENSING a platform for social web sensing”. In: (2011).

[39]

Coutaz, Joelle et al. “Context is key”. In: Communications of the ACM 48.3 (2005), pp. 49–53.

[40]

Darianian, Mohsen and Michael, Martin Peter. “Smart home mobile RFID-based Internet-of-Things systems and services”. In: Advanced Computer Theory and Engineering, 2008. ICACTE’08. International Conference on. IEEE. 2008, pp. 116– 120.

[41]

De Kool, Dennis and Van Wamelen, Johan. “Web 2.0: a new basis for egovernment?” In: Information and Communication Technologies: From Theory to Applications, 2008. ICTTA 2008. 3rd International Conference on. IEEE. 2008, pp. 1–7.

[42]

Decker, Stefan and Frank, Martin. “The social semantic desktop”. In: WWW2004 Workshop Application Design, Development and Implementation Issues in the Semantic Web. Vol. 9. 2004, p. 10.

[43]

Delin, Kevin A. “The Sensor Web: A macro-instrument for coordinated sensing”. In: Sensors 2.7 (2002), pp. 270–285.

[44]

Demirbas, Murat et al. “Crowd-sourced sensing and collaboration using twitter”. In: World of Wireless Mobile and Multimedia Networks (WoWMoM), 2010 IEEE International Symposium. IEEE. 2010, pp. 1–9.

[45]

Desai, Pratikkumar. “A Semantic Situation Awareness Framework for Indoor Cyber-Physical Systems”. PhD thesis. Wright State University, 2013.

[46]

Dey, Anind K. “Understanding and using context”. In: Personal and Ubiquitous Computing 5.1 (2001), pp. 4–7.

[47]

Dobson, Simon et al. “Construct: An Open Source Pervasive Systems Platform”. In: Consumer Communications and Networking Conference (CCNC), 2007, 4th IEEE, Las Vegas, NV, USA. Jan. 2007, pp. 1203–1204. isbn: 1-4244-0667-6.

[48]

Duquennoy, Simon, Grimaud, Gilles, and Vandewalle, J-J. “The Web of Things: interconnecting devices with high usability and performance”. In: Embedded Software and Systems, 2009. ICESS’09. International Conference on. IEEE. 2009, pp. 323– 330.

185

[49]

Eisenhauer, Markus, Rosengren, Peter, and Antolin, Pablo. “Hydra: A development platform for integrating wireless devices and sensors into ambient intelligence systems”. In: The Internet of Things. Springer, 2010, pp. 367–373.

[50]

Ellison, Nicole B et al. “Social network sites: Definition, history, and scholarship”. In: Journal of Computer-Mediated Communication 13.1 (2007), pp. 210–230.

[51]

Endler, Markus et al. “Defining situated social context for pervasive social computing”. In: Pervasive Computing and Communications Workshops (PERCOM Workshops), 2011 IEEE International Conference on. IEEE. 2011, pp. 519–524.

[52]

Evans, Dave. Social media marketing: the next generation of business engagement. John Wiley and Sons, 2010.

[53]

Farooqi, Ashfaq Hussain and Khan, Farrukh Aslam. “Intrusion detection systems for Wireless Sensor Networks: A survey”. In: Communication and networking. Springer, 2009, pp. 234–241.

[54]

George, Darren. SPSS for windows step by step: A simple study guide and reference, 17.0 update. Pearson Education India, 2003.

[55]

Ghosh, Saptarshi et al. “Cognos: crowdsourcing search for topic experts in microblogs”. In: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. ACM. 2012, pp. 575–590.

[56]

Grammel, Lars and Storey, Margaret-Anne. “A survey of mashup development environments”. In: The smart internet. Springer, 2010, pp. 137–151.

[57]

Greaves, Mark. “Semantic Web 2.0”. In: IEEE Intelligent Systems 2 (2007), pp. 94– 96.

[58]

Greenfield, Adam. Everyware: The dawning age of Ubiquitous Computing. New Riders, 2010.

[59]

Groza, Tudor et al. “The NEPOMUK Project - On the way to the Social Semantic Desktop”. In: Proceedings of I-Semantics’ 07 (2007), pp. 201–211.

[60]

Gruber, Tom. “Collective knowledge systems: Where the social web meets the semantic web”. In: Web semantics: science, services and agents on the World Wide Web 6.1 (2008), pp. 4–13.

[61]

Guinard, Dominique and Trifa, Vlad. “Towards the web of things: Web mashups for embedded devices”. In: Workshop on Mashups, Enterprise Mashups and Lightweight Composition on the Web (MEM 2009), in proceedings of WWW (International World Wide Web Conferences), Madrid, Spain. 2009, p. 15.

186

[62]

Guinard, Dominique et al. “Interacting with the soa-based internet of things: Discovery, query, selection, and on-demand provisioning of web services”. In: Services Computing, IEEE Transactions on 3.3 (2010), pp. 223–235.

[63]

Guinard, Dominique et al. “Towards physical mashups in the web of things”. In: Networked Sensing Systems (INSS), 2009 Sixth International Conference on. IEEE. 2009, pp. 1–4.

[64]

Gummadi, Ramakrishna, Gnawali, Omprakash, and Govindan, Ramesh. “Macroprogramming Wireless Sensor Networks using Kairos”. In: Distributed Computing in Sensor Systems. Springer, 2005, pp. 126–140.

[65]

Guo, Bin, Zhang, Daqing, and Imai, Michita. “Toward a cooperative programming framework for context-aware applications”. In: Personal and Ubiquitous Computing 15.3 (2011), pp. 221–233.

[66]

Hadim, Salem and Mohamed, Nader. “Middleware: Middleware challenges and approaches for Wireless Sensor Networks”. In: IEEE distributed systems online 3 (2006), p. 1.

[67]

Haller, Armin et al. “Wsmx-a semantic service-oriented architecture”. In: Web Services, 2005. ICWS 2005. Proceedings. 2005 IEEE International Conference on. IEEE. 2005, pp. 321–328.

[68]

Harper, Christopher. “Journalism in a digital age”. In: Health 34.27 (2005), p. 29.

[69]

Heinzelman, Wendi B et al. “Middleware to support sensor network applications”. In: Network, IEEE 18.1 (2004), pp. 6–14.

[70]

Hendler, Jim. “Web 3.0 Emerging”. In: Computer 42.1 (2009), pp. 111–113.

[71]

Hendler, Jim and Berners-Lee, Tim. “From the Semantic Web to social machines: A research challenge for AI on the World Wide Web”. In: Artificial Intelligence 174.2 (2010), pp. 156–161.

[72]

Honkola, Jukka et al. “Smart-M3 information sharing platform.” In: ISCC. 2010, pp. 1041–1046.

[73]

Hossmann, Theus et al. “Twitter in disaster mode: Opportunistic communication and distribution of sensor data in emergencies”. In: Proceedings of the 3rd Extreme Conference on Communication: The Amazon Expedition. ACM. 2011, p. 1.

[74]

Hsieh, C et al. “Experts vs the crowd: Examining popular news prediction performance on twitter”. In: Proceedings of the WWW13 conference, Rio de Janeiro. 2013.

187

[75]

Huang, Jiangmiao, Zhao, Hui, and Zhang, Jie. “Detecting Flu Transmission by Social Sensor in China”. In: Green Computing and Communications (GreenCom), 2013 IEEE and Internet of Things (iThings CPSCom). IEEE International Conference on, IEEE Cyber, Physical, and Social Computing. 2013, pp. 1242–1247.

[76]

Huberman, Bernardo A, Romero, Daniel M, and Wu, Fang. “Social networks that matter: Twitter under the microscope”. In: Available at SSRN 1313405 (2008).

[77]

Hui, Glenn and Hayllar, Mark Richard. “Creating Public Value in E-Government: A Public-Private-Citizen Collaboration Framework in Web 2.0”. In: Australian Journal of Public Administration 69.s1 (2010), S120–S131.

[78]

Humphreys, Lee. “Mobile social networks and social practice: A case study of Dodgeball”. In: Journal of Computer-Mediated Communication 13.1 (2007), pp. 341–360.

[79]

So-In, Chakchai et al. “Mobile Animal Tracking Systems using light sensor for efficient power and cost saving motion detection”. In: Communication Systems, Networks and Digital Signal Processing (CSNDSP), 2012 8th International Symposium on. IEEE. 2012, pp. 1–6.

[80]

Issarny, Valerie et al. “Erratum to: Service-oriented middleware for the Future Internet: state of the art and research directions”. In: Journal of Internet Services and Applications 2.3 (2011), pp. 271–271.

[81]

Jain, Raj. “Internet 3.0: ten problems with current internet architecture and solutions for the next generation”. In: Military Communications Conference, 2006. MILCOM 2006. IEEE. IEEE. 2006, pp. 1–9.

[82]

Jain, Vishwas Raj et al. “wildcense: Gps based animal tracking system”. In: Intelligent Sensors, Sensor Networks and Information Processing, 2008. ISSNIP 2008. International Conference. IEEE. 2008, pp. 617–622.

[83]

Jazayeri, Mehdi. “Some trends in web application development”. In: Future of Software Engineering, 2007. FOSE’07. IEEE. 2007, pp. 199–213.

[84]

Jones, M Cameron and Churchill, Elizabeth F. “Conversations in developer communities: a preliminary analysis of the yahoo! Pipes community”. In: Proceedings of the fourth international conference on Communities and technologies. ACM. 2009, pp. 195–204.

[85]

Kabadayi, Sanem, Pridgen, Adam, and Julien, Christine. “Virtual sensors: Abstracting data from physical sensors”. In: Proceedings of the 2006 International Symposium on on World of Wireless, Mobile and Multimedia Networks. IEEE Computer Society. 2006, pp. 587–592.

188

[86]

Kambil, Ajit. “What is your Web 5.0 strategy?” In: Journal of business strategy 29.6 (2008), pp. 56–58.

[87]

Katasonov, Artem et al. “Smart Semantic Middleware for the Internet of Things.” In: ICINCO-ICSO 8 (2008), pp. 169–178.

[88]

Kawamoto, Kevin. Digital journalism: Emerging media and the changing horizons of journalism. Rowman and Littlefield Publishers, 2003.

[89]

Khedo, Kavi K, Perseedoss, Rajiv, Mungur, Avinash, et al. “A wireless sensor network air pollution monitoring system”. In: arXiv preprint arXiv:1005.1737 (2010).

[90]

Kim, Won, Jeong, Ok-Ran, and Lee, Sang-Won. “On social Web sites”. In: Information systems 35.2 (2010), pp. 215–236.

[91]

Kleis Nielsen, Rasmus. “The Story So Far: what we know about the business of digital journalism”. In: Digital Journalism 1.1 (2013), pp. 174–175.

[92]

Kortuem, Gerd et al. “Smart objects as building blocks for the internet of things”. In: Internet Computing, IEEE 14.1 (2010), pp. 44–51.

[93]

Kranz, Matthias, Roalter, Luis, and Michahelles, Florian. “Things that twitter: social networks and the internet of things”. In: What can the Internet of Things do for the Citizen (CIoT) Workshop at The Eighth International Conference on Pervasive Computing (Pervasive 2010). 2010, pp. 1–10.

[94]

Kushwaha, Manish et al. “OASIS: A programming framework for service-oriented sensor networks”. In: Communication systems software and middleware, 2007. comsware 2007. 2nd international conference on. IEEE. 2007, pp. 1–8.

[95]

Lampos, Vasileios and Cristianini, Nello. “Tracking the flu pandemic by monitoring the social web”. In: Cognitive Information Processing (CIP), 2010 2nd International Workshop on. IEEE. 2010, pp. 411–416.

[96]

Lassila, Ora and Hendler, James. “Embracing Web 3.0”. In: Internet Computing, IEEE 11.3 (2007), pp. 90–93.

[97]

Laukkarinen, Teemu, Suhonen, Jukka, and Hannikainen, Marko. “A survey of wireless sensor network abstraction for application development”. In: International Journal of Distributed Sensor Networks 2012 (2012).

[98]

Le Roy, Bertrand C, Sanabria, Andres, and Calvert, Simon. Peer to peer social networking. US Patent App. 12 139,342. June 2008.

[99]

Lee, Cheng-Jung et al. “Toward a new paradigm: Mashup patterns in web 2.0”. In: WSEAS Transactions on Information Science and Applications 6.10 (2009), pp. 1675–1686.

189

[100]

Lee, Kathy, Agrawal, Ankit, and Choudhary, Alok. “Real-time disease surveillance using twitter data: demonstration on flu and cancer”. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM. 2013, pp. 1474–1477.

[101]

Levis, Philip Alexander et al. Trickle: A self regulating algorithm for code propagation and maintenance in Wireless Sensor Networks. Computer Science Division, University of California, 2003.

[102]

Liao, Q Vera et al. “Understanding experts’ and novices’ expertise judgment of twitter users”. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM. 2012, pp. 2461–2464.

[103]

Linda, Ondrej et al. “Fuzzy logic based anomaly detection for embedded network security cyber sensor”. In: Computational Intelligence in Cyber Security (CICS), 2011 IEEE Symposium on. IEEE. 2011, pp. 202–209.

[104]

Linda, Ondrej et al. “Towards resilient critical infrastructures: Application of Type2 Fuzzy Logic in embedded network security cyber sensor”. In: Resilient Control Systems (ISRCS), 2011 4th International Symposium on. IEEE. 2011, pp. 26–32.

[105]

Liu, Zhong et al. “Cyber-physical-social systems for command and control”. In: IEEE Intelligent Systems 26.4 (2011), pp. 92–96.

[106]

Lu, Chenyang et al. “Rap: A real-time communication architecture for large-scale Wireless Sensor Networks”. In: Real-Time and Embedded Technology and Applications Symposium, 2002. Proceedings. Eighth IEEE. IEEE. 2002, pp. 55–66.

[107]

Luo, Liqian et al. “Envirosuite: An environmentally immersive programming framework for sensor networks”. In: ACM Transactions on Embedded Computing Systems (TECS) 5.3 (2006), pp. 543–576.

[108]

Lynch, Clifford. “Big data: How do your data grow?” In: Nature 455.7209 (2008), pp. 28–29.

[109]

Mainetti, Luca, Patrono, Luigi, and Vilei, Antonio. “Evolution of Wireless Sensor Networks towards the internet of things: A survey”. In: Software, Telecommunications and Computer Networks (SoftCOM), 2011 19th International Conference on. IEEE. 2011, pp. 1–6.

[110]

Manyika, James et al. “Big data: The next frontier for innovation, competition, and productivity”. In: (2011).

[111]

Marsh, David et al. “Autonomic Wireless Sensor Networks”. In: Engineering Applications of Artificial Intelligence 17.7 (2004), pp. 741–748.

190

[112]

Maximilien, E Michael, Ranabahu, Ajith, and Gomadam, Karthik. “An online platform for web apis and service mashups”. In: Internet Computing, IEEE 12.5 (2008), pp. 32–43.

[113]

Maximilien, E Michael et al. A domain-specific language for web apis and services mashups. Springer, 2007.

[114]

McCooI, R. “Rethinking the semantic web. Part 2”. In: Internet Computing, IEEE 10.1 (2006), pp. 93–96.

[115]

McCool, Rob. “Rethinking the semantic web. Part I”. In: Internet Computing, IEEE 9.6 (2005), pp. 88–86.

[116]

Mikroyannidis, Alexander. “Toward a social semantic web”. In: Computer 40.11 (2007), pp. 113–115.

[117]

Milanovic, Nikola and Malek, Miroslaw. “Current solutions for web service composition”. In: IEEE Internet Computing 8.6 (2004), pp. 51–59.

[118]

Mottola, Luca and Picco, Gian Pietro. “Middleware for Wireless Sensor Networks: an outlook”. In: Journal of Internet Services and Applications 3.1 (2012), pp. 31– 39.

[119]

Mottola, Luca and Picco, Gian Pietro. “Programming Wireless Sensor Networks with logical neighborhoods”. In: Proceedings of the first international conference on Integrated internet ad hoc and sensor networks. ACM. 2006, p. 8.

[120]

Mueller, Paul and Reuther, Bernd. “Future Internet Architecture-A Service Oriented Approach”. In: it-Information Technology Methoden und innovative Anwendungen der Informatik und Informationstechnik 50.6 (2009), pp. 383–389.

[121]

Murdoch, Olga et al. “Reflecting on the Ambient Intelligence Vision: A CyberPhysical-Social Perspective”. In: Fourth International Symposium on Business Modelling and Software Design (2014).

[122]

Murugesan, San. “Understanding Web 2.0”. In: IT professional 9.4 (2007), pp. 34– 41.

[123]

Ng, Joanna W, Chignell, Mark, and Cordy, James R. “The smart internet: transforming the web for the user”. In: Proceedings of the 2009 Conference of the Center for Advanced Studies on Collaborative Research. IBM Corp. 2009, pp. 285–296.

[124]

O’Hare, Gregory M. P. et al. “Sensor Web Interaction”. In: International Journal on Artificial Intelligence Tools 21.02 (2012), p. 1240006. issn: 0218-2130.

[125]

Olguin, Daniel Olguin and Pentland, Alex Sandy. “Social sensors for automatic data collection”. In: AMCIS 2008 Proceedings (2008), p. 171.

[126]

O’Reilly, Tim. What is web 2.0. ” O’Reilly Media, Inc.”, 2009. 191

[127]

Osimo, David. “Benchmarking eGovernment in the Web 2.0 era: what to measure, and how”. In: European Journal of ePractice 4 (2008), pp. 33–43.

[128]

Pan, Jianli, Paul, Subharthi, and Jain, Raj. “A survey of the research on future internet architectures”. In: Communications Magazine, IEEE 49.7 (2011), pp. 26– 36.

[129]

Paulino, Herve and Santos, Joao Ruivo. “A middleware framework for the web integration of sensor networks”. In: Sensor Systems and Software. Springer, 2011, pp. 75–90.

[130]

Le-Phuoc, Danh et al. “The linked sensor middleware–connecting the real world and the semantic web”. In: Proceedings of the Semantic Web Challenge (2011).

[131]

Picco, Gian Pietro. “Software engineering and Wireless Sensor Networks: happy marriage or consensual divorce?” In: Proceedings of the FSE SDP workshop on Future of software engineering research. ACM. 2010, pp. 283–286.

[132]

Poovendran, RADHA. “Cyber-physical systems: Close encounters between two parallel worlds [Point of view]”. In: Proceedings of the IEEE 98.8 (2010), pp. 1363– 1366.

[133]

Prekop, Paul and Burnett, Mark. “Activities, context and Ubiquitous Computing”. In: Computer Communications 26.11 (2003), pp. 1168–1176.

[134]

Rahman, Md Abdur, El Saddik, Abdulmotaleb, and Gueaieb, Wail. “Data visualization: From body sensor network to social networks”. In: Robotic and Sensors Environments, 2009. ROSE 2009. IEEE International Workshop on. IEEE. 2009, pp. 157–162.

[135]

Rajkumar, Ragunathan Raj et al. “Cyber-physical systems: the next computing revolution”. In: Proceedings of the 47th Design Automation Conference. ACM. 2010, pp. 731–736.

[136]

Richard, WM. “A sensor classification scheme”. In: IEEE Trans. Ultrason. Ferroelectr. Freq. Control, UFFC-34 2 (1987).

[137]

Robertson, Judy. “Likert-type scales, statistical methods, and effect sizes”. In: Communications of the ACM 55.5 (2012), pp. 6–7.

[138]

Rogers, Yvonne. “Moving on from weiser’s vision of calm computing: Engaging ubicomp experiences”. In: UbiComp 2006: Ubiquitous Computing. Springer, 2006, pp. 404–421.

[139]

Rosi, Alberto et al. “Social sensors and pervasive services: Approaches and perspectives”. In: Pervasive Computing and Communications Workshops (PERCOM Workshops), 2011 IEEE International Conference on. IEEE. 2011, pp. 525–530.

192

[140]

Rouached, Mohsen, Baccar, Sana, and Abid, Mohamed. “RESTful sensor web enablement services for Wireless Sensor Networks”. In: Services (SERVICES), 2012 IEEE Eighth World Congress on. IEEE. 2012, pp. 65–72.

[141]

Sadri, Fariba. “Ambient intelligence: A survey”. In: ACM Computing Surveys (CSUR) 43.4 (2011), p. 36.

[142]

Saha, Suman and Matsumoto, Mitsuji. “A framework for data collection and wireless sensor network protocol for disaster management”. In: Communication Systems Software and Middleware, 2007. COMSWARE 2007. 2nd International Conference on. IEEE. 2007, pp. 1–6.

[143]

Sakaki, Takeshi, Okazaki, Makoto, and Matsuo, Yutaka. “Earthquake shakes Twitter users: real-time event detection by social sensors”. In: Proceedings of the 19th international conference on World wide web. ACM. 2010, pp. 851–860.

[144]

Sauro, J. “Measuring Usability With The System Usability Scale (SUS)”. In: Measuring Usability (2011), pp. 1–5.

[145]

Schaupp, L Christian, Fan, Weiguo, and Belanger, France. “Determining success for different website goals”. In: System Sciences, 2006. HICSS’06. Proceedings of the 39th Annual Hawaii International Conference on. Vol. 6. IEEE. 2006, 107b–107b.

[146]

Schneier, Bruce. “A taxonomy of social networking data”. In: Security and Privacy, IEEE 8.4 (2010), pp. 88–88.

[147]

Schuster, Daniel et al. “Pervasive social context-taxonomy and survey”. In: ACM Transactions on Intelligent Systems and Technology (TIST) (2012).

[148]

Scotch, Matthew, Yip, Kevin Y, and Cheung, Kei-Hoi. “Development of grid-like applications for public health using Web 2.0 mashup techniques”. In: Journal of the American Medical Informatics Association 15.6 (2008), pp. 783–786.

[149]

Scott, Ben. “A contemporary history of digital journalism”. In: Television and New Media 6.1 (2005), pp. 89–126.

[150]

Sheng, Quan Z et al. “SELF-SERV: a platform for rapid composition of web services in a peer-to-peer environment”. In: Proceedings of the 28th international conference on Very Large Data Bases. VLDB Endowment. 2002, pp. 1051–1054.

[151]

Sheth, Amit P. “Citizen sensing, social signals, and enriching human experience”. In: IEEE Internet Computing 13.4 (2009), p. 87.

[152]

Sheth, Amit and Anantharam, Pramod. “Physical cyber social computing for human experience”. In: Proceedings of the 3rd International Conference on Web Intelligence, Mining and Semantics. ACM. 2013, p. 1.

193

[153]

Sheth, Amit, Anantharam, Pramod, and Henson, Cory. “Physical-cyber-social computing: An early 21st century approach”. In: Intelligent Systems, IEEE 28.1 (2013), pp. 78–82.

[154]

Shi, Jianhua et al. “A survey of cyber-physical systems”. In: Wireless Communications and Signal Processing (WCSP), 2011 International Conference on. IEEE. 2011, pp. 1–6.

[155]

Sinclair, James and Cardew-Hall, Michael. “The folksonomy tag cloud: when is it useful?” In: Journal of Information Science 34.1 (2008), pp. 15–29.

[156]

Song, Zhexuan, Cardenas, Alvaro A, and Masuoka, Ryusuke. “Semantic middleware for the Internet of Things.” In: IoT. 2010.

[157]

Stelzner, Michael A. “Social media marketing industry report”. In: Social Media Examiner 41 (2011).

[158]

Stevenson, Graeme et al. “ConStruct: A Decentralised Context Infrastructure for Ubiquitous Computing Environments”. In: Computing (2005).

[159]

Stirbu, Vlad. “Towards a restful plug and play experience in the web of things”. In: Semantic computing, 2008 IEEE international conference on. IEEE. 2008, pp. 512– 517.

[160]

Storey, Margaret-Anne, Grammel, Lars, and Treude, Christoph. “Smart media: bridging interactions and services for the smart internet”. In: The smart internet. Springer, 2010, pp. 152–169.

[161]

Sugihara, Ryo and Gupta, Rajesh K. “Programming models for sensor networks: A survey”. In: ACM Transactions on Sensor Networks (TOSN) 4.2 (2008), p. 8.

[162]

Sun, Bo et al. “Intrusion detection techniques in mobile ad hoc and Wireless Sensor Networks”. In: Wireless Communications, IEEE 14.5 (2007), pp. 56–63.

[163]

Tan, Ying, Goddard, Steve, and Perez, Lance C. “A prototype architecture for cyber-physical systems”. In: ACM Sigbed Review 5.1 (2008), p. 26.

[164]

Teixeira, Thiago et al. “Service oriented middleware for the internet of things: a perspective”. In: Towards a Service-Based Internet. Springer, 2011, pp. 220–229.

[165]

Thiesse, F and Michahelles, F. “Building the internet of things using RFID”. In: IEEE Internet Computing 13.3 (2009), pp. 48–55.

[166]

Toma, Ioan, Simperl, Elena, and Hench, Graham. “A joint roadmap for semantic technologies and the internet of things”. In: Proceedings of the Third STI Roadmapping Workshop, Crete, Greece. Vol. 1. 2009.

194

[167]

Tullis, Thomas S and Stetson, Jacqueline N. “A comparison of questionnaires for assessing website usability”. In: Usability Professional Association Conference. 2004, pp. 1–12.

[168]

Tuten, Tracy L. Advertising 2.0: social media marketing in a web 2.0 world. Greenwood Publishing Group, 2008.

[169]

Tuten, Tracy L and Solomon, Michael R. Social media marketing. SAGE, 2014.

[170]

Tynan, Richard et al. “Intelligent agents for Wireless Sensor Networks”. In: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems. ACM. 2005, pp. 1179–1180.

[171]

Vermesan, Ovidiu et al. “Internet of things strategic research roadmap”. In: O. Vermesan, P. Friess, P. Guillemin, S. Gusmeroli, H. Sundmaeker, A. Bassi, et al. Internet of Things: Global Technological and Societal Trends 1 (2011), pp. 9–52.

[172]

Vis, Farida. “Twitter as a reporting tool for breaking news: Journalists tweeting the 2011 UK riots”. In: Digital Journalism 1.1 (2013), pp. 27–47.

[173]

Wang, Fei Yue. “The emergence of intelligent enterprises: From CPS to CPSS”. In: IEEE Intelligent Systems 25.4 (2010), pp. 85–88. issn: 15411672.

[174]

Wang, Miao-Miao et al. “Middleware for Wireless Sensor Networks: A survey”. In: Journal of computer science and technology 23.3 (2008), pp. 305–326.

[175]

Weiser, Mark. “Some computer science issues in Ubiquitous Computing”. In: Communications of the ACM 36.7 (1993), pp. 75–84.

[176]

Weiser, Mark. “The computer for the 21st century”. In: Scientific american 265.3 (1991), pp. 94–104.

[177]

Wong, Jeffrey and Hong, Jason. “What do we mashup when we make mashups?” In: Proceedings of the 4th international workshop on End-user software engineering. ACM. 2008, pp. 35–39.

[178]

Wu, Fang-Jing, Kao, Yu-Fen, and Tseng, Yu-Chee. “From Wireless Sensor Networks towards cyber physical systems”. In: Pervasive and Mobile Computing 7.4 (2011), pp. 397–413.

[179]

Yan, Lu et al. The Internet of things: from RFID to the next-generation pervasive networked systems. CRC Press, 2008.

[180]

Ye, Fan et al. “A two-tier data dissemination model for large-scale Wireless Sensor Networks”. In: Proceedings of the 8th annual international conference on Mobile computing and networking. ACM. 2002, pp. 148–159.

[181]

Yu, Jin et al. “Understanding mashup development”. In: Internet Computing, IEEE 12.5 (2008), pp. 44–52. 195

[182]

Zappen, James P, Harrison, Teresa M, and Watson, David. “A new paradigm for designing e-government: web 2.0 and experience design”. In: Proceedings of the 2008 international conference on Digital government research. Digital Government Society of North America. 2008, pp. 17–26.

[183]

Zhang, Weishan and Hansen, Klaus Marius. “Semantic web based self-management for a pervasive service middleware”. In: Self-Adaptive and Self-Organizing Systems, 2008. SASO’08. Second IEEE International Conference on. IEEE. 2008, pp. 245– 254.

[184]

Zyl, TL van, Simonis, Ingo, and McFerren, Graeme. “The sensor web: systems of sensor systems”. In: International Journal of Digital Earth 2.1 (2009), pp. 16–30.

196