ELS issues in robotics and steps to consider them Part 2 ... - euRobotics

6 downloads 50 Views 603KB Size Report
Jul 16, 2016 - cross cutting issues like security/hacking, safety, data protection and liability. The document ...... The scale and scalability of cloud computing make ... You can imagine there are all sorts of things that fit into that .... employers should be guided by principles designed to minimize any risks that such methods.
Robotics Coordination Action for Europe Grant Agreement Number: 611247 17.01.2013 – 16.07.2016 Instrument: Coordination and Support Action

ELS issues in robotics and steps to consider them Part 2: Robotics and Regulations Barbara Bottalico 1, Amedeo Santosuosso 2, Oliver Goodenough 3, Roeland de Bruin 4, Chris Holder 5, Charlotte Gôme 6, Ysens de France 7, Camille Niel-Aubin 8, Nadine Bender 9, Christophe Leroux 10 Deliverable D3.4.1 part 2 Lead contractor for this deliverable:

CEA

Due date of deliverable:

June 30, 2016

Actual submission date:

June 30, 2016

Dissemination level:

Public

Revision:

1.0

1

Post-doc Researcher, Research Center ECLT at the University of Pavia (I) Professor of Law, University of Pavia; Judge and President of the 1st Chamber, Court of Appeal, Milan. 3 Director, Center For Legal Innovation, Professor Of Law,Faculty Representative, Vermont Law School Board Of Trustees 4 Researcher, Assistant Professor at the Centre for Access to and Acceptance of Autonomous Intelligence (CAAAi), Utrecht University; IT-lawyer at Mitopics. 5 Legal adviser 6 Cabinet Alain Bensoussan 7 Ph.D., Candidate, Robots in conflicts, a new legal basis?, University of Poitiers. 8 IP Legal In-House Counsel 9 Analyst of Social Impacts of Robotics, KUKA Roboter GmbH 10 Researcher, expert in robotics, CEA 2

RockEU Deliverable D3.4.1 part 2

Page 1 of 45

Executive summary Progress in research is making robots more present in our environment at work as well as at home. Regulations are consequences of this deployment of robotics in our life in order to ensure safety of people at work or to guarantee one’s privacy. This document analyses the current regulations in robotics and attempts to define some tracks for further investigations. We limited our study to some domains of application of robotics: •

Robots in the Industry, that has been quite broaden since the interconnections of systems and sensors beyond human perception;



Urban Robotics and Self-Driving Cars, which questions the existing regulation currently and in the coming years;



Robots in the Healthcare, which is an emerging topic



Drones and military robots, which stressed the importance of a balanced regulation.

We chose a “top down” approach starting from existing legislations to analysing their impact on individual case studies. Despite our focus on four domains of application mentioned above, we identified a set of cross cutting issues like security/hacking, safety, data protection and liability. The document also addresses two specific issues: common laws (compared to civil law) and electronic and robot personhood. We conclude in giving a set of recommendations for future investigations •

we recommend not to overregulate a nascent industry



we recommend to bring flexibility in the actual specific regulation



we recommend to have a prospective vision on regulation on autonomous agents



we recommend to better take into account the regulations on military robots

This report has been produced for the Partnership for Robotics SPARC in Europe via RockEU: Robotics Coordination Action for Europe, a Coordination and Support Action (CSA) project that has received funding by under the EU FP7 research and innovation programme (grant agreement 611247).

RockEU Deliverable D3.4.1 part 2

Page 2 of 45

Content 1

Introduction ....................................................................................................................................... 5

2

Cross cutting issues ......................................................................................................................... 8

3

2.1

Common laws .......................................................................................................................... 8

2.2

Electronic and robot personhood............................................................................................. 9

Robots in the industry..................................................................................................................... 11 3.1 3.1.1

Internet of Things ............................................................................................................... 11

3.1.2

Industrial Robotics and Cloud Robotics............................................................................. 11

3.1.3

Big Data - Learning Machine ............................................................................................. 12

3.2

4

5

6

The technical point ................................................................................................................ 11

Legal aspects......................................................................................................................... 14

3.2.1

Existing regulation ............................................................................................................. 14

3.2.2

Legal analysis and research goals .................................................................................... 19

Urban robotics and self-driving cars ............................................................................................... 20 4.1

The technical point: vehicle automation degrees .................................................................. 20

4.2

Legal issues ........................................................................................................................... 21

4.2.1

Levels of existing regulation .............................................................................................. 21

4.2.2

Legal analysis and research goals .................................................................................... 22

Robots in the healthcare ................................................................................................................ 32 5.1

The Technical Point ............................................................................................................... 32

5.2

Existing regulations ............................................................................................................... 32

5.3

Legal analysis and research goals ........................................................................................ 33

Robotics, a dual technology: The case of drones. ......................................................................... 34 6.1

The Technical Point - History of this technology. .................................................................. 34

6.2

Legal issues: .......................................................................................................................... 35

6.2.1

Existing regulations and Classification. ............................................................................. 35

6.2.2

France’s specific case. ...................................................................................................... 35

6.2.3

Challenges with safety and security. ................................................................................. 36

6.2.4

Law initiatives. ................................................................................................................... 37

6.2.5

Insights from the USA ........................................................................................................ 38

6.2.6

UK cases prosecuted by the Civil Aviation Authority and the Crown ................................ 39

6.2.7

Legal analysis and research goals .................................................................................... 40

7

Conclusions .................................................................................................................................... 42

8

Appendix: statement from Björn Matthias from ABB ...................................................................... 44 8.1

Introduction ............................................................................................................................ 44

8.2

Basic legislation today ........................................................................................................... 44

8.3

Possible new legal questions arising ..................................................................................... 44

8.4

Societal progress and ethical questions ................................................................................ 45

8.5

Summary and recommendations........................................................................................... 45

RockEU Deliverable D3.4.1 part 2

Page 3 of 45

RockEU Deliverable D3.4.1 part 2

Page 4 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

1 Introduction This document analyses current regulation in robotics in order to identify issues to the development of robotics activities in Europe in the industry and in research. The report proposes tracks to overcome these obstacles. This document was set in the context of RockEU coordination action. The analysis is not exhaustive. Our analysis focuses on a limited number of domains of application of robotics. These domains were chosen according to the impact they represent in terms of market impact or number of cross cutting issues they raise. Foreground considerations and basic concepts Analysing regulation issues in robotics may lead to a limited study lacking of perspective. The analysis set out to widen concepts of ICT and “autonomous systems” in order to be more general and less specific to robotics and to better identify the robotics specificities as well as to propose solutions application to various technologies. The advantages of this approach being to encompass a large spectrum of technological agents and the possibility to identify commonalities as well as discrepancies regarding regulations in ICT. This does not imply regulations should be the same for all the items falling into the definition. This approach enables to broadly encompass many kinds of artificial, decision-capable agents operating without the direction of a human being. Examples include physical robots and drones as well as purely digital software agents. We chose this term “agent” in order to include both physical and non-physical robots. It should be noted, however, that any definitions of such things as ‘autonomous system’, ‘robots’, or ‘AI’ will need to be thought through very carefully. One man’s ‘robot’ may well be another man’s simple machine and therefore, when we begin to discuss giving robots legal personhood, for example, we do need to be precise and give some thought to what it is, we are actually talking about. A set of automatic sliding doors may fit within certain people’s views as to what an autonomous system is, but it might be entirely inaccurate to start talking about restrictive regulations covering all autonomous systems, when there is a clear difference between the operation and impact of sliding doors on humanity to those potentially created by autonomous attack systems which may be applied by security and armed forces around the world. Law principles addressed in the document Some issues are raised all other the document and seemed to be common to all the hottopics of this document: •

The difference of approach between Common law and civil law ;



The reference to security or hacking which is constantly stressed out when the question of autonomous agents is studied;



The identification or the specificity of the autonomous agent which would lead to the recognition of a robot personhood.

Previous effort on the topic Around 2010, transnational dialogues between the legal and robotics community started more intensively in Europe. In particular:

RockEU Deliverable D3.4.1 part 2

Page 5 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

2010-2012: Within the euRobotics Coordination Action, the document “Suggestion for a green paper on legal issues in robotics” 1112 was released, with the aim at providing a description of legal issues in robotics, together with a set of recommendations and a roadmap. The critical areas were identified by the Document according to: i)

Robots abilities

ii)

Robots market law and consumer law

iii)

Intellectual property rights facing to the developments of robotics in Europe

iv)

Labour Law and Robotics,

v)

Data Privacy Law and Robots,

vi)

Criminal law, Europe and Robots,

vii)

Conflicts and litigation involving robots,

viii)

Non-human agents and electronic personhood.

The document was prepared with a top-down approach, instead of a bottom up one. This choice was taken after considering the risks of a bottom up approach: the risk to forget some legal issues; the risk of fragmentation of the problem; the risk to miss commonalities between robotics and other technological disciplines; the risk to consume too much time before having the chance to draw the first conclusions of the study. The European project RoboLaw 13, funded within the 7th Framework Programme, focused on legal issues of Robotics, with the aim of understanding the legal and ethical implications of emerging robotic technologies. Time has now passed, and we aim to provide an update about the fast developments in autonomous systems in the last years. Further, the approach we suggest is partially different because we don’t observe in a static perspective the way how law can deal with robotics. We rather assume a “generative” approach, aiming to find the best way to facilitate the introduction of robots, and autonomous systems on the market in general. Nevertheless, we also want to provide tools in order to prevent the risks involving autonomous systems. This Document aims at taking a step further in the analysis of legal challenges posed by robotics and adopts a methodology which is more specific and more oriented to practical applications. In short: The Document focuses on a number of “hot-topics”: •

Robots in the Industry, which can be seen as a classical topic but that has been quite broaden since the interconnections of systems and sensors beyond human perception;



Urban Robotics and Self Driving Cars, which is the topic that might question the existing regulation in the coming years;



Robots in the Healthcare, which is an emerging topic but too sensitive not to be approach in this study;



Drones and military robots, which stressed the importance of a balanced regulation.

11

C Leroux, R Labruto, euRobotics CA, D3.2.1 Ethical Legal and Societal issues in robotics, Grant Agreement Number: 248552, public report, Dec 2012 12 C Leroux, R Labruto, euRobotics CA, A green paper on legal issues in robotics, Grant Agreement Number: 248552, public report, Dec 2012 13 Palmerini, et al.. D6.2 – Guidelines on regulating robotics. Robolaw project, 2014. RockEU Deliverable D3.4.1 part 2

Page 6 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

These areas are now the most promising and impacting fields for robotics applications and it is particularly worthy analysing them more in depth. The legal questions addressed The legal analysis is driven by questions like the following ones: •

What are the legal obstacles to put advanced robotics on the market?



Are those obstacles due to national or European regulations?



What are the international legal difficulties?



Do legal obstacles contribute to increased social and cultural prejudices against machines?



What specific changes are needed in regulation at national and European level?

The chosen approach is once again “top down”: it starts from existing legislations to analyse their impact on individual case studies. The purpose of this document is to •

Map the latest challenges of robotics and to put them in light of the regulations existing inside and outside Europe, and to see which regulations represent a barrier to European efforts.



Give recommendations suggesting specific changes in regulation at national and European level to possibly overcome legal barriers.

In the next chapters we present the analysis on regulation issues by field of application. We start the presentation in the next chapter discussing some cross cutting issues to all domains.

RockEU Deliverable D3.4.1 part 2

Page 7 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

2 Cross cutting issues Regulations cover some cross cutting issues to domains of applications of robotics for instance •

Security / hacking



Security and safety (how the concept of safety changes: for industry and in general)



Innovation / market /regulations / public funding of research



Data Protection



Liability

We address here two specific issues: •

common laws



electronic and robot personhood

2.1 Common laws The potential impact of robotics technology and the way in which society will use and interact with it will create issues for existing laws and regulations that have been drafted with a view to the ‘old world’. This new technology will cut across all aspects of life and so all existing laws and regulations may need to adapt or change to take this into account. For example, if autonomous systems are now capable of ‘talking’ to each other as part of a globally connected ‘Internet of Things’ and can create new software programs or ways of working, then who should be the owner of such intellectual property? Should it be the software programmer who wrote the algorithm that enabled the autonomous systems to communicate or learn? Or must it be another individual or corporation that sits somewhere in the chain? If Intellectual Property laws have to change as a result of machines now being capable of creating ‘works’, then are we looking at specific regulations or parts of a civil code around the world or are we looking at codes of conducts or previous precedent in common law countries that need to be adapted to take new technologies into account? There is an inherent flexibility in common law countries that will allow judges and lawyers to look at analogous events from the past and draw new conclusions based upon new circumstances. This flexibility may not be as easy for civil code countries where ways of working are made subject to strictly codified practice – and so the code must change, either specifically (the intellectual property code needs to be re-written to deal with inventions created by autonomous systems) or generally (autonomous systems need to be recognized as a third class of legal status behind individuals and corporations and so they have the ability now to own property). The difficulty in redrafting existing codes/regulations or creating a new ‘Robot Law’ governing autonomous systems and their interaction with the world at large are many and varied – whereas relying on principles of approach or precedent based upon practice may provide the flexibility to deal with change and not necessarily over-regulate a nascent industry. By way of example, let us look at product liability and negligence. There is a raft of product liability laws in Europe but arguably, there will need to be new rules relating to the use of autonomous systems that regulate the dangers of the technology in certain areas. Think healthcare and robotic surgeons performing operations autonomously, without any human

RockEU Deliverable D3.4.1 part 2

Page 8 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

involvement and one can see a clear case where existing regulations around medical devices and practices will need to be changed. As to negligence, in common law jurisdictions one has the concept of one party owing a duty of care to another and this duty of care changes when the risk increases of harm being caused by one party to the other. One would argue that there would be a lower duty of care for manufacturers of autonomous vacuum cleaners than there would be for manufacturers of heavily armed security robots that may be left to roam around warehouses or land. Potentially the same technology is used by both but they have different applications and this difference could be easier to recognize in common law jurisdictions rather than civil code jurisdictions.

2.2 Electronic and robot personhood 2.2.1 Individuals have a legal personality as well as a right to property, a right to identity, a right to personality. Legal entities, such as companies, also have a legal personality. Some adjustments had to be made: for example, the closure of a company is the equivalence of life imprisonment for a human. So, we can deduce that: •

humans have a general legal personality ;



Companies have a special legal personality.

One issue is to determine if robots can have “Personhood”. In this world 4.0, Alain Bensoussan has created and talks about a singular legal personality for the robots. As a justification, the author introduces the main components of what he named the “Robot personhood”, which can be divided into 3 points: liability, capital and privacy issues. Following the concept of personality, the robot person would have a singular personality depending on the robots specificities, which specifically features: •

legal attribute or civil status: such as a last and a first name, an identification number, an address and an owner



a capital depending on the risks



an insurance

As the degree of robot’s autonomy increases, the issue of civil liability will rise. However, the regime of civil liability specific to robots does not exist in the actual state of law. In the French civil code for instance, two kinds of liability may apply to machines: •

liability for damage caused by a manufacturing defect, that is a lack of safety of a product ;



liability for damage caused by an interaction of the machine with its environment

Both kinds of liability do not suit the specific case of autonomous machines. There is actually a legal vacuum regarding liability to the extent that it is impossible to determine who is liable. According to Alain Bensoussan, the system of values established by Asimov does not allow to identify who will be responsible for an accident caused by a robot that will anticipates the situation and takes unfortunate initiative. In order to take in account the autonomous and intelligent characteristics, experts like Alain Bensoussan, propose to apply the principle of cascading liability applied to robots which identifies: •

First, the designer of the platform of artificial intelligence



Second the user



Third the owner



Fourth the seller

RockEU Deliverable D3.4.1 part 2

Page 9 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.



and lastly the manufacturer

Another component of the robot personhood is the allocation of share capital, similarly to corporations. The allocation of a capital to the robot should be proportional to the risks potentially created by the robot and combined with the obligation to take out an insurance policy. The last component of the robot personhood is the privacy issues. Protecting the privacy of an individual means protecting the memory of the robot. Robots will become “intimate hubs” of the life of their user. The EU General Data Protection Regulation should therefore be fortified. Regarding the retention period of personal data, the possibility to reset the robot’s memory should be implemented. In France for example, law on computer fraud should be applied with some adjustments to robots. To conclude, some experts think that a robot’s charter should be written and then include: definitions of robot, robot personhood, dignity, manufacturing, liability and more. 2.2.2 This kind of charter has been inspired by the Isaac Asimov three laws of robotics. Asimov itself began to slightly modify the laws because of their imperfections and their possibility of being bypassed. A fourth zeroth law had to be added in order to protect the concept of humanity. The Korean government tried to establish a charter but this kind of documents has to be considered more like guidelines for ethical innovation rather than compulsory regulation. Moreover the heterogeneity of autonomous agents and their constant evolution forbid to create general rules for the next years. Nevertheless regulation has to address the autonomisation of the robots and their abilities in order to protect fundamental freedoms and to insure the possibility of indemnification in case of involvement of an autonomous agent.

RockEU Deliverable D3.4.1 part 2

Page 10 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

3 Robots in the industry The first part of this chapter provides some technical background about robotics in the industry and the new frontier of Industrial Internet: Internet of Things, Big data analysis; reorganization of work, people at work and their rights. We then analyse the resulting legal issues.

3.1 The technical point 3.1.1 Internet of Things The significance of the IoT pertains to the real time and significant interconnections between objects and the internet. No longer does the object relate just to a person, but it is now connected to the surrounding objects and to remote databases. Many scholars have attempted to define this concept. Its initial use has been attributed to Kevin Ashton, an expert on digital innovation. 14 While many of us may still be thinking about being connected in terms of computers, tablets and smartphones, the IoT describes a world where just about anything – including robots can communicate and be connected in an intelligent fashion. In other words, with the Internet of Things, the physical world becomes one big information system. How can this affect industries and the workplace? For examples, industrial machines can have sensors that notify workers when a problem occurs; surveillance systems can use locks and camera systems that inform the building manager in real time when someone is attempting to enter; plant managers can receive information and data about the production in a continuing flow, and so on. Another key ability is predictive planning, where stops in production due to missing material can be avoided by having the robot connected to the storage facilities.

3.1.2 Industrial Robotics and Cloud Robotics An industrial robot commonly refers to a robot arm used in a factory environment for manufacturing applications. Traditional industrial robots can be classified according to different criteria such as the type of movement (degrees of freedom), application (manufacturing process), architecture (serial or parallel) and brand (Pandilov, Dukovsky 2014). Most recent robots and automation systems are not limited by on-board computation, memory, or software. Rather than viewing robots and automated machines as isolated systems with limited computation and memory 15, "Cloud Robotics” suggests a new paradigm where they exchange data and perform computation via networks. Cloud computing was indeed born as “a way of delivering computing resources as a utility service via a network, typically the Internet, which can be scaled up and down according to user requirements” (Millard 2013). As such, the cloud may prove to be a disruptive innovation, “as was the

14

“I'm fairly sure the phrase "Internet of Things" started life as the title of a presentation I made at Procter & Gamble (P&G) in 1999”, (Ashton 2009). 15 In the late 90s he stated: If we had computers that knew everything there was to know about things using data they gathered without any help from us - we would be able to track and count everything, and greatly reduce waste, loss and cost. We would know when things needed replacing, repairing or recalling, and whether they were fresh or past their best. (Ashton 2009). Prof. Ken Goldberg’s suggestion is to consider a new paradigm where robots and automation systems exchange data and perform computation via networks: “Cloud Robotics and Automation”, available at URL: http://goldberg.berkeley.edu/cloud-robotics/ [last accessed: 07/06/2015]. RockEU Deliverable D3.4.1 part 2

Page 11 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

emergence of cheap electricity on demand a century or so ago” (Carr 2009). Such computing resources may range from raw processing power and storage, such as servers or storage equipment, to full software applications. Users can ‘rent’ IT resources from third parties when needed, instead of purchasing their own. The scale and scalability of cloud computing make it suitable for environments where the demand for IT resources may fluctuate widely and rapidly. The cloud may also be relevant for supporting the deployment of mobile devices and applications on a large scale (Millard 2013). Finding a shared regulatory approach across the field of robotics is not easy: the variety of technologies, both basic and new or advanced, is relevant, and it can also be difficult to separate robotics from other technologies with which it is integrated, considering that its technical components also have different applications. Cloud robotics does, however, provide a different degree of functionality that may be harnessed by corporations and/or individuals. If one accepts the proposition that autonomous systems are different from existing IT systems that require human involvement, then this new functionality will provide corporations and consumers with new ways of working. However, how are these new ways of working presented? Are users of such autonomous systems able to rely on the solutions/answers that such systems produce or do the corporations selling these new cloud robotics solutions merely view these solutions as an example of enhanced functionality and therefore whatever ‘solution/answer’ is provided by the autonomous system is up to the user to interpret and use. No liability is assumed by the corporation for the solution/answer provided. If this is the case, then the implications for commercial contracting in the cloud remain exactly the same – put very simply, users will need to contract for levels of expected functionality only and it is when such functionality is not available for productive use that users may claim for breach of contract.

3.1.3 Big Data - Learning Machine The collection and combination of all data generated by users and machinery with other sources and analytics can be a tremendous resource for society, policy makers, public authorities and the private market 16. This huge potential lies in the expected predictive capacity that they can offer if analysed in a certain domain. Further, data is becoming more available and more adapted to the needs and capabilities of computers. Most of them are “unstructured” and Internet-based, and would not be ready for traditional computational databases, but the computer tools for gathering knowledge and insights from data in this Internet era are gaining ground rapidly (Lohr 2012). As reported by a study conducted by the consulting company “Boston Consulting Group” in 2015, analytics based on large data sets have emerged only recently in the manufacturing scenario, and the use of these innovative tools would have the benefit of opening the possibility to optimize production quality, save energy, and improve equipment service 17. In

16

See the Document published by the Organization for Economic co-operation and development OECD E-Government Project GOV/PGC/EGOV(2012)7/REV1, 1 March 2013, available at: URL: [last accessed: 07/06/2015] 17 Industry 4.0. The future of Productivity and Growth in manufacturing Industries, The Boston Consulting Group, April 2015, available at: URL: < http://www.voesi.or.at/wpcontent/uploads/2013/02/Industry_40_Future_of_Productivity_April_2015_tcm80-185183.pdf> [last accessed: 06/06/2015]. RockEU Deliverable D3.4.1 part 2

Page 12 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

an Industry 4.0 context, the availability of data from many different sources (production equipment and systems, enterprise and customer-management systems, and so on) may become the basis to support real-time decision making within the production 18. Big data combined with learning machines can create a self-enhancing development. Learning machines are indeed particularly promising as a tool to improve the capacity of the technology by a self-development of new skills, grounded on previous experience. DeepMind, a learning system recently acquired by Google and developed by a UK enterprise, is quite a clear example: (http://deepmind.com/): this type of advance in computing power means that learning (or artificial intelligent) systems can handle much larger data sets, considering that watching an Atari game corresponds to processing about 2 million pixels of data a second. Google may be interested in this software in order to better analyse the huge amount of data it collects, since the pixels of the game can be considered analogous to the data Google has on each user, and the score would be their advertising revenue 19. Demis Hassabis, the co-founder of DeepMind declared in a recent interview to New Scientist "We can't say anything publicly about this but the system is useful for any sequential decision making task. You can imagine there are all sorts of things that fit into that description" (Aron 2015). The question of ‘data as infrastructure’ for autonomous systems will also need to be addressed in the future. Data collection is one thing – and there are existing laws and regulations that are in place at the moment which, arguably, are already out of date when it comes to autonomous system and robot technology – but data analysis and the way data is manipulated by autonomous systems to make better informed decisions (either made by other machines or humans) should also be discussed and addressed. Should there be an ethical analysis of what types of data can be analyzed notwithstanding the anonymized nature of that data? Is it right to collect genomic data for analysis by commercial enterprises looking to develop drugs or insurance strategies or should such data only be used for research purposes? The potential for the creation of ethical standards for mass data analysis is something that is currently being explored in the UK. The problems arising: i.

Data storage – who owns data?

ii.

Data security? What happens if data is hacked?

iii.

Machine learning opens a new field of liability issues: who is liable for what themachine is learning? What if machine is hacked to learn new/possibly illegal things??

18

Id. For a broader comment about possible uses of DeepMind by Google, see: Jacob Aron, Google DeepMind AI outplays humans at videogames, New Scientists Tech, 25/05/2015. 19

RockEU Deliverable D3.4.1 part 2

Page 13 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

3.2 Legal aspects 3.2.1 Existing regulation 3.2.1.1 On safety of industrial robots Directive on Machinery 2006/42/EC 20: The directive is applied in the whole EU starting from 2009. National regulations that adopt the directive (in the previous or the new version): •

UK: Supply of Machinery (safety) regulation 2005 as amended by the Supply of Machinery (Safety) Amendment Regulations 2011;



Finland: Valtioneuvoston paatos koneiden turvaillisuudesta (vnp 1314/1994);



France : Décret n°2008-1156 du 7 novembre 2008 relatif aux équipements de travail et aux équipements de protection individuelle;



Holland: Warenwet, Besluit Machines;



Belgium: KB 05/05/1995, AR 05/05/1995;



Germany: ISO Normen 10218/13849/12100/TS 15066.

CE marking Obtaining the CE marking is essential for all companies who wish to market their products in Europe: it is a compliance certification, mandatory for all those products covered by European Directives and for the EU market. The CE mark is a certificate of compliance with the strict European standards on safety, the requirements of which are binding on all products marketed within the EU. The Machinery Directive 2006/42/ EC regulates the safety of machines and equipment in Europe with a very large scope as it applies to many products used in machinery installation (article 1) : machinery itself, interchangeable equipment, safety components, lifting accessories, chains, ropes and webbing, removable mechanical transmission devices, partly completed machinery. The Machinery Directive requires a risk analysis and simulation of failures for all products covered by it, as required by the European standard EN ISO 12100. All autonomous agents have to obtain this CE marking in order to ensure safety of persons, like workers or consumers but also domestic animals and goods. Each part of the robot has to respect these requirements. For example, the level of force defined for the closure of a lift door can be used to any equipment which can have a contact with a human body.

Directive 2009/104/EC – use of work equipment Of 16 September 2009 concerning the minimum safety and health requirements for the use of work equipment by workers at work (second individual Directive within the meaning of Article 16(1) of Directive 89/391/EEC) 21 With specific concern to robots in the industry, we also propose to evaluate the possibility to follow the path of the ISO 12100 that was prepared by Technical Committee ISO/TC 199, Safety of machinery in order to see what specifically should be regulated for robots.

20 21

http://eurlex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:157:0024:0086:en:PDF http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32009L0104

RockEU Deliverable D3.4.1 part 2

Page 14 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

List of national laws implementing this Directive: http://eurlex.europa.eu/search.html?type=advanced&qid=1446466244741&DN=72009L0104*

3.2.1.2 Protection of employees’ rights in the workplace. Privacy and surveillance EU level In April 2015 the Committee of Ministers to Member States made a recommendation CM/REC(2015)5 on the processing of personal data in the context of employment This recommendation starts from the assumption that the previous REC(89)2 should be revised in light of “the increasing use of new technologies and means of electronic communication in the relations between employers and employees, and the corresponding advantages thereof” and based on the believe that the use of data processing methods by employers should be guided by principles designed to minimize any risks that such methods might pose to employees’ rights and fundamental freedoms. Part I of the Recommendation focuses on the general principles to be given particular priority for electronic correspondence, i.e. Human Dignity, the Right to Privacy, the protection of personal data and the relevance of the aims pursued by the collection of data. Part II deals specifically with the case of email correspondence, and Article 14 provides that: “14.

Use of Internet and electronic communications in the workplace

14.1. Employers should avoid unjustifiable and unreasonable interferences with employees’ right to private life. This principle extends to all technical devices and ICTs used by an employee. The persons concerned should be properly and periodically informed in application of a clear privacy policy, in accordance with principle 10 of the present recommendation. The information provided should be kept up to date and should include the purpose of the processing, the preservation or back-up period of traffic data and the archiving of professional electronic communications. 14.2. In particular, in the event of processing of personal data relating to Internet or Intranet pages accessed by the employee, preference should be given to the adoption of preventive measures, such as the use of filters which prevent particular operations, and to the grading of possible monitoring on personal data, giving preference for non‑individual random checks on data which are anonymous or in some way aggregated. 14.3. Access by employers to the professional electronic communications of their employees who have been informed in advance of the existence of that possibility can only occur, where necessary, for security or other legitimate reasons. In case of absent employees, employers should take the necessary measures and foresee the appropriate procedures aimed at enabling access to professional electronic communications only when such access is of professional necessity. Access should be undertaken in the least intrusive way possible and only after having informed the employees concerned. 14.4. The content, sending and receiving of private electronic communications at work should not be monitored under any circumstances.”22.

22

Extracts of Article 14, Recommendation CM/Rec(2015)5 of the ‘Committee of Ministers to member States on the processing of personal data in the context of employment’, available at: https://wcd.coe.int/ViewDoc.jsp?id=2306625 (last accessed: 4 June 2015). RockEU Deliverable D3.4.1 part 2

Page 15 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

National level Here we want to present you a small selection of examples from EU member states legislation. Italian Data Protection Authority for E-mail and Internet (Act n. 13, 1 March 2007), 23 which promotes: i) the adoption of a specific company policy; ii) the use of filters to prevent access to inappropriate sites; iii) the use of anonymous data and storing data for a limited time; iv) the provision of both shared e-mail addresses and a different e-mail account for personal use of the employees; v) the delegation of a trustee of the employee for access his/her email; vii) automatic responses in case of absence, indicating a colleague’s name as an alternate contact for immediate needs; and viii) consultation with the trade union in cases where the intervention of the employer may constitute a “remote control”, forbidden according to article 4 of the Statute of Workers 24. In general, the employer’s control must maintain a "human dimension", which should not be inflamed by the use of technologies that can exacerbate the employer’s supervision of employees without any confidentiality and autonomy in the management of the employment relationship. In France the CNIL ensures that citizens can effectively access their data contained in the processing, notably concerning human resources (supervision mechanism like video surveillance or the geolocalisation of vehicles) applying the Act n°78-17 of 6 January 1978 on information technology, data files and civil liberties 25. The CNIL made a deliberation 26 and created a specific norm n° 51 for geolocalisation of vehicles 27. In The Netherlands, the Autoriteit Persoonsgegevens (Data Protection Authority, DPA) has a standing policy on monitoring of employees by employers, 28 for instance through camera surveillance; monitoring of email and phone use. 29 An overview of the general principles applicable to monitoring/control: •

23

The employer must have a ‘legitimate interest’ to deploy monitoring/control, which outweighs the privacy interests of employees;

The Guidelines are available, in Italian and English, at the link:

http://www.garanteprivacy.it/web/guest/home/docweb/-/docweb-display/docweb/1408680 (last accessed: 3

June 2015). 24 The Italian Data Protection Authority confirmed the Guidelines in the last Vademecum “Privacy and Work” approved in April 2015 and available - in Italian - at the link: http://194.242.234.211/documents/10160/3844886/Privacy+e+lavoro+-+vademecum+2015.pdf (last accessed: 2 June 2015). 25 The CNIL give access to a document to “Travail & Vie privée La géolocalisation des véhicules” available at the link: https://www.cnil.fr/sites/default/files/atoms/files/fichetravail_geolocalisation_1.pdf. 26 Délibération n°2015-165 du 4 juin 2015 portant adoption d'une norme simplifiée concernant les traitements automatisés de données à caractère personnel mis en œuvre par les organismes publics ou privés destinés à géolocaliser les véhicules utilisés par leurs employés (norme simplifiée n° 51) available at the link: https://www.legifrance.gouv.fr/affichCnil.do?oldAction=rechExpCnil&id=CNILTEXT000030750827. 27 Norme simplifiée NS-051 « Géolocalisation des véhicules des employés » available at the link: https://www.cnil.fr/fr/declaration/ns-051-geolocalisation-des-vehicules-des-employes. 28 See https://autoriteitpersoonsgegevens.nl/nl/onderwerpen/werk-uitkering/controle-van-personeel (only available in Dutch). 29 See: https://autoriteitpersoonsgegevens.nl/sites/default/files/atoms/files/cbp_cameratoezicht.pdf (only available in Dutch). RockEU Deliverable D3.4.1 part 2

Page 16 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.



Monitoring/control may only take place when there is no ‘lighter’ modality available for protecting the employers’ interests;



Notification of monitoring/control must be made to the DPA;



The employer must inform the employees on a) the applicable rules; and b) the monitoring/control thereof;



The employer must seek permission from the works council for the monitoring/control.

In a recent decision, the European Court of Human Rights has held that monitoring an employee’s private chat history, while at work, was not a breach of his Article 8 right to respect for private and family life. 30 The intrusion could be justified by the employer’s need to ensure that the employee was carrying out his professional duties during working time. [Bărbulescu v Romania (application 61496/08).]

Robots and discrimination law The Equality Act 2010 prohibits discrimination and harassment in respect of a number of protected characteristics, namely: age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex, and sexual orientation. 31 There are different types of discrimination, namely direct and indirect discrimination, harassment, victimisation, and instructing, causing, inducing or helping discrimination. 32 Employers must not engage in discrimination in respect of job applicants or their staff, and in some situations are liable for the discriminatory acts of their employees. Unlike unfair dismissal, compensation for discrimination claims is uncapped. In a scenario of large-scale redundancies resulting from humans being replaced by robots in the workplace, employers would need to be aware of discrimination risks. For instance, employers cannot discriminate on the grounds of age. If older employees are disproportionately affected, this might give rise to claims of indirect discrimination. Staff cannot be selected for redundancy on the basis of a protected characteristic such as age either, so assuming that older staff will be less technically skilled and therefore less able to work with or alongside robots could be discriminatory. Assuming that younger employees will be more tech-savvy and acting on that basis in any context is likely to result in claims for discrimination. Similarly, a re-training bursary or grant to be awarded on redundancy (discussed above) would have to be administered carefully to avoid claims of discrimination. If this is only offered to younger employees for instance, there might be a claim for indirect discrimination. Equally, if employers shift their hiring practices to target younger employees who might be more open to working with robots this is likely to raise similar concerns. Perhaps a more obvious interface between robotics and discrimination law arises in relation to disability discrimination. At the moment, under UK law, an individual has a disability if they have a physical or mental impairment which has a substantial and long-term adverse impact on their ability to carry out normal day-to-day activities. 33 Employers must not discriminate on the basis of something arising out of the individual’s disability (e.g. prolonged or frequent bouts of sickness absence) 34 and they have a duty to make reasonable adjustments. 35 This

31

Section 4 Equality Act 2010. Sections 13, 19, 26, 27, 111 and 112 Equality Act 2010 respectively. 33 Section 6, Equality Act 2010. 34 Section 15(1) Equality Act 2010. 32

RockEU Deliverable D3.4.1 part 2

Page 17 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

duty arises where a provision, criterion, or practice applied by the employer puts a disabled person at a substantial disadvantage; where a physical feature of the employer’s premises puts a disabled person at a substantial disadvantage; and where a disabled person would, but for the employer’s provision of an auxiliary aid, be put at a substantial disadvantage. In these situations, the employer must take such steps as are reasonable to avoid the disadvantage, or to provide the auxiliary aid. The key question is what makes an adjustment reasonable? It can include measures such as transferring the employee to a new role, altering their duties, working hours, or place of work, modifying disciplinary or grievance procedures or other policies or procedures (in particular, sickness absence policies). What is reasonable is ultimately fact-sensitive. To bring a claim alleging a breach of the duty, the employee will usually have to specify the adjustments that should have been made. It is then open to the employer to demonstrate either that each suggested amendment would not have worked, or that it would have been impractical for some other reason. There is no duty on employers to take measures that would impose a disproportionate burden on them, and the financial cost of making an adjustment is one factor which may be taken into account when determining reasonableness. The use of robotic technology to provide auxiliary aids is a fascinating area of development. Robotic prostheses have been developed that can allow an individual to regain lost functionality, and in fact entire exoskeletons have been developed that can allow paralysed individuals to walk again. 36 It is entirely possible that in time these sorts of technologies could become reasonable adjustments for employers to make for disabled staff. Probably the biggest barrier at the moment is cost - a bespoke and one-off product like this is incredibly expensive and it would therefore be a disproportionate burden on the employer to have to provide it. As these technologies become more affordable however the sorts of auxiliary aid that employers might be expected to provide are likely to become increasingly sophisticated. As the technology develops further, this potentially raises some interesting questions. What if the exoskeleton actually allows the disabled person to run further, or faster than a normal human? Maybe a robotic hand could allow someone to lift a heavier weight, or carry out certain tasks with greater accuracy. Reasonable adjustments are about removing a disadvantage to a disabled person. Is there anything that would require employers to ‘enable’ employees who do not have a disability, but who would nevertheless like to benefit from an exoskeleton that enhanced their natural abilities? We might see this situation arising in a military context, where the idea of enhancing the capabilities of a normal soldier has long been a topic of research. At present, the duty to make reasonable adjustments is only triggered where someone with a disability experiences a disadvantage or is treated less favourably than a non-disabled comparator. The situation in which an adjustment is so effective that the disabled person is actually subsequently at an advantage as compared to the comparator is not expressly dealt with. However, it seems clear that non-disabled staff would not be able to claim that they were being discriminated against or treated unfavourably by virtue of not having the benefit of the adjustment. The Equality Act 2010 requires employers to make reasonable adjustments in order to remove a disadvantage or to ameliorate a situation in which a disabled person is treated less favourably - taking positive action to further enable healthy employees goes beyond the scope of the legislation. If robotic enhancements become commonplace, then this might be an area that needs to get revisited in the future.

35 36

Section 20 Equality Act 2010. http://www.bbc.co.uk/news/technology-33369732

RockEU Deliverable D3.4.1 part 2

Page 18 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

3.2.2 Legal analysis and research goals •

Investigate other countries for regarding their national regulations that adopt the Directive on Machinery 2006/42/EC



Explore what’s new at national level concerning the adoption of the Directive 2009/104/EC – use of work equipment



Investigate

the

efficacy

of

UNI

EN

norms.

(e.g.

Manipulation

robot

http://conference.ing.unipi.it/vgr2006/archivio/Articoli/080.pdf )



Using the ISO rules, investigate what rules should be added specifically for learning robots in the industry



Investigate if new regulation is necessary at the national or European level to protect employees’ rights when working together with robots



Regulation/standardisation is important already during the early technology development process (safety aspect), but on the other hand it should not be over regulated as to not hinder new developments



Research on the protection of legal rights for “smart workers /e-workers” in regard to hacking threats



Discrimination of human workers



Human Robot Collaboration also calls for new and/or refined safety standards: 1. First step: analyse existing norms and regulations (e.g. Machinery Directive) and include newest technological insights by refining existing HRC norms (e.g. ISO 12100/10218/13849/15066) for example regarding newest biomechanical threshold values for collisions in power or force limited robots. 2. Second step: following the results of step one, identify remaining gaps in current legislation and work on new norms

RockEU Deliverable D3.4.1 part 2

Page 19 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

4 Urban robotics and self-driving cars 4.1 The technical point: vehicle automation degrees Step by step, cars will become more capable of driving themselves, eventually without needing a human driver. It is predicted that autonomous driving will be gradually developed and introduced into society. 37 The US-based Society for Automotive Engineering (SAE) has developed a “Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems”, 38 (hereinafter referred to as: “SAE Taxonomy”), which is becoming widely adopted in literature, 39 and also by European Union policy makers as a de facto categorization of self-driving cars. 40

37

Morgan Stanley has predicted (in 2014) that fully autonomous vehicles may ‘hit the road’ in the US by 2026: http://www.businessinsider.com.au/morgan-stanley-utopian-society-in-2026-2014-2. 38

. 39 See for instance Gillian Yeomans, Autonomous Vehicles – handing over control: opportunities and risks for insurance, LLOYD’S 2014, via https://www.lloyds.com/~/media/lloyds/reports/emerging%20risk%20reports/autonomous%20vehicles %20final.pdf; See also: Tom Gasser, Daniel Westhoff, “BASt-study: definition of automation and legal th issues in Germany”, German Federal Highway Research Institute, 25 July 2012; John Bradburn et al., “Connected & Autonomous Vehicles – Introducing the Future of Mobility”, Atkins, via http://www.atkinsglobal.com/~/media/Files/A/Atkins-Corporate/uk-and-europe/uk-thoughtleadership/reports/CAV_A4_digital_250915_FINAL.pdf. 40 EP Briefing “Automated Vehicles in the EU”, January 2016, via: http://www.europarl.europa.eu/RegData/etudes/BRIE/2016/573902/EPRS_BRI(2016)573902_EN.pdf RockEU Deliverable D3.4.1 part 2

Page 20 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

4.2 Legal issues 4.2.1 Levels of existing regulation 4.2.1.1 International regulations These regulations promote road safety by establishing uniform rules

1949: Geneva convention of Road Traffic (for United States) Article 4: “driver is defined as any person who drives a vehicle, or who is in actual physical control of the same.” Article 8 provides that every vehicle shall have a driver who is at all times able to control it.

1968: Vienna convention on Road Traffic (for European countries) Article 8: “every moving vehicle or combination of vehicles shall have a driver; every driver shall at all times be able to control his vehicle or to guide his animal.”

4.2.1.2 National regulations They provide road traffic codes

France : Highway Code and Act n°85-677 of 5 July 1985 (Badinter Act) Art. L.110-1 of the French Highway Code : a “power-driven vehicle” means any self-propelled terrestrial vehicle, including trolleybus, running on a road under its own power, other than a rail-borne vehicle. Badinter Act: compensation paid by the insurer to the victim since the custodian (gardien) or driver of the vehicle is involved.

Italian national traffic code (Legislative Decree n. 285/2002) Art. 46: “Vehicles are defined as all machines that circulate along the roads and are driven by man.” Art. 119: “A person who is affected by physical or psychic illness, or organic deficiencies, or psychic, physical or functional handicap, which are able to prevent him to securely conduct motor vehicles, will not be allowed to get the driving license.” Art 140: “Road users must behave so as not to represent a danger or obstacle to the traffic circulation and so as to preserve the road safety in any case.”

The Netherlands: Wegenverkeerswet (Road traffic act) The Netherlands’ Wegenverkeerswet 1994 41 implicates that motor vehicles need to be operated by a (capable) human driver. Among other things, the Wegenverkeerswet stipulates that a driver under the influence of substances of which he “knows, or reasonable needs to know that the use thereof – whether or not in combination with another substance – can have a negative impact on his driving ability”, 42 (art. 8(1)) is forbidden to drive a motor vehicle. Many material requirements of the Wegenverkeerswet have been elaborated in the Regeling Voertuigen (Vehicles Regulation) 43 and the Besluit ontheffingverlening exceptionele transporten (Exceptional Transports (Exemption) Decree). This latter decree has been

41

http://wetten.overheid.nl/BWBR0006622/2016-03-15 Unofficial translation by Roeland de Bruin. 43 http://wetten.overheid.nl/BWBR0025798/2012-04-01 42

RockEU Deliverable D3.4.1 part 2

Page 21 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

amended in 2015 in order to “create a climate in the Netherlands which will support and encourage the development of autonomous vehicles” 44. The Rijksdienst voor het Wegverkeer (RDW, Vehicle and Driver Licensing Authority) may under this decree “permit the use of vehicles which rely on or include innovative, automatic systems to be operated on the public highway. Before granting permission, RDW will conduct a full technical inspection to ensure that such vehicles can be tested safely on the public highway”.

Examples of new legislation in USA New regulations have been approved in the USA in order to regulate the testing of autonomous vehicles 45. An interesting example is: California SB-1298 Vehicles – autonomous vehicles: safety and performance requirements, 2012 “The State of California, which presently does not prohibit or specifically regulate the operation of autonomous vehicles, desires to encourage the current and future development, testing and operation of autonomous vehicles on the public roads of the state. The state seeks to avoid interrupting these activities while at the same time creating appropriate rules intended to ensure that the testing and operation of autonomous vehicles in the state are conducted in a safe manner” Interesting definitions: •

“Autonomous vehicles means technology that has the capability to drive a vehicle without the active physical control or monitoring by a human operator”



“Autonomous vehicle means any vehicle equipped with autonomous technology that has been integrated into that vehicle”



“An “operator” of an autonomous vehicle is the person who is seated in the driver’s seat, or if there is no person in the driver’s seat causes the autonomous technology to engage”

Among the provisions: “The driver shall be seated in the driver’s seat, monitoring the safe operation of the autonomous vehicle, and capable of taking over immediate manual control of the autonomous vehicle in the event of an autonomous technology failure or other emergency”. 46 [in 2013 U.S. Department of Transportation Releases Policy on Automated Vehicle Development 47]

4.2.2 Legal analysis and research goals 4.2.2.1 Consistency with international and national regulations Obligations provided by the International conventions aimed to foster road safety partly by ensuring that vehicles could be controlled. Control is thus the key concept, which the

44

See for a summary in English: https://www.internetconsultatie.nl/zelfrijdendevoertuigen/document/1342. 45 See the legislative and regulatory developments related to automated driving, automatic driving, autonomous driving, self-driving vehicles, and driverless cars: . http://cyberlaw.stanford.edu/wiki/index.php/Automated_Driving:_Legislative_and_Regulatory_Action. 46 For a comprehensive analysis of the new legislation on autonomous vehicles in USA, see: Briant Walker Smith, “Automated vehicles are probably legal in the United States”, 1 Tex A&M L. Rev. 411 (2014) 47

http://www.nhtsa.gov/About%20NHTSA/Press%20Releases/U.S.%20Department%20of%20Transpo rtation%20Releases%20Policy%20on%20Automated%20Vehicle%20Development RockEU Deliverable D3.4.1 part 2

Page 22 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

analysis can start from, in order to see if these regulations are satisfied by automated vehicles. Definitions of control (by Farlex): 1. To command, direct, or rule 2. To check, limit, curb, or regulate; restrain 3. To regulate or operate (a machine) When is the requisite of control satisfied? •

When a human driver is present in the vehicle? Only if we consider that the only possibility to control a vehicle is that a human driver does it in person; otherwise, not necessarily



When a human driver is able to intervene in operations of the vehicle? Yes.



When the vehicle operates within limits set by human intervention? Not necessarily.

The purposes of control can be defined as i)

ensuring compliance with the road traffic code,

ii) ensuring reasonableness in situations not regulated by the codes / in case of fallacies of the vehicle / in case of unlawful conduct or mistakes by other drivers. As to point (i), the compliance with the road traffic code can be codified in a set of rules that will be programmed in the self-driving machine (design). As to point (ii), there are more difficulties. Situations not regulated by the codes and fallacies of the technical system can occur. Then the real-time human intervention is the only possibility to solve the problem. According to the present regulations, currently there are no problems with partial automation vehicles or driver assistance systems: the control is still possible and the vehicle operates within the monitoring activities of the human driver. In high automation and full automation systems, however, there is no control by the human driver, but he/she could be able to intervene in case of necessity. At the state of the art, control is still required. Questions: •

Is an average human driver capable of supervising an autonomous vehicle?



Is a plane’s auto pilot a useful analogy for such systems? (possible differences: an autonomous vehicles driver is expected to immediately react to quick events, while for a pilot events are not so quick; a pilot is extensively trained, while a human driver is not) Europe - Other normative proposals

As to “DAS-Driver Assistance Systems” 48, the Inlands Transport Committee, Working party of Road Traffic Safety in 2011 and 2012 (experts from France, Germany, Russia, Spain) elaborated definitions and rules, suggesting to include them in the Vienna Convention. They suggest that they should be allowed if: they are over-ridable at any time or can be switched off; they only optimize at technical level some functions with operating tasks

48

DAS: a built-in system intended to help the driver in performing his driving task and which have an influence on the way the vehicle is driven, especially aimed at the prevention of road accident RockEU Deliverable D3.4.1 part 2

Page 23 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

depending on the driver only; they solely operate in case of emergency when the driver has lost control. The United Nations has a forum (“WP29“) which aims to avoid such problems by harmonizing vehicle regulations. Draft proposal for terms: https://www2.unece.org/wiki/download/attachments/25264474/%28ITS-AD_0103%29%20Revised%20draft%20of%20Terms%20of%20Reference.pdf?api=v2 This website: http://www.globalautoregs.com/ maintains an up-to date list of cross-referenced documents related to WP29. MIT experiments: http://web.mit.edu/reimer/www/pdfs/Reimer_NEMPA_2012.pdf )

4.2.2.2 Liability in case of damages to other vehicles or people Autonomous vehicles will eventually become safer than human driven cars. However, malfunctioning due to for example a production defect, faults of those inside a vehicle (in earlier stages of automation: drivers; or other passengers of the cars), or malicious interference by third parties (hacking) could cause serious material and immaterial damage. For an (part of an) autonomous vehicle that is defective, the producer (e.g. the manufacturer or importer) may be held liable based on for instance the harmonized rules of the Product Liability Directive (PLD). 49 A short overview on the PLD-rules when applied to autonomous vehicles is provided below under a). Rules on determining liability for damage caused to other vehicles or people by other actors than producers (i.e. drivers, passengers or hackers) have to date not been harmonized in the European Union. The individual regimes of the 28 Member States show a wide variation in rules for determining liability for damage in situations where motor vehicles are used. Three examples are elaborated below under b), giving an overview of the regulation of liability for motor vehicles in France, The Netherlands and the United Kingdom. 50

a) Product Liability Directive The PLD addresses liability for damage caused by defective, unsafe products. The PLD installed a no-fault liability regime for producers, creating rights for consumers which cannot be limited or waived, 51 to seek compensation for damages caused to people and goods by defective products. 52 The regime operates alongside and without prejudice to national regulations on either contractual or non-contractual liability, “in so far as these provisions also serve to attain the objective of effective protection of consumers”. 53 This implicates that there may well be other regimes used in the Member States besides the implemented PLD rules to hold producers liable. Objective of the PLD is to create a one-stop-shop for consumers who have suffered damages caused by industrially produced movable goods 54 that are defective. Defective are those products lacking a certain amount of safety “which the public at large is entitled to expect, taking all circumstances into account, including (a) the presentation of the product; (b) the use to which it would reasonable be expected that the product would be put”,

49

Directive 85/374/EC. Parts of these texts are also published in R.W. de Bruin, “Autonomous Intelligent Cars on the European intersection of liability and privacy– Regulatory challenges and the road ahead”, European Journal for Risk Regulation, forthcoming 2016. 51 Article 12 PLD. 52 See the –non-numbered – considerations of the Product Liability Directive. 53 Article 12 PLD as well as the considerations at this point. 54 Including electricity (Article 2 PLD). 50

RockEU Deliverable D3.4.1 part 2

Page 24 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

and “(c) the time when the product was put into circulation”. 55 As long as consumers can prove the defect, the damage and the causal relationship between those two criteria, 56 compensation can be sought not only from the actual producer, but also from among others the importer, the manufacturer of raw material and components and the end-supplier of the defective product. 57 In order to assess the ‘defectiveness’ of an autonomous vehicle, one could assume that the public at large may expect a system that is able to drive at least as well as an excellent human driver. However, proving that there was a defect in the software requires a deep technological understanding of the functioning and the malfunctioning of (parts of) autonomous vehicles. It is likely that most victims do not have easy access to this kind of knowledge. 58 Proving defects will thus be problematic. Victims furthermore have to prove the causal relationship between defect and damage, which may also be observed to form a heavy burden for claimants. The losses covered for in the PLD may consist in damage caused by death or personal injuries, and damage to items intended and/or used for private consumption by the injured person, other than the defective product itself “with a lower threshold of 500 ECU” 59 (Euro). This implicates inter alia that pure economic loss, or (immaterial) damage following from a personal data breach (for instance as a result of hacking) are not included in the scope of the PLD. 60 Whenever damages are caused by both a defective product and a fault of the victim, thus constituting a ‘contributory negligence’, 61 this may lead to reduction or disallowance of liability of the producer. Liability for producers may be exonerated, when a defect came into existence after the marketing by the producer, when this was for instance caused by hacking of a vehicle. 62 Furthermore, the PLD states 63 that a producer cannot be held liable when “the state of scientific and technical knowledge at the time when he put the product into circulation was not such as to enable the existence of the defect to be discovered”. 64 The ECJ ruled that a producer relying on this “state of the art”-defence, must prove that the objective, most advanced state of the art, not limited to the industrial sector concerned, was not “such as to enable the existence of the defect to be discovered”, 65 as long as that knowledge had been accessible at the time when the defective product was put into circulation. Despite this judgment, there still is no consensus amongst EU Member States on the application of the

55

Article 6(1) PLD. Article 4 PLD. 57 Article 3 and 5 PLD. 58 See also Yeomans 2014, at p. 18. 59 Article 9(b) PLD. 60 Controllers or processors of personal data under the Data Protection Directive (DPD), and the new General Data Protection Regulation (GDPR), can be held liable for data breaches. However, when a car manufacturer cannot be qualified as a controller (the entity determining means and purposes for the processing of personal data) or a processor (the entity carrying out processing for and under the responsibility of a controller), he cannot be held liable under the DPD or GDPR. 61 defined in the considerations as “contributory negligence”. 62 Article 7(b) PLD. 63 This is a minimum-harmonization provision, allowing states to take regulatory measures allowing the establishment of liability for producers even when can be proved that a defect could not have been discovered based on the state of the scientific and technological knowledge at the time a product was put into circulation, see Article 15(1)(b) PLD. 64 Article 7(e) PLD. 65 ECJ 29 May 1997, C-300/95, Commission/United Kingdom. 56

RockEU Deliverable D3.4.1 part 2

Page 25 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

state of the art defence, concluded the European Commission in 2011. 66 Some member states even argue in favour of removing this liability exclusion clause from the Directive, for that would lead to a better functioning of the internal market. Industry and insurance companies point out that when such should happen, this would have a direct negative impact on the speed of innovation and the development of new products, and lead to higher insurance costs. 67

b) Non-hamonized liability for motor vehicles The European Commission attempted to harmonize a regulatory framework on liability for motor vehicles 68, however, to date these issues are still individually addressed by the legal regimes of the Member States. The EC has issued six Directives on Insurance for motor vehicles, of which one currently is still in force. 69 The Motor Insurance Directive 70 regulates civil liability insurance for motor vehicles throughout the EU Member States, without setting material norms for determining and apportioning liability. France The Loi Badinter was enacted in France in 1985. 71 Its objectives are to compensate victims of all traffic accidents involving a motor vehicle, to reduce the amount of procedures before court and the large amount of ‘unfair’ transactions between insurers of motorists and victims of traffic accidents. 72 One of the major rationales for the French legislator to implement a riskbased liability system for motorists, it is stated that it is often hard, if not impossible to establish the exact cause of an accident, and the level to which a conduct or negligence of the victim had contributed to the accident. 73 The Loi Badinter installed a strict no-fault liability regime for drivers or custodians of motor vehicles which are involved in traffic accidents, obliging the drivers to compensate damages suffered by all “victims other than drivers”. 74 There is one exception: only in case an accident is caused by the inexcusable fault of the victim, the driver does not have to compensate the damages. 75 The Loi Badinter furthermore provides that if victims are younger than 16 years of age, older than 70 or at least 80% invalidated, they are to receive full compensation.

66

Fourth report on the application of Council Directive 85/374/EEC of 25 July 1985 on the approximation of the laws, regulations and administrative provisions of the Member States concerning liability for defective products amended by Directive 1999/34/EC of the European Parliament and of the Council of 10 May 1999, COM(2011), 547 final, (Fourt Report PLD) p. 9-10. 67 Idem. 68 Its attempt was to harmonize rules on liability for damage caused by motor vehicles to (nonmotorized) cyclists and pedestrians, see Van Dam 2006, at p. 369. 69 See for an overview: http://ec.europa.eu/finance/insurance/legislation/index_en.htm#maincontentSec8. 70 Directive 2009/103/EC Relating to insurance against civil liability in respect of the use of motor vehicles, and the enforcement of the obligation to insure against such liability 71 Loi “tendant à l’amélioration de la situation des victimes d’accidents de la circulation et à l’accélération des procédures d’indemnisation". 72 See A. Tunc, “The ‘Loi Badinter’ – Ten Years of Experience, Maastricht Journal of European and Comparative Law, vol. 3, 1996 (Tunc 1996), p. 329-311. 73 See Tunc 1996, p. 333: “… it is often difficult, practically, to judge it [the victim’s behaviour, RW] and decide whether it was impeccable, erroneous or faulty. Most of the time, the circumstances of the accident are not clear enough to permit a judgment”. 74 See Tunc 1996, p. 330. Article 3 reads: “Les victimes hormis les conducteurs […] sont indemnisées des dommages résultant des atteintes à leur personne qu’elles ont subis, sans que puisse leur être opposée leur propre faute”. 75 “à l ’exception de leur faute inexcusable si elle a été la cause exclusive de l’accident”. RockEU Deliverable D3.4.1 part 2

Page 26 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

The rules of the Loi Badinter have been explained by and further developed in case law. It became clear that also vehicles that are not actively involved in an accident, for instance parked cars, 76 or cars not involved in the accident at all, which were in the proximity of the accident, 77 may fall under its scope, which Tunc describes as “any kind of relation between the car and the accident is sufficient to create the insurers duty” 78 to compensate damages. The notion of inexcusable fault of a victim is narrowly construed: only if the driver can prove that a victim intentionally caused the accident and the damages related thereto, the driver (or custodian) may not be held liable. However, the case in which a victim who wore dark clothes at night, who was walking in the middle of the road, drunk, trying to fetch a lift home, and who was eventually caught by a car, still led to the liability of the driver. Tunc observes that “for practical purposes, the exception to automatic protection of non-driver victims has been eliminated from the law”. 79 It is likely that in all 6 steps of the SAE-Taxonomy, the driver or custodian of a car can be held liable under the Loi Badinter. Italy Italian civil code: Article 2054 c.c. The driver of a vehicle is obliged to compensate the damage caused to persons or the use of the vehicle, if not proof that he has done everything possible to avoid the damage. In the event of a collision between vehicles it is presumed, until proven otherwise, that each driver has contributed equally to produce the damage sustained by individual vehicles. The vehicle owner is jointly liable with the driver, unless it can prove that the use of the vehicle took place against her will. In any case the persons mentioned in the preceding paragraphs are responsible for damages resulting from construction defects or lack of maintenance of the vehicle 80. •

the vehicle is such when it is driven by a human.



the responsibility in case of accident is on the driver who has not done anything possible to avoid the damage



the responsibility in case of damages for product defects is on the driver and the owner jointly. The Netherlands

In the Netherlands, the Wegenverkeerswet (WVW) provides rules for assessing liability for damages caused by motor vehicles. Article 185 WVW installed a (semi-)strict liability system, which states that the owner or keeper of a motor vehicle can be held liable for damages to goods and persons, other than those inside a motor vehicle, caused by an accident in which

76

See for a overview of case law on this point Van Dam 2006, p. 360, footnote 28. See Van Dam 2006, p. 360. 78 Tunc 1996, p. 335. 79 Tunc 1997, p. 335. 80 (Il conducente di un veicolo senza guida di rotaie è obbligato a risarcire il danno prodotto a persone o a cose dalla circolazione del veicolo, se non prova di aver fatto tutto il possibile per evitare il danno. Nel caso di scontro tra veicoli si presume, fino a prova contraria, che ciascuno dei conducenti abbia concorso ugualmente a produrre il danno subito dai singoli veicoli. Il proprietario del veicolo, o, in sua vece, l'usufruttuario o l'acquirente con patto di riservato dominio, è responsabile in solido col conducente, se non prova che la circolazione del veicolo è avvenuta contro la sua volontà.) In ogni caso le persone indicate dai commi precedenti sono responsabili dei danni derivati da vizi di costruzione o da difetto di manutenzione del veicolo. 77

RockEU Deliverable D3.4.1 part 2

Page 27 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

a motor vehicle is involved. 81 It must be noted that the WVW addresses different actors than for instance the Loi Badinter, which appoints drivers or custodians as risk bearers. Under the WVW, keepers or owners of motor vehicles can escape from liability proving force majeure: occurrences of which not the driver but rather the victim, or a third party must be accounted for, causing the accident – and the damages following from it. Force majeure is not easily accepted by the Dutch courts: the driver of a motor vehicle must prove that he cannot be blamed, and that the accident was caused solely by “improbable and unforeseeable behaviour of others”, which the driver could not reasonably take into account. In that respect, drivers of motor vehicles have a duty to anticipate the incautious behaviour of other traffic participants. 82 The Netherlands’ system sets strict rules on the apportionment of damages to be paid to victims of accidents in which motor vehicles are involved. Under all circumstances (i.e. when force majeure cannot be proved), at least 50% of the damages needs to be compensated by the owners or keepers of the motor vehicle. In the apportionment and the allocation of damages amongst the parties involved in the accident, judges take into account if, and to what extent, the ‘own fault’ of the victim contributed to the accident. Percentages of ‘causality’ between the motorist and victim are first taken into account, assessing to what extent behaviour of both parties contributed to the origination of the accident in an objective manner, which can afterwards be ‘corrected’ if reasonableness requires so. Case law of the Supreme Court provides that this correction for reasonableness implicates that whenever a victim is younger than 14 years of age, he will receive 100% compensation, unless intent or gross recklessness by the victim can be proved. 83 For victims older than 14, the Supreme Court formulated the 50%-rule: reasonableness requires that even when the victim contributed to the accident (but not in such way that this would constitute force majeure), at least 50% of the damages are to be compensated by the motorist. The notion ‘Betriebsgefahr’ 84 forms the rationale of this rule. 85 For the other 50% of the damages, first the causality is assessed, after which correction for reasonableness can be applied. It is not likely that the advent of autonomous driving, irrespective of the SAE-Taxonomy level will lead to other outcomes of correction for reasonableness- or causality assessments, since that determines the nature and the behaviour of the victim. It can be foreseen that under all 6 SAE-Taxonomy levels owners or keepers of autonomous vehicles can (still) be held liable.

81

Compensation for damages suffered by victims inside a motor vehicle is governed by the general rules on liability, stated in article 6:162 of the Dutch Civil Code. 82 See for instance T. Hartlief, “Hoofdstuk 5 Aansprakelijkheid voor motorrijtuigen”, in: J. Spier, T. Hartlief, A.L.M. Keirse, G.E. van Maanen and R.D. Vriesendorp, Verbintenissen uit de wet en Schadevergoeding, Deventer: Kluwer 2012, (Hartlief 2012), p. 165-166. However, a bus driver who hit a bicyclist did not reasonably need to expect the behaviour of the cyclist ignoring a red traffic light, while the chauffeur drove slowly (30-35 km/h), passing a green traffic light (Supreme Court of the Netherlands 22 May 1992 (ABP/Winterthur), NJ 1992/527). 83 Supreme Court of the Netherlands 31 May 1991 (Marbeth van Uitregt), NJ 1991/721. See also Supreme Court of the Netherlands 1 June 1990 (Ingrid Kolkman), NJ 1991/720, furthermore Hartlief 2012, p. 163-164. 84 Dangers involved in and inherent to the operation of a motor vehicle on the public road. 85 Supreme Court of the Netherlands 2 June 1995 (Marloes de Vos e.a.), NJ 1997/700-702, and 5 December 1997 (Saïd Hyati e.a.), NJ 1998/400-402. The notion of ‘Betriebsgefahr’ is borrowed from the German Straβenverkehrsgesetz. RockEU Deliverable D3.4.1 part 2

Page 28 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

United Kingdom Negligence rules determine the liability for damage inflicted by motor vehicles in the United Kingdom. There is no strict liability regime in place in the UK. 86 However, the standard of care required from drivers of motor vehicles is relatively high. Negligence rules are for a significant part shaped by case law, which inter alia illustrates that a driver losing consciousness through no fault of his own is (nevertheless) acting negligently, 87 and so is the driver whose brakes fail when this failure could not have been foreseen. 88 In the UK, victims of car accidents have to prove individually that the drivers had acted negligently. When drivers are held liable for acting negligently, they may defend themselves by contending contributory negligence on the part of the victims, regardless of their age, which may well be part of the liability assessment in for instance the Netherlands. 89 Other differences from continental (civil law oriented) European liability systems are that, victims have to prove the negligent conduct of the driver who caused them harm, whereas, for instance, in the Netherlands and France there is a strict/no fault liability regime for either the owners or keepers of motor vehicles (the Netherlands), or the drivers of motor vehicles (France) who are involved in an accident. Furthermore, English drivers may invoke a contributory negligence claim, which may – in theory – completely excuse them from liability, while this is virtually impossible in France, and in the Netherlands this may lead to a maximum reduction of 50% of the liability of owners/keepers of motor vehicles causing damage to non-motorised traffic participants. It is hard to predict how the English system would respond to the advent of autonomous cars. Especially SAE-Taxonomy levels 3 (conditional automation), 4 (high automation) and 5 (full automation) could make it truly troublesome for victims to prove the negligence of ‘drivers’, since human intervention in driving no longer would have been necessary in these cases. Questions: •

Is it possible to distinguish between human driver’s responsibility and vehicle’s one? What element of proof can be brought?



How can we prevent the misuse of a highly autonomous system?



Who is responsible of the damage caused by a vehicle that has been hacked/misused? The driver of the vehicle or the operator that hacked the vehicle?

Precautionary principle The precautionary principle enables rapid response in the face of a possible danger to human, animal or plant health, or to protect the environment. In particular, where scientific

86

There is one rule of a statutory duty that – to some degree – establishes strict liability for drivers of motor vehicles approaching a crossing in the road: “The driver of every vehicle approaching a crossing shall, unless he can see that there is no pedestrian crossing, proceed at such speed as to be able, if necessary, to stop before reaching such crossing”, as cited in Van Dam 2006, at p. 365, footnote 57, referring to Reg. 3 of the Pedestrian Crossing Places (Traffic) Regulations 1941, replaced by the Zebra Pedestrian Crossing Regulations 1971, SI 1971, No. 1524. A defence that a driver has in this respect is force majeure. 87 Roberts v. Ramsbottom [1980] 1 WLR 823, also cited in Van Dam 2006, at p. 364, footnote 52. 88 Henderson v. HE Jenkins & Sons and Evans [1970] AC 282, cited in Van Dam 2006, at p. 364, footnote 53. Van Dam further takes note of Worsley v Hollins [1991] RTR 252 (CA), in which the judges held that the victim’s claim for negligence failed because the defendant could prove that although his braking systems had failed, thereby causing damage, his minibus had recently been serviced and passed its MOT. 89 Van Dam 2006, p. 375 notes that “it is hard to establish contributory negligence of young children under the age of 7.” RockEU Deliverable D3.4.1 part 2

Page 29 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

data do not permit a complete evaluation of the risk, recourse to this principle may, for example, be used to stop distribution or order withdrawal from the market of products likely to be hazardous. 90 Should it be applied to the case of self-driving cars? As part of its recent review of laws and regulations relating to driverless cars, the UK Department for Transport (DfT) said: "for cars with high automation, we consider that the situation [regarding strict liability for product defects] would not be significantly different to the current situation with technologies such as ABS [anti-lock braking] and ACC [adaptive cruise control], where malfunctioning can cause collisions and injuries. It is anticipated that the regime of strict manufacturer liability would continue to apply.” Driving and Road Traffic Laws in the UK The current legal requirements governing drivers of traditional cars would not seem appropriate in the age of the driverless car and new requirements would have to be considered if they are to be rolled out to the general public. What type of licence (if any) would be required to operate a driverless car? Should a licence be required for high automation cars in which the human would (in theory at least) never need to take control? If a licence is required, would it be permissible for newly-qualified drivers to operate a driverless car, or should a certain number of years of experience driving a traditional car be a prerequisite? What training should users have to undertake before operating a driverless car and who should provide such training? Would over-regulation in this area negate one of the key benefits autonomous vehicles could bring, the independence it could provide to those who would not be able to operate a normal car (such as the young and the disabled)? Equally, some of the existing driving rules would no longer seem relevant, such as the ban on using a mobile phone while driving 91. More generally, the basis of current road traffic laws in many countries 92 is that a driver is always required to be in control of a vehicle. Recent moves to soften this requirement to allow for the evolution of autonomous vehicles have not gone far enough. An international agreement on the best way forward would be the best approach. The UK and the US are signatories to the less strict Geneva Convention, and so may be able to satisfy the requirements with the concept of a ‘driver’ remote from the vehicle. Regulators will need to clarify the question of what constitutes ‘driver’ control. Data Protection and Cyber Security As with most other connected technologies, driverless cars will collect and analyse personal data, for example to ensure safety and analyse accidents. Also, the integrated set of systems required for a driverless car to work will be vulnerable to malicious attacks by hackers to disrupt, or take over control of, its functioning. Accordingly, many (if not all) of the data protection and cyber security issues discussed elsewhere in this edition that apply to robotics (see section 4 and edition II) will apply equally to driverless cars. Ethical Issues Much of the discussion around the ethics of driverless cars concerns the inherent absence of a human making moral decisions in extreme situations. Should a driver swerve to avoid a

90

Definition by EUR-Lex (http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=URISERV:l32042 )

91

s110 Road Vehicles (Construction and Use) Regulations 1986/1078 Vienna Convention on Road Traffic of 1968 and its predecessor the Geneva Convention on Road Traffic of 1949 92

RockEU Deliverable D3.4.1 part 2

Page 30 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

pedestrian at the risk of hitting a car on the other side of the road? Does it make any difference who the pedestrian is – a child, adult or elderly person? Many argue that an autonomous car, pre-programmed to behave in a certain way in a given situation, should not be allowed to itself make these ethical choices and that a human driver should always be present to take over the vehicle in extreme situations. Driverless car technology is already prominent in the public consciousness. This will only increase as the testing of driverless cars on the roads increases and the first commercial driverless cars become available. Any kind of accident involving a driverless car, no matter how minor, will generate a lot of publicity (certainly if the coverage of incidents involving Google’s trial cars are anything to go by). Those which involve serious injury or death in the early stages of the adoption of the technology will generate intense scrutiny and generate concern. A coherent, strategic approach to regulation – preferably at the international level – would stand the best change of striking the right balance between promoting innovations, dealing with liability issues and protecting the public. In the interim between now and full adoption, perhaps the greatest challenge to regulators is to look forward to full autonomy while partially-autonomous test vehicles are being trialled. As control is slowly removed from the driver, the driver is encouraged to give more control to the vehicle. Perhaps the most dangerous prospect is a failure to address the points on the sliding scale between full control and full autonomy without dealing with all of the potential regulatory and safety consequences presented by the rise of the autonomous vehicle.

RockEU Deliverable D3.4.1 part 2

Page 31 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

5 Robots in the healthcare 5.1 The Technical Point The European commission recently consulted with experts and stakeholders, and it has identified 21 main innovation areas that can be considered key product/market combinations and from which six representative areas can be regarded ripe for further investigation and roadmapping: •

Smart medical capsules (for endoscopy, biopsy and targeted drug delivery)



Intelligent prosthetics



Robotised patient monitoring systems



Robotised surgery (a combination of the areas related to the facilitation of the surgeon in the operating room)



Robotised motor coordination analysis and therapy



Robot assisted mental, cognitive and social therapy 93;

In these cases, it is particularly interesting to use the expression “autonomous agents” because the technological innovations concern also apps, learning and autonomous devices for telemedicine and further technologies which interact with “physical” robots.

5.2 Existing regulations Council Directive 93/42/EEC of 14 June 1993 concerning Medical Devices 94 Essential requirements: Medical devices must meet the essential requirements contained in Annex 1 of the Directive, i.e. the necessary measures to ensure a high level of safety and performance of these devices. They must not compromise the clinical condition or the safety of patients, or the health and safety of users, or, where applicable, other persons, when they are used under the conditions and for the purposes intended. They must also achieve the performances intended by the manufacturer. In addition to the general requirements set out above, these essential requirements relate to the design and manufacture of devices, including the information which must be provided by the manufacturer to allow their safe use. These instructions can now, in certain cases and under certain conditions, be provided in electronic form. EC guidelines Guidelines on the qualification and classification of standalone software used in healthcare within the regulatory framework of medical devices 95 - 2012- European Commission

93

EC-robotics for healthcare, Final Report, 2008: http://www.ehealthnews.eu/images/stories/robotics-final-report.pdf. 94 http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=URISERV:l21010b&from=IT 95 http://ec.europa.eu/health/medical-devices/files/meddev/2_1_6_ol_en.pdf

RockEU Deliverable D3.4.1 part 2

Page 32 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

5.3 Legal analysis and research goals Exploring the necessity of: •

the development of safety standards for healthcare robots



guidelines and principles that cover potentially unforeseeable consequences of the design process



enough flexibility to respond to the rapid pace of technology

For what specifically concerns Telemedicine and Robots (autonomous systems): •

Hypothesis 1 - Devices programmed to check the patient’s conditions and indicate the dosage of medicines or, more in general, what behaviour to get.



Hypothesis 2 - InTouchHealth and IRobot are creating a robot designed to provide remotely located physicians (e.g. family doctors, specialists, etc.) with insight into how their patients are recovering when treated at area hospitals. Physicians will be able to control the robot remotely via a computer and joystick. The doctor can move the robot to the patient bedside to get a first-hand look at their condition via an on-board highdefinition video camera and ask the patients a series of questions via the robot interface

Questions: •

Who is responsible if the patient does not follow the path?



Who is responsible if the patient follows the path but he does not improve his conditions or he has a deterioration of his conditions?



Responsibility of the doctor who chose the type of device for the specific patient?



For adaptive devices: Increased level of responsibility for the patient according to the length of time that has passed since he started using the device.

RockEU Deliverable D3.4.1 part 2

Page 33 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

6 Robotics, a dual technology: The case of drones. 6.1 The Technical Point - History of this technology. A significant turning point in the history of robotics took place in the 80’s with the network revolution. Information superiority was transformed into combat power. Information and 96 Communication Technologies (ICT) have become an essential tool in the C4i system . Drones, one of the instruments of this system, are the best illustration of this revolution. Integrated into the Network Centric Warfare, these Unmanned Aerial Systems (UAS) guarantee a decision-making superiority for commanders because they have information about the location and status of the enemy as well as that of friendly forces. This mastery of space is due, above all, to the channels of communication – the network and available links – which permit the drone, a simple programmable computer, to be in communication with the human who guides it on its mission. At the beginning, this computer-electric technology was military-born but its intrinsic dual nature has favored its spread into the civil sector which has invested massively in it. Because of the rate of innovation associated with defense budget reductions, as well as the constant rise in costs of new weapons systems, armies have become progressively dependent on commercially developed technologies such as ICT. This new balance “commercial/military”, calls spin-on, intensifies with the boom in the security market. Drones are an iconic example. Companies can benefit from multiple funding from different sectors for a single product and therefore adapt the drone to specific purposes. Drones can be used for hobbies, precision 97 98 99 100 agriculture, environment monitoring , transportation , surveillance , inspection , 101 102 and law enforcement . Even if the military sector isn’t a priority for these protection companies, it remains nevertheless a financial opportunity. However, even though this kind of modularity and multi-function capability is a guarantee of profits, it also represents a high risk of proliferation and unintended utilization. The use of the drone by Islamic state group to attack some friendly infrastructures has proven it103. This is why a discussion about civil drone regulations must take place between this sector AND the armies in order to bring together the compelling need for improved regulations to support development WITH the demands for control due to the dual nature of this kind of technology.

96

C4i refers to a system which includes infrastructures, person’s roles, military equipment and the way to use it. The acronym C4I stands for “Command, Control, Communication, Computers, and Intelligence and mixes different capacities for that: surveillance nonstop, information and analysis. 97 Geo-spatial mapping, weather monitoring, and air testing. 98 Medical assistant, postal packet. 99 Frontier, conflict aera. 100 Oil rig, sensitive area. 101 Surveillance, detection and reaction. 102 Surveillance and control. 103 Tabqa’s attack (23/08/2014) in Syria / Destruction of EI’s drone by USA in March,2015 RockEU Deliverable D3.4.1 part 2

Page 34 of 45

Chapter Error! Use the Home tab to apply Titre 1 to the text that you want to appear here.

6.2 Legal issues: 6.2.1 Existing regulations and Classification. In line with the definition provided by the International Civil Aviation Organization (ICAO) in 104 its “Circular 328/AN/190 on Unmanned Aircraft Systems“, the term drone refers to an uninhabited platform (called UAV, Unmanned Aerial Vehicles) and can be remotely piloted by a human (called RPAS, Remotely Piloted Aircraft Systems) or be autonomous, (« an unmanned aircraft that does not allow pilot intervention in the management of the flight »). There are many different types of drones which are classified and regulated, based on weight and use. Some specifications can be added such as systems control functions, speed, range, flight endurance. The drones above 150KG fall under European Aviation Safety Agency (AESA) Regulation (216/2008/EC) and those under 150KG fall under Member’s State Regulations.

6.2.2 France’s specific case. As a pioneer in this field, France has recently adopted a new legislation about the use of drones on 17th of December 2015. The first decree concerns the conception, the terms of use and tele pilot skills. The second regulates their integration in space area. The goal is to simplify some of the administrative procedures, i.e., the authorization and declaration schemes to ensure the security and safety of their use. This regulation defines: •

Three types of use « aeromodelling » for leisure and races; drones for « specific activities » and for experimental flights —that dictate the legal applicable regime.



The obligation for tele pilots to fly under sight, i.e., direct and constant sight. These 105 requirements will vary based on the weight of the drone and the area where it flies .



The obligation to have a fitness certificate (except for aero model 2Kg) should not fly more than 1Km from its tele pilot, therefore potentially out of direct sight of the tele pilot. S3: In inhabited area, drone (

Suggest Documents