Reactive Supervisory Control of Open Discrete-event Systems

Reactive Supervisory Control of Open Discrete-event Systems

arXiv:1804.00037v1 [eess.SP] 30 Mar 2018

Alireza Partovi, Jin Dai and Hai Lin

Abstract— Traditional supervisory control theory of discreteevent systems mainly deals with closed systems, and only ensures correct behaviors that can be generated by the system under certain environments that have been captured by the plant model. However, such control methods cannot be applied for many industrial systems nowadays since they are open systems and their behaviors heavily depend on nontrivial interactions with external environments. The key challenge on the control of these open systems is due to the need of the correctness guarantee with respect to all possible environments, which are dynamic, uncertain and sometimes unpredictable. In this paper, we are motivated to extend the conventional DES supervisory control theory to open DESs. Specifically, we consider control objectives that characterize the input-output behaviors of such an open system, and propose a game-theoretic approach to design a reactive supervisor in order to meet the specifications regardless of the external environment behaviors. Necessary and sufficient conditions for the existence of such a reactive supervisor is provided and illustrative examples are given throughout the paper to explain the key definitions and the effectiveness of the proposed game-theoretic supervisor design method.

I. I NTRODUCTION Discrete-event systems (DESs) refer to a class of dynamical systems that admit a discrete state space and evolve in accordance with abrupt occurrences of discrete events. DESs can be found in many industrial applications, ranging from intelligent manufacturing systems to transportation networks [1]. Due to the wide applications in both academia and industry, a significant amount of research efforts has been devoted to the formal verification and synthesis problems of DESs in the past two decades. On the one hand, facing an automated engineering system with complex architecture, one needs to first verify [2] whether or not the system satisfies certain property of interest. If the verification result turns to be negative, it is necessary to synthesize a supervisor or controller that enforces the property by appropriately restricting the system’s behaviors. Following the pioneering work of Ramadge and Wonham on supervisory control of DESs modeled by finite automata [3], [4], different controltheoretic architectures, such as control under partial observations [5], decentralized control [6], [7], concurrent/modular control [8], [9] and distributed control [10], [11], have been proposed. In addition to regular languages, supervisory control problem for a variety of formal specifications, such as The partial support of the National Science Foundation (Grant No. CNS-1446288, ECCS-1253488, IIS-1724070) and of the Army Research Laboratory (Grant No. W911NF- 17-1-0072) is gratefully acknowledged. The authors are with the Department of Electrical Engineering, University of Notre Dame, Notre Dame, IN, 46556 USA. [email protected],

[email protected], [email protected]

ω-regular languages [12], [13] and temporal logic formulas [14], [15], have also been investigated.

Despite the aforementioned achievements, the majority of existing results on DES supervisor synthesis were only suitable for closed systems, in which the interactions between the plant and environments were assumed to be known a priori and can be fully captured by the given uncontrolled DES plant. Nevertheless, this assumption may not be appropriate for many modern engineering systems since they are open systems that are directly exposed to and interact with dynamic and uncertain environments. Due to the inherent uncertainty of environments, the control specification for open systems need to be guaranteed with respect to all possible behaviors performed by the environments. This requirement in fact goes beyond the above-cited traditional supervisory control theory of DESs, which guarantees a given control objective only for a specified environment that can be modeled in the plant [16]. As an example of an open DES, a secure data access system that grants the users different level of access to confidential information, can be modeled as an open system [17]. The system task is to receive the users access request and grant them the access such that the low-level users can not deduce any high-level confidential data [18]. In order to meet this requirement, the system requires to reactively adjust it’s output, i.e., granting access, based on history of it’s external input, i.e., user requests [19]. This control requirement goes beyond supervisory control of DES as the supervisor is required to dynamically adjust the control strategy according to the external input behavior [16]. Hence, we are motivated to propose an open DES model whose behaviors are essentially depended on the interactions with the environment. We extend the conventional DES to a class of open DESs and the supervisory control is realized in a similar way to the reactive synthesis techniques [20], which allows the external environment to provide any subset of events at any system state. Our main objective here is to study the input-output behaviors of an open DES and develop a correct control policy that renders the system to satisfy the specifications regardless of the behaviors of the environment. Toward this aim, we first consider regular language specifications defined over the system’s input-output behaviors, and then propose a game-theoretic design approach for the reactive supervisor such that the controlled system achieves the specification regardless of the environment behavior. Our basic idea is to construct a two-player game between the external environment, and a supervisor representing all the controllable system outputs. It turns out that synthesizing the appropriate reactive supervisor can be reduced to finding a winning strategy for the supervisor player.

The rest of this paper is organized as follows. Section II provides a review over the related works from supervisory control and reactive synthesis literature. In Section III, we introduce the open DES model and define its recognized languages. The necessary and sufficient condition for the existence of a reactive supervisor is presented in Section VI. An illustrative example is utilized throughout the paper to explain the key definitions and the effectiveness of the proposed game-theoretic supervisor design method. II. R ELATED W ORK A. Reactive Synthesis Our work is closely related to the reactive synthesis rooted from computer sciences. Reactive synthesis was first introduced as Church’s problem in [21]. It basically concerns the existence of a reactive system that receives possibly (in)finite stream of input from an external environment and transforms them to an (in)finite output stream to satisfy a given specification, regardless how the environment behaves. Since then, reactive synthesis has been an active research area in the formal methods literature, see e.g., [22] and the references therein. There are two main approaches to reactive synthesis problem: one approach is game-theoretic oriented, and the main idea is to construct a two-player game between the system and its environment; the other one is based on automatatheory by translating the existence issue to an emptiness problem of a constructed tree automata. In game theoretic approaches, a correct implementation of the specification can be viewed as a winning strategy for the system player of the game [20]; whereas the tree automata-based approach alternatively suggests to construct a tree that branches according to the environment choices and the nodes labeled by an output chosen by the system. The specification then is realizable, if and only if the language of the tree automaton that accepts all the constructed trees, and satisfies the specification is non-empty [23]. It is worth pointing out that the majority of reactive synthesis literature did not consider plant model with a few exceptions, e.g., [24], which considered a plant that consisted of both system and environment states, upon which control problems were soloved for computational tree logic (CTL) specifications. In their problem setting, however, all the output events were assumed to be controllable and no controllability constraints were taken into consideration. B. I/O Automata To bridge the gap between conventional DES supervisory control theory and open systems, recent years have witnessed efforts devoted to the supervisory control of input/output DES. Non-blocking output supervisors were synthesized for DES with outputs in [25]. Yin [26] studied the supervisor synthesis technique for Mealy automata, whereas Petreczky investigated the control of input/output DES by designing a supervisor that can be realized by a Moore automaton. Among these, the closest to our work is [27] which presented

a framework for supervisory control problem of Mealy automata with non-deterministic output functions. The control objective in the aforementioned works aims to restrict the controllable input events such that the system output behavior meets a specification. The supervised plant therefore, is a closed-system and can not accept reactive specifications expressing input-output behavior of the plant. Different from conventional supervisory control of DES [28] and I/O automaton [27] that only the closed-loop system output was of concern, we considered specifications over input-output behavior of the open system, and proposed a game-theoretic method to design a reactive supervisor to meet the specifications. Necessary and sufficient conditions for existence of the reactive supervisor is provided to guarantee that the supervised plant can satisfy the specification regardless of the external environment behavior. Our work is at the intersection between supervisory control and reactive synthesis, which are two quite distinct fields and very few cross talks exist. Until recently, a comparative study on the connection between reactive synthesis and supervisory control of DES was conducted in [16], [29], which mainly focused on the conversion between a supervisory control problem and reactive synthesis problem. Our goal here is to provide a unified framework to study supervisory control of DESs with reactive specifications. Different from conventional reactive synthesis, we consider an internal plant model for the uncontrolled open DES that captures both controllable and uncontrollable output behavior of the system. III. O PEN D ISCRETE - EVENT S YSTEMS To formally define open DESs, we first review the following notations. For a given finite set (alphabet) of events Σ, a finite word w = σ0 σ1 . . . σm is a finite sequence of elements in Σ and we denote by |w| the length of w. The notation 2Σ refers to power set of Σ, that is, the set of all subsets of Σ. We denote Σ∗ the set of all finite words including the empty word ǫ. A subset of Σ∗ is called a ∗−language over Σ. The prefix-closure of a language L ⊆ Σ∗ , denoted as L, is the set of all prefixes of words in L, i.e., L = {s ∈ Σ∗ |(∃t ∈ Σ∗ )[st ∈ L]}. L is said to be prefix-closed if L = L. Definition 1: Let’s denote the input set ΣI = 2ΣX ∪ {ǫ}, and the output set as ΣO = 2ΣY ∪ {ǫ}, where ΣX , ΣY are respectively input and output alphabet. The uncontrolled open DES plant is defined by P = (Qp , Σp , ΣX , ΣY , λI , λO , qp0 , δp , Qpm ), where • Qp is the set of plant states. • Σp is the finite set of internal events which is partitioned into disjoint sets of controllable events Σc , and uncontrollable events Σuc . • ΣX is the external environment input set. • ΣY is the plant output set. • qp0 ∈ Qp is the initial state. • δp : Qp ×Σp ×ΣI → Qp is the partial transition function of the plant. • λI : Qp × Σp → ΣI is a non-deterministic function that receives the external input.

λO : Qp × Σp → ΣO is the output function which assigns deterministically an output symbol to pair of the state and events. • Qpm ⊆ Qp is the set of mark states. Without lost of generality, we assume the events sets are mutually disjoints ΣY ∩ ΣX ∩ Σp = ∅, and all the state q ∈ Q are reachable from initial state qp0 . The transition function δp is extended to a partial function on δp : Qp × Σ∗p in a standard way [30, p. 17]. We call P a closed system if ΣX = ∅, and otherwise it is called open DES. The size of an open DES is P is defined by |P| = |Qp | + |δp |. The proposed open DES formalism is syntactically similar to I/O automata of [31] and the interface automata introduced in [32]. In similar fashion, every state in open DES is allowed to be receptive toward all possible external input events, however unlike I/O and interface automata, the output behavior is defined to be a mapping function from the internal system behavior which itself is influenced by the external environment. This condition is important to be considered in reactive synthesis of open systems since the environment may restrict the system output behavior. The following definitions introduce the generated finite languages of the open DES P. Definition 2: Let’s denote all the events in the system as Σe = ΣI × Σp × ΣO , we define the projection functions, Pα = Σ∗e → Σ∗α where the index α belongs to a set {p, i, o} respectively representing internal Σp , external input ΣI , and output event set ΣO . Projection function for input-out event set also is Pio = Σ∗e → (ΣI × ΣO )∗ . Internal generated ∗−language of P is defined in a standard way over internal event set Σp . Definition 3: The generated and marked internal ∗−language of P is defined as ∗ • L(P) = {w ∈ Σp | δp (qp0 , w, x)!, for any x ∈ ΣI }, • Lm (P) = {w ∈ Σ∗p | δp (qp0 , w, x) ∈ Qpm , for any x ∈ ΣI }. The notation δp (q, σ, x)! means the function δp is defined, and is non-empty at state q ∈ Qp , σ ∈ Σp , and x ∈ ΣI . The extended input-output ∗−language of P captures the generated internal and the corresponding output behavior of the plant for any possible external environment behavior. It formally is defined as follows. Definition 4: The plant extended input-output ∗−language denotes by Le (p) ⊆ Σ∗e , and is defined recursively by: • {ǫ, ǫ, ǫ} ∈ Le (p); ∗ • ∀w ∈ (ΣI ∪Σp ∪ΣO ) , and (x, σ, y) ∈ (ΣI ∪Σp ∪ΣO ): w(x, σ, y) ∈ Le (p) ↔ w ∈ Le (p)∧ δp (q0p , Pp (w), x)! ∧ y = λO (δp (q0p , Pp (w)), σ). We can also drive the marked extended input-output ∗−language of the plant as: Le,m (P) = {∀w ∈ Le (P) | δp (q0p , Pp (w), Po (w)) ∈ Qmp }. In the design of reactive supervisor, the input-output ∗−language of the plant is our interest. Definition 5: The input-output ∗−language of the plant P is denoted by Lio (P), and the corresponding input•

output marked language is shown by Lio,m (P). They are respectively defined by: ∗ • Lio (P) = {w ∈ (ΣI × ΣO ) | w ∈ Pio (Le (p))}, ∗ • Lio,m (P) = {w ∈ (ΣI × ΣO ) | w ∈ Pio (Le,m (P))}. We illustrated the introduced open DES model and the generated ∗−languages in the following example. Example 1: Consider an open DES shown in Figure 1, where Qp = {q0 , q1 , q2 , q3 }, q0 is the initial state, and q2 , q3 are the marked state. The internal events set is Σp = {σ01 , σ02 , σ12 , σ13 }, the external input set is ΣX = {x1 , x2 }, and the output set is ΣY = {y1 , y2 }. Let the input and output power set be ΣI = {x1 , x2 , {x1 , x2 }, ǫ}, and ΣO = {y1 , y2 , {y1 , y2 }, ǫ}. The external input function is λI (q, σ) = ΣI for all q ∈ Qp which receives all subset of ΣI at any state q ∈ Qp . The definition of output function λO (q, σ) is clear in the picture. For example λ(q, σ02 ) = y2 . The generated ∗−language is L(P) = {σ02 , σ01 , σ01 σ12 , σ01 σ13 }. Some prefixes of extended input-output ∗−language are {(x1 , σ02 , y2 ), (x2 , σ02 , y2 ), (ǫ, σ02 , y2 )} ⊂ Le (P), and Some prefixes of input-output ∗−language are {(ǫ, ǫ), (x1 , y1 ), h(x1 , y1 ), (x1 , {y1 , y2 })i} ⊂ Lio (P).  q0 2

,λ I 2

(q 0

,σ 0

start

y2 ) }/ {σ01 ,λI (q0 ,σ01 )}/y1

{σ 0

q2

{σ12 ,λI (q1 ,σ12 )}/{y1 ,y2 }

q1

{σ13 ,λI (q1 ,σ13 )}/y1

q3

Fig. 1: Open discrete event system for Example 1. IV. S EQUENTIAL I NPUT- OUTPUT B EHAVIOR Finite reactive specification defines a relational between an external input and an output set such that for any subset of input this relationship holds. In the reactive synthesis formalism, the input-output relationship is typically defined over a Mealy or Moore transducer [33]. Definition 6 (Mealy transducer): A Mealy transducer is a tuple K = (Qk , q0k , ΣI , ΣO , δk , γk , Qmk ), where Qk is the set of states, q0k ∈ Qk is the initial state, ΣI and ΣO are respectively disjoint set of input and output event sets, δk : Qk ×ΣI → Qk is the transition function, γk : Qk ×ΣI → ΣO is the state output function, and Qmk ⊆ Qk is the marked states. We assumes ǫ ∈ ΣI and the output function for empty input is empty ǫ = γk (q, ǫ). An accepted run in K over a finite input word x = x0 , x1 , . . . , xn ∈ Σ∗I , is a r = q0 , q1 , . . . , qn , where q0 = q0k , and qi+1 = δk (qi , xi ) for i ∈ {0, . . . , n − 1}. The run r over input word wx generates a finite word L(wx ) ∈ (ΣI ∪ ΣO )∗ such that L(wx )i = γk (qi , xi ) ∪ {xi } for i ∈ {0, . . . , n − 1}. The input-output language or simply language of K is L(K) = {L(x) | ∀x ∈ Σ∗I }, and the marked language Lm (K) = {L(x) | δk (q0k , x) ∈ Qmk , ∀x ∈ Σ∗I }. We call ∗−language K ⊆ (ΣI ∪ ΣO )∗ a finite reactive specification

if it can be realized by a transducer K such that K = L(K), and K = Lm (K). The open DES plant at the state q ∈ Qp , sequentially executes an available internal event σ ∈ Σp , reads the environment input x ∈ ΣI , transits according to the transition function δp , and generates the output events through λO (q, σ, x). We use a notation of sequential input-output relationship to formalize this behavior [34]. Definition 7 (Sequential input-output relation): A sequential input-output relation is a function R : ΣI ∗ → Σ∗O if the following conditions hold: C1 : For all w ∈ (ΣI )∗ , R(w) 6= ∅. C2 : For all wi ∈ (ΣI )∗ , and wo ∈ Σ∗O , wi and wo have the same length. C3 : R is prefix preserving. The above conditions ensure R is non-empty and preserve a same length for input and output words. Sequential inputoutput properties ensures that an open DES can be modeled by a finite-state Mealy or Moore transducer [33]. This property allows the system to accept a reactive specification. Proposition 1: Plant P has a sequential input-output relation with respect to λI and λO . Proof: Let’s consider the plant transition function δp : Qp × Σp × ΣI → Qp . For all wx ∈ Σ∗I let’s defines a relationship function Λ(wx ) = {wo ∈ Σ∗O | λO (δp (qp0 , σ, wx )!, ∀σ ∈ Σp }. Λ meets C1 since Λ(ǫ, ǫ) = ǫ as λO (q, σ) 6= ∅ for all σ ∈ Σp based on the output function λO definition. The relation Λ also meets C2 since a given input wx ∈ Σ∗I , the |Λ(wx )| = |λO (δp (qp0 , σ, wx )| for wp ∈ Σ∗p , and hence |Λ(wx )| = |wx |. Furthermore C3 also holds by wy = Λ(wx ) since (wx , wy ) ∈ Lio (P), and Lio (P) is prefix-closed. Example 2: Consider the open DES shown in Figure 1, with input ΣI = {x1 , x2 , {x1 , x2 }, ǫ}, and output sets ΣO = {y1 , y2 , {y1 , y2 }, ǫ}. Since λI (q, σ) = ΣI for all q ∈ Qp , the plant can receive any subset of external event x ∈ ΣI in any state. The transducer with respect to input-output behavior of plant is illustrated in Figure 2. For clarity of the figure, any two edges xi /y and xj /y are combined, and represented by {xi , xj }/y for xi , xj ∈ ΣI , and y ∈ ΣO .  start {x 1

q0

,x 2 }

/y 1

q1

{x1 ,x2 }/{y1 }

q3

{x1 ,x2 }/{y1 ,y2 }

{x

1 ,x 2 }/

y2

q2 Fig. 2: Mealy Transducer for the plant of Example 1. V. R EACTIVE S UPERVISORY C ONTROL P ROBLEM The architecture of reactive supervisory and an open DES is depicted in Figure 3. In the proposed control architecture, the reactive supervisor observes the history of internal generated ∗−language Σ∗p , the environment input Σ∗I , the plant output Σ∗O , and selects a control pattern θ ∈ Θ for the

plant. The open DES then only executes a state transition if it is not restricted by the environment, and it is allowed by the supervisor. Intuitively control pattern is a set of events that the supervisor enables at any given state of the plant. The supervisor is not allowed to disable any uncontrollable event. The subset θ ⊆ Σp is called control pattern, and Θ = {θ ∈ 2Σp | Σu ∈ θ} is set of all control pattern. The reactive supervisor is defined as a function S : Le (P) → Θ. Controller Σ∗I

Σ∗p

Θ Plant

Σ∗O

Fig. 3: Reactive Supervisory Control Architecture. Let’s denote the supervised plant P controlled by the reactive supervisor S by S/P, which formally is defined as follows. Definition 8: The extended input-output ∗−language of supervised plant Le (S/P) is recursively defined by: • {ǫ, ǫ, ǫ} ∈ Le (S/P); ∗ • ∀w ∈ Σe , and (x, σ, y) ∈ Σe : w(x, σ, y) ∈ Le (S/ P) ↔ w(x, σ, y) ∈ Le (P) ∧ σ ∈ S(w), and the marked set of that is Le,m (S/P) = Le (S/P) ∩ Le,m (P). The input-output ∗−language of S/P also denotes by Lio (S/P) = {w ∈ (ΣI ∪ ΣO )∗ | w ∈ Pxy (Le (S/P))}, and the marked set Lio,m (S/P) = Lio (S/P) ∩ Lio,m (P). Given a reactive specification K for a plant P, the reactive supervisor task is to control the internal controllable events such that the input-output behavior of supervised plant meets the specification, Lio,m (S/P) = K. Note that the reactive specification already included the all possible external inputs words, therefore, if supervised plant achieves the specification, it holds for all environment behavior. An important extra requirement for supervisory control of DES is a non-blocking property [3]. The non-blocking condition ensures that the supervisor does not disable all the controllable events in the state which results a deadlock state in the supervised plant. In other word, all the prefix of generated language should be able to be extended to the marked states Lm (S/P) = L(S/P) [3]. In open DESs however, the external input may also limit L(S/P). Example 3: Consider the open DES shown in Figure 1, with having Σc = Σp . Let the reactive supervisor be  σ01 w ∈ {(ΣI , ǫ, ǫ)}    σ13 w = (x1 , σ01 , y1 ) S(w) = ǫ w ∈ {(x2 , σ13 , ΣO ), (ΣI , σ02 , ΣO )},    Σc Otherwise. The generated internal ∗−language for supervised plant are L(S/P) = {σ01 , σ13 , σ02 , {σ01 σ13 }}, and the corresponding marked ∗−language is Lm (S/P) = {σ02 , {σ01 σ13 }}. The conventional non-blocking condition

holds here, Lm (S/P) = L(S/P), however, the environment can force the supervisor to block the plant at q1 by only providing input word w ∈ {xx2 ∗ x}, for x ∈ Σ∗I .  As Example 3 illustrated, we need stronger definition of nonblocking that can be held for any environment behavior. We define this requirement over history of supervised open DES as it captures all possible generated ∗−language of the plant for any given environment input. Definition 9: History of open DES is a finite sequence (q0 , x0 , σ0 ), . . . , (qn , xn , σn ), where σi ∈ Σp , xi ∈ λI (qi , σ), and it is accepted by P if q0 ∈ q0p and δp (qi , σi , xi )! for all i ∈ {0, . . . , n}. Let’s denote H(P) a set of all histories accepted by P. A state q ∈ QP is called deadlock state if (q, σ, q ′ ) 6∈ δp for all q 6= q ′ and q ′ ∈ Qp , and history (q0 , x0 , σ0 ), . . . , (qn , xn , σn ) is a deadlock history if the last state, qn , is a deadlock state. Let’s define projection function PΣp : H(P) → Σ∗p as PΣp (wh ) = {σ0 σ1 . . . σn | wh = (q0 , x0 , σ0 ), . . . , (qn , xn , σn ) ∈ H(P)}. The following lemma provides the non-blocking conditions on open DESs. Lemma 1: [Non-blocking] The Non-blocking condition Lm (S/P) = L(S/P) holds if and only if H(S/P) is deadlock free. Proof: If H(S/P) is deadlock free Lm (S/P) ⊆ L(S/P) = PΣp (H(S/P)) always hold by definition. We need to show L(S/P) ⊆ Lm (S/P). If H(S/P) is deadlock free, for any s ∈ L(S/P), we have w ∈ H(S/P) such that PΣp (w) = s. For w there exist a w′ ∈ PΣp (H(S/P)) ∩ Lm (S/P) in which w ∈ w′ . Hence s = PΣp (w) ∈ PΣp (w′ ) ⊆ Lm (S/P). The control objective a non-blocking reactive supervisor is to regulate the internal behavior of plant such that the input-output ∗−language satisfies a reactive specification. This problem is formally defined as follows. Problem 1: Given a non-empty control specification K ⊆ Lio,m (P), uncontrolled open DES P, and control pattern set Θ, synthesis a non-blocking reactive supervisor S : Le (P) → Θ, such that K = Lio,m (S/P). VI. T HE E XISTENCE OF S UPERVISOR In Ramadge-Wonham supervisory control of DES, controllability of specification is a necessary condition for existence of the supervisor [4]. The conventional controllability of ∗−language specification K, is defined with respect to uncontrollable events Σu , and plant internal ∗-language L(P), and is defined as K(Σu ) ∩ L(P) ⊆ K. This controllability requirement is defined over closed-loop system output, the reactive specification however, expresses the input-output behavior of the system. Specifically, the uncontrollability of output language in open DES not only depends on internal uncontrollable events, but also can cause by an external input word that enforces execution of an undesired output event. Let’s define uncontrollable input-output behavior of the plant as Σio u = {(x, y) | x ∈ ΣI , δk (q, σu , x)! ∧ y ∈ λO (q, σu ), σu ∈ Σu , q ∈ Qp }. Set Σio u basically captures the plant output corresponding to the uncontrollable internal events. The input-output controllable language in reactive supervisory control formally is defined as following.

Definition 10 (Output Controllable Language): A nonempty reactive specification K ⊆ Lio (P) is called output controllable with respect to input-output event set ΣI ∪ ΣO , and input-output language Lio (P), if K(Σio u ) ∩ Lio (P) ⊆ K In addition to output controllability condition, the control specification also is required to be closed. We call a reactive specification K closed with respect to Lio,m (P), if Po (K) = Po (K) ∩ Po (Lio,m (P)). Note that controllability a reactive specification is only depend on the plant’s output language since the input component Pi (K), is generated by the external environment, and any input word wx ∈ Σ∗I , by the definition of reactive specification, it should be included in the specification, i.e., wx ∈ Pi (K). With these ingredients, we provide the necessary conditions for existence of a reactive supervisory control in the following lemma. Lemma 2: [Necessary Condition] If there exist a nonblocking reactive supervisor S such that Lio,m (S/P) = K, then K is output controllable and is closed with respect to Lio,m (P). Proof: Since the supervisor is non-blocking, we have K ∩ Lio,m (P) = Lio,m (S/P) ∩ Lio,m (P) = Lio,m (S/P) = K, and hence K is closed with respect to Lio,m (P). Consider any w ∈ Po (K), x ∈ ΣI , and σ ∈ (ΣI ∪ ΣO ) in which (x, w)σ ∈ K(ΣI ∪ ΣO ) ∩ Lio,m (P). Since (x, w) ∈ K = Lio (S/P), there exist h ∈ H(S/P) such that PΣp (h) = a. Since a 6∈ S(h), we have Po (h(q, x, a)) ∈ Lio (S/P) = K, and hence K is output controllable. A. Game-based Reactive Supervisor Design We present a game-theoretic method to check if the reactive supervisory can achieve the reactive specification without blocking. We proposed to design a twoplayer turned-based game between the external environment and the reactive supervisor. For given open DES P = (Qp , Σp , ΣX , ΣY , λI , λO , qp0 , δp , Qpm ), and control pattern Θ, consider a game arena G = (Qg , δg , Σg , q0g ), where QG is state of the game, and is partitioned into two disjoint set of Qe and Qs representing environment and supervisor states respectively. The event set is Σg := ΣI ∪ Σp ∪ ΣO ∪ {σ⊥ }, and the transition function is defined as δg := δe ∪ δs ∪ δ⊥ where δe ⊆ Qe × ΣI × Qs , and δs ⊆ Qs × Σp × Qe . The rests are defined as following, e • Qe := {qi | i ∈ {1, . . . , |Qp |} ∪ q0p ∪ q⊥ . s • Qs := {(qi , θ, x) | i ∈ {1 . . . |Qp |}, ∀θ ∈ Γ, ∀x ∈ ΣI }. • q0g := q0p ∈ Qe is the initial state. e e • δe (qi , x) = {(qi , θ, x) ∈ Qs | qi ∈ Qp , qi ∈ Qe , ∀x ∈ ΣI , ∀θ ∈ θ}. e • δs ((q, θ, x), σ) = qi ∈ Qe | (q, θ, x) ∈ Qs , σ ∈ θ, qi = δp (q, σ, x)}. • δ⊥ ((q, θ, x), σ⊥ ) = q⊥ ∈ Qe | (q, θ, x) ∈ Qs , δp (q, σ, x) = ∅ ∀σ ∈ θ}. The transition function δe represents the transitions according to all possible subset of environment input, and δs defines the transitions according to supervisor choice to enforce an output. In the constructed game a deadlock occurs when the supervised plant enters a state that there is no

enabled event or when the supervisor selects a control pattern that disables all possible events at the state. The event σ⊥ and transition function δ⊥ captures the DES enters deadlocks. The next step is to product the game arena G and the specification transducer. Given game arena G, and a reactive specification K = (Qk , q0k , ΣI , ΣO , δk , γk , Qmk ), the product ˜ g = Qg × Qk , δ˜g , Σe , (q0g , q0k )). game is defined as G˜ := (Q k k Consider q , s ∈ Qk , ∀y ∈ ΣO , and ∀x ∈ ΣI , the transition function δ˜g is constructed as following: e ˜g: • For all q ∈ Qe ⊆ Q e k ˜ δg (q , q , x) = {((q, θ, x), sk ) | ∀(q, θ, x) ∈ δe (q e , x) ∧ δk (q k , x) = sk }, ˜ g , and ∀σ ∈ θ: • For all (q, θ, x) ∈ Qs ⊆ Q k e δ˜g ((q, θ, x), q , σ) = {(q , sk ) | q e ∈ δs ((q, θ, x), σ) ∧ λO (q, σ) = γk (q k , x)}. ˜ gm , and is defined as The accepting state in G˜ denotes by Q ˜ gm = {(q, θ, x) × Qkm | q ∈ Qm , (q, θ, x) ∈ Q ˜ g } ∪ {q e × Q i ˜ g , qi ∈ Qm }. Qkm | qie ∈ Qe ⊆ Q Example 4: Consider plant model in Figure 1. Let’s assume Σuc = {σ01 }, and the remaining events are controllable. the control pattern θ1 = {σ01 , σ02 }, and θ2 = {σ01 , σ12 }. The reactive specification is K = {(ǫ, ǫ), {(x, y1 ), (x, {y1 , y2 })}, {(ǫ, y1 ), (ǫ, {y1 , y2 })}, {(ǫ, y1), (x, {y1 , y2 })}, {(x, y1 ), (ǫ, {y1 , y2 })}}. The product game for plant in Example 1 and reactive specification K is shown in Figure 4. The circle are the environment states and the square are the reactive supervisor states. The environment inputs are e1 = {ǫ}, e2 = {x1 }, e3 = {x2 }, e4 = {x1 , x2 }. The output labels are only shown for few edges for clarity of the game arena figure.  g0 e1 (q0 , θ1 , e1 )

σ02 , y2

e3

(q0 , θ2 , e2 )

σσ02 01

start

σ01

(q0 , θ1 , e3 )

σ01

g2 σ12 , {y1 , y2 } (q1 , θ1 , e3 )

σ12

e4

e2

(q0 , θ2 , e4 )

σ01 , y1

g1 e2

(q1 , θ2 , e4 )

e4

e1

(q1 , θ1 , e1 )

e3

(q1 , θ2 , e2 )

Fig. 4: Game construction for Example 4. The product game G˜ is a two player reachability game to visit marked states. The set runs over this game produce all the extended input-output behavior of the plant that are compatible the reactive specification K. Formally, let’s ˜ ⊆ Le (P) which inductively is defined as: denote it as Ck (G)

˜ {ǫ, ǫ, ǫ} ∈ Ck (G); ′ ∗ io • ∀w ∈ Σe , x, ∈ ΣI , yc ∈ Σo , σc ∈ Σu , qe , qe ∈ Qe ⊆ Qg , and (q, θ, x) ∈ Qs ⊆ Qg : ˜ ↔ w(x, σc , yc ) ∈ Ck (G) ˜ ∧ w(x, σc , yc ) ∈ Ck (G) ′ ˜ ˜ δg (qe , (q, θ, x), x)! ∧ δg ((q, θ, x), qe , σc )! ∧ yc = λO (q, σc , x). The notation of realizability of a specification in reactive synthesis formalism is the existence of an reactive system that satisfies the specification for any environment behavior. Here we adapt this definition to existence of a reactive ˜ and can extend supervisor S that characterize the set Ck (G) the history of the game to the mark states. The SP-realizable notation is given as following. Definition 11: A reactive specification is called K ⊆ Lio,m realizable with respect to Le (P) and reactive super˜ 6= ∅, and for all visor S, in short SP-realizable, if Ck (G) ˜ there exist a we ′ ∈ we such that we ′ ∈ K. we ∈ Ck (G) We show in Theorem 1 that SP-realizability beside the output controllability and language closeness, are necessary and sufficient conditions for the existence of a non-blocking reactive supervisor. Theorem 1: Let a non-empty reactive specification be K ⊆ Lio,m (P), reactive supervisor S satisfies Lio,m (S/ P) = K without blocking if and only if K is output controllable, language closed with respect to Lio,m (P), and ˜ is SP-realizable. Ck (G) Proof: Necessity: Since it is assumed there exist a reactive supervisor that satisfies the specification, Lio,m (S/ P) = K, according to Lemma 2, K is output controllable and language close. Furthermore it is SP-realizable by the definition. Sufficiency: Here we need to prove if K is output controllable, closed with respect to Lio,m (P), and SP-realizable 6 ∅ then Lio,m (S/P) = K. We first know that Lio (P) ∩ K = since K is assumed to be non-empty, and furthermore the plant respects the sequential input-output condition proven in Proposition 1, hence (ǫ, ǫ) ∈ Lio (P) ∩ K. For any σu ∈ Σio u and w ∈ Lio (P) ∩ K, let’s first assume wσu ∈ Lio (P). We have wσu ∈ K since K is output controllable K(Σio u)∩ Lio (P) ⊆ K. If σc ∈ (ΣI ∪ ΣO ) − Σio , then there exist u we σe ∈ Le (S/P) such that Pio (we ) = w and Pp (σe ) = σp ∈ Σp , and since by the definition of extended input-output language of supervised plant Pio (Le (S/P)) ⊆ K, we thus have Lio (S/P) ⊆ K. Now let’s consider wσ ∈ K then since ˜ is SP-realizable, we have we σe ∈ Ck (G) ˜ such that Ck (G) Pio (we σe ) = wσ ∈ Pio (Le (S/P)). Therefore Lio (S/P) = K. Moreover, since K is closed with respect to Lio,m (P), we have Lio,m (S/P) = Lio (S/P) ∩ Lio,m (P) = K. •

The supervisor strategy is to observe the history of the game and select a control pattern such that it reaches the ˜ The supervisor is called accepting states of game arena G. winning if it has winning strategy at initial state (q0g , q0k ). Solving G˜ is a standard reachability problem for two-player game with complexity of PSPACE-complete, and if exists, a finite realization of supervisor can be obtained [35].

VII. C ONCLUSION In this paper, we studied a reactive synthesis problem for open DES with finite regular language specifications. The proposed DES is a reactive system with deterministic internal behavior and non-deterministic input function. The control objective for the supervisor is to achieve the specification regardless of how the environment behaves. We provided a necessary and sufficient conditions for existence of the reactive supervisor. If the condition holds, a two-player game arena can be constructed to drive a finite state representative the supervisor. In our future work, we will study reactive supervisory control with other specifications such as ω−regular language and LTL formulas. Moreover, we will extend this study to open DESs with non-deterministic transition function and satisfaction with respect to bi-simulation. R EFERENCES [1] C. G. Cassandras and S. Lafortune, Introduction to Discrete Event Systems, 2nd ed. New York: Springer, 2008. [2] C. Baier, J.-P. Katoen, and K. G. Larsen, Principles of model checking. Cambridge: MIT press, 2008. [3] P. J. Ramadge and W. M. Wonham, “Supervisory control of a class of discrete event processes,” SIAM J. Control Optim., vol. 25, no. 1, pp. 206–230, 1987. [4] ——, “The control of discrete event systems,” Proc. of the IEEE, vol. 77, no. 1, pp. 81–98, 1989. [5] R. Cieslak, C. Desclaux, A. S. Fawaz, and P. Varaiya, “Supervisory control of discrete-event processes with partial observations,” IEEE Trans. Autom. control, vol. 33, no. 3, pp. 249–260, 1988. [6] K. Rudie and W. M. Wonham, “Think globally, act locally: Decentralized supervisory control,” IEEE Trans. Autom. control, vol. 37, no. 11, pp. 1692–1708, 1992. [7] T.-S. Yoo and S. Lafortune, “A general architecture for decentralized supervisory control of discrete-event systems,” Discrete Event Dynam. Syst., vol. 12, no. 3, pp. 335–377, 2002. [8] Y. Willner and M. Heymann, “Supervisory control of concurrent discrete-event systems,” Int. J. Control, vol. 54, no. 5, pp. 1143–1169, 1991. [9] B. Gaudin and H. Merchand, “An efficient modular method for the control of concurrent discrete event systems: A language-based approach,” Discrete Event Dynam. Syst.: Theory Applicat., vol. 17, no. 2, pp. 179–209, 2007. [10] K. Cai and W. M. Wonham, “Supervisor localization: a top-down approach to distributed control of discrete-event systems,” IEEE Trans. Autom. Control, vol. 55, no. 3, pp. 605–618, 2010. [11] J. Komenda, T. Masopust, and J. H. van Schuppen, “Coordination control of discrete-event systems revisited,” Discrete Event Dynam. Syst.: Theory Applicat., vol. 25, no. 1-2, pp. 65–94, 2015. [12] P. J. G. Ramadge, “Some tractable supervisory control problems for discrete-event systems modeled by b¨uchi automata,” IEEE Trans. Autom. Control, vol. 34, no. 1, pp. 10–19, 1989. [13] R. Kumar, V. Garg, and S. I. Marcus, “On supervisory control of sequential behaviors,” IEEE Trans. Autom. Control, vol. 37, no. 12, pp. 1978–1985, 1992. [14] J.-Y. Lin and D. Ionescu, “A reachability synthesis procedure for discrete event systems in a temporal logic framework,” IEEE Trans. Syst., Man, Cybern., vol. 24, no. 9, pp. 1397–1406, 1994.

[15] S. Jiang and R. Kumar, “Supervisory control of discrete event systems with ctl* temporal logic specifications,” SIAM J. Control Optimiz., vol. 44, no. 6, pp. 2079–2103, 2006. [16] R. Ehlers, S. Lafortune, S. Tripakis, and M. Y. Vardi, “Supervisory control and reactive synthesis: a comparative introduction,” Discrete Event Dynam. Syst.: Theory Applicat., vol. 27, no. 2, pp. 209–260, 2017. [17] R. Dimitrova, B. Finkbeiner, M. Kov´acs, M. N. Rabe, and H. Seidl, “Model checking information flow in reactive systems,” in International Workshop on Verification, Model Checking, and Abstract Interpretation. Springer, 2012, pp. 169–185. [18] M. Bishop, Computer security: art and science. Addison-Wesley Professional, 2003. [19] J. Rushby, Noninterference, transitivity, and channel-control security policies. SRI International, Computer Science Laboratory, 1992. [20] A. Pnueli and R. Rosner, “On the synthesis of a reactive module,” in Proc. 16th ACM SIGPLAN-SIGACT Symposium on Principles of programming languages. ACM, 1989, pp. 179–190. [21] A. Church, “Applications of recursive arithmetic to the problem of circuit synthesis,” Summaries of the Summer Institute of Symbolic Logic, vol. 1, pp. 3–50, 1957. [22] B. Finkbeiner, “Synthesis of reactive systems.” Dependable Software Systems Engineering, vol. 45, pp. 72–98, 2016. [23] M. O. Rabin, “Automata on infinite objects and church’s problem.” American Mathematical Society, Boston, MA, USA, 1972. [24] O. Kupferman, P. Madhusudan, P. S. Thiagarajan, and M. Y. Vardi, “Open systems in reactive environments: Control and synthesis,” in Proc. 11th Conf. Concurrency Theory (CONCUR 2000). Springer, 2000, pp. 92–107. [25] P. Mahdavinezhad, P. Gohari, and A. G. Aghdam, “Supervisory control of discrete-event systems with output: Application to hybrid systems,” Int. J. Control, vol. 81, no. 9, pp. 1451–1461, 2008. [26] X. Yin, “Supervisor synthesis for mealy automata with output functions: A model transformation approach,” IEEE Trans. Autom. Control, vol. 62, no. 5, pp. 2576–2581, 2017. [27] T. Ushio and S. Takai, “Nonblocking supervisory control of discrete event systems modeled by mealy automata with nondeterministic output functions,” IEEE Trans. Autom. Control, vol. 61, no. 3, pp. 799–804, 2016. [28] R. Kumar and V. K. Garg, Modeling and Control of Logical Discrete Event Systems. Boston: Kluwer, 1995. [29] R. Ehlers, S. Lafortune, S. Tripakis, and M. Vardi, “Bridging the gap between supervisory control and reactive synthesis: Case of full observation and centralized control,” IFAC Proceedings Volumes, vol. 47, no. 2, pp. 222 – 227, 2014, 12th IFAC International Workshop on Discrete Event Systems (2014). [30] J. E. Hopcroft, Introduction to Automata Theory, Languages, and Computation, 3rd ed. Pearson Addison Wesley, 2007. [31] N. A. Lynch and M. R. Tuttle, “Hierarchical correctness proofs for distributed algorithms,” in Proceedings of the sixth annual ACM Symposium on Principles of distributed computing. ACM, 1987, pp. 137–151. [32] L. De Alfaro and T. A. Henzinger, “Interface automata,” in ACM SIGSOFT Software Engineering Notes, vol. 26, no. 5. ACM, 2001, pp. 109–120. [33] S. Eilenberg, Automata, languages, and machines. Academic press, 1974. [34] M. Petreczky, R. J. M. Theunissen, R. Su, D. A. van Beek, J. H. van Schuppen, and J. E. Rooda, “Control of input/output discrete-event systems,” in Proc. 2009 European Control Conference (ECC). IEEE, 2009, pp. 1967–1972. [35] C. L. Barrett, H. B. Hunt III, M. V. Marathe, S. Ravi, D. J. Rosenkrantz, and R. E. Stearns, “Complexity of reachability problems for finite discrete dynamical systems,” Journal of Computer and System Sciences, vol. 72, no. 8, pp. 1317–1345, 2006.