Towards Shape-Changing Devices: Physical Interface Control ... - MDPI

SS symmetry Article

Towards Shape-Changing Devices: Physical Interface Control with an Active Contour Model Byung-Kil Han 1 , Seung-Chan Kim 2 and Dong-Soo Kwon 1, * 1 2

*

Department of Mechanical Engineering, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Korea; [email protected] Department of Interdisciplinary Studies, Hallym University, 1 Hallymdaehak-gil, Chuncheon-si, Gangwon-do 24252, Korea; [email protected] Correspondence: [email protected]; Tel.: +82-42-350-3082

Received: 30 November 2017; Accepted: 21 February 2018; Published: 1 March 2018

Abstract: This paper proposes a novel tangible interface system to enhance the immersive experience in virtual reality environments. The proposed system allows representing physical properties of the tool, such as the deflection of the elastic rods, which users handle in a virtual and physical environment. This system is composed of two parts; one is an articulated interface to visually represent the physical behavior and the other is a computational algorithm that can compute a set of 6-DOF positions of the links. The proposed computational algorithm extends an active contour model, which is used primarily in computer vision and image processing, incorporating a spring and damping constraint energy functional. An elastic rod is modeled as a series of rigid line segments with a symmetric relationship between neighboring segments, and its shape is modeled to be influenced by energies that are induced by a user, and the external deformation of the spline. The symmetric and sparse properties of the proposed model enable an efficient energy minimization process, and the modification of a number of the line segments. Based on this configuration, we construct an energy generation method based on the positional displacement of the base element to generate the deflection behavior of the contour according to the user’s motion in the space. Therefore, the physical device can simulate a variety of deformable objects by modulating energy parameters during the energy minimization process. Experimental results demonstrate the feasibility of emulating various behaviors of deformable splines, and applying to virtual reality system without interfering with the motion-to-photon latency. We also discuss the method’s limitations and explore its potential. Keywords: mobile virtual reality; active contour model; robotic interface; shape changing interface; immersion

1. Introduction The aim of a virtual reality system is to offer a variety of experiences similar to the physical world. Conventional virtual reality systems have focused on visually representing users and their environment using head-mounted displays and arrays of sensors [1,2]. The interface connecting a user to a virtual world requires a large workspace and, thus, reducing a system’s form factor to make it portable and lightweight has been key. A large variety of mobile applications that facilitate gesture-command functionalities within simple 3D worlds and complicated 3D games are being released into the market for mobile platforms. Although these technologies enable enhanced interaction with 3D media, such conventional interfaces offer a limited experience to users by restricting the representation of virtual geometry [3] and producing unnatural physical behavior [4]. To resolve these limitations, haptic feedback is often employed to provide realistic sensations during immersive interactions. In particular, haptic rendering generates a reaction force and torque feedback that prevents the penetration of the user’s fingertip into the object [5,6]. Based on this Symmetry 2018, 10, 57; doi:10.3390/sym10030057

www.mdpi.com/journal/symmetry

Symmetry 2018, 10, 57

2 of 13

algorithm, a variety of haptic devices have been developed, which provide a virtual presence to the users. Similarly, the information transferred by mobile devices is focused on the contact between users and virtual objects, and the reaction force is substituted to a cutaneous force [7] by representing a user’s hand to a single point or series of points. This method supports the penetration of a user’s hand into an object, and is acceptable for the selection and manipulation of objects, but makes it difficult to express events lacking contact or collision. To overcome this shortcoming, enhancing the expressiveness of mobile devices has been considered. Studies have focused on changing the form factor of mobile devices from rigid to malleable to express information virtually and physically. This enables bi-directional interaction between the physical and virtual worlds, significantly enhancing usability and the realism of the interaction with a virtual world [8]. This paper proposes an actuated interface system to enrich realism and immersion in virtual environments. This system includes a computational algorithm that models a flexible rod into a series of rigid lines with physical properties. Using the active contour model as a baseline algorithm, the method can generate a variety of physical behaviors according to the user’s position input. Therefore, it can be applied for an intuitive control of a line-shaped graphical object, or for enhancing the level of realism during a mid-air interaction by driving articulated shape-changing interfaces. Specifically, the proposed method enables the simulation of a rod with various mechanical properties; therefore, the graphical object or the articulated device can dynamically change its shape into a fencing sword, a whip, or other shapes. We modeled the interface device as a set of serially-linked splines with spring-damper connections. This paper is organized as follows: Section 2 outlines mobile interfaces for virtual reality environments. Section 3 describes the proposed method to develop an actuated interface based on active contour model. Section 4 analyzes the possibilities of providing flexible behavior using the proposed method. Section 5 discusses applications of the proposed method, and Section 6 draws conclusions and discusses directions for future research. 2. Related Research To enhance a user’s immersive experience in virtual environments, several studies have proposed interfaces capable of conveying a variety of information, focusing on information expression via haptic feedback and actuated shape changes. In haptic feedback, the representation of digital information—especially geometric information—relies on reaction forces and torque feedback generated by mechanically-grounded linkages [9–11]. Those linkages can be applied for realistic virtual geometric representation [12–14], and for assisting the user’s operation from remote locations by applying virtual fixtures [15,16]. However, since mobile interfaces are mechanically ungrounded, they cannot generate a reaction force that can resist a user’s movement; thus, feedback is, instead, provided by cutaneous and inertial forces. Portable haptic interfaces also aim to simulate the experiences of touching a virtual object with a user’s hand. They convey geometric information relating to virtual objects in terms of cutaneous forces by calculating the penetration depth between a user’s hand and the object. The calculated cutaneous force is transferred on the fingertip via various techniques, such as a push mechanism [17,18] or tilting platform [19]. Inertial forces can also be generated by angular momentum provided by flywheel rotation [20,21]. These methods can deliver information regarding virtual object geometry even when devices are ungrounded. However, they only contact a virtual object, and cannot deliver information when the user is away from the object. Instead of providing virtual object geometry information during contact, other studies have focused on tool characteristics, such as flexibility and form factor, based on the notion that actuated interfaces can significantly enhance the usability or realism of interaction with a virtual world [8]. Physical shape changes can offer a variety of information to a user [22], and can be achieved by point [23,24] and plane actuation [25,26]. For curved interfaces, Katsumoto et al. [27] proposed an interface that can change its shape and flexibility, composed of multiple passive joints and utilizing a shape locking mechanism to hold the current state. It can switch from a flexible to rigid structure, and

Symmetry 2018, 10, 57

3 of 13

Symmetry 2018, 10, x FOR PEER REVIEW

3 of 14

vice versa. With this, the user can modulate the interface’s shape into a variety of forms. It has a simple passive structure, but cannot information properties other than flexible or rigid locking mechanism to hold provide the current state. It canabout switchphysical from a flexible to rigid structure, and vice states. Nakagaki al.user proposed a shape-changing byofaforms. seriesItofhas linkages versa. With this,etthe can modulate the interface’sinterface shape intoformed a variety a simpledriven passiveactuators structure, that but cannot provide information otherathan flexible or by rotary can form various shapes.about This physical interfaceproperties can provide variety of physical rigid states. Nakagaki et al. proposed a shape-changing interface by a series of linkages affordances [28,29], or mirrors the orientation of the other usersformed for teaching purposes [30]driven by actively by rotary actuators that can formdid various shapes. This interface can provide a variety ofabout physical rotating its joints. However, it also not address the issue of providing information physical affordances [28,29], or mirrors the orientation of the other users for teaching purposes [30] by actively properties or interaction with virtual environments. rotating its joints. However, it also did not address the issue of providing information about physical propertiesMethod or interaction with virtual environments. 3. Proposed Proposed Method 3.1. 3. Proposed Concept 3.1. Proposedsituations Concept often involve touching a rigid object with a rod, leading to a deflection of Real-life certain mechanical properties. rod touching deforms awith to thea user’s movement, according Real-life situations oftenThe involve rigidrespect object with rod, leading to a deflection of to its mechanical properties, such as stiffness and damping characteristics. These properties can be perceived certain mechanical properties. The rod deforms with respect to the user’s movement, according to its as not only inertial forces,such but also throughand the damping deflection behavior. Virtual systems mechanical properties, as stiffness characteristics. Thesereality properties can likewise be as dealing not only inertial butofalso through deflection behavior. Virtual realityIfsystems offerperceived scenarios with a forces, variety tools with the different mechanical properties. an interface likewise offer scenarios dealing with a variety of both tools graphically with different mechanical properties. If an system provides the deflection behavior of tools and physically, enhanced realism interface system provides the deflection behavior of tools both graphically and physically, enhanced and immersion may be achieved. Based on this hypothesis, this paper proposes a mobile interface realism and immersion may be achieved. Based on this hypothesis, this paper proposes a mobile system to physically and graphically express a variety of tool characteristics (Figure 1). The system is interface system to physically and graphically express a variety of tool characteristics (Figure 1). The composed of two parts; one is an algorithm that drives the interface system in a flexible manner, and system is composed of two parts; one is an algorithm that drives the interface system in a flexible the other is an articulated interface to visually represent this physical behavior. manner, and the other is an articulated interface to visually represent this physical behavior.

Figure designfor for the proposed method. Figure1.1. Conceptual Conceptual design the proposed method.

The proposed method visualizes the physical behavior of the flexible rod according to the user’s The proposed method visualizes the physical behavior of the flexible rod according to the user’s position input. Hence, the algorithm should consider the time taken to reflect the user’s motion and position input. Hence, the algorithm should consider the time taken to reflect the user’s motion and to display on the virtual reality system, which is called motion-to-photon latency [31]. To make the to display the virtualinreality system, which is the called motion-to-photon latency To make user feelon the presence the virtual environment, value of motion-to-photon latency[31]. should not the userexceed feel the in theit virtual the value of motion-to-photon latency should not 20 presence ms. Otherwise, induces environment, a poor virtual reality experience and may cause motion sickness. exceed ms. Otherwise, it induces virtual reality experience andand may cause motion sickness. The20 physical rod can be modeled as aa poor mass spring-based discretized model finite element-based model [32]. Although finitespring-based element-based model canmodel achieve precise it The continuum physical rod can be modeled as the a mass discretized and finite results, element-based requires a large computation time. Studies conducted in the past focused on developing a finite continuum model [32]. Although the finite element-based model can achieve precise results, it requires element analysis method to reduce the need for computation resources with increased precision [33– a large computation time. Studies conducted in the past focused on developing a finite element analysis 35]. However, it is still difficult to obtain a sufficiently low value of motion-to-photon latency. method to reduce the need for computation resources with increased precision [33–35]. However, Therefore, in this study, a physical rod is modeled to a mass spring-based discretized structure. Based it is still difficult to obtain a sufficiently low value of motion-to-photon latency. Therefore, in this on this structure, the proposed algorithm to generate the physical response is motivated by an active

study, a physical rod is modeled to a mass spring-based discretized structure. Based on this structure, the proposed algorithm to generate the physical response is motivated by an active contour model that is an energy-based spline control method. It is a pseudo-physical model, capable of emulating

Symmetry 2018, 10, 57

4 of 13

the physical behavior of the rod. It can also be extended to a variety of scenarios such as geometric information transfer [25] and affective interaction [36–38] by assigning additional energy functionals. Based on this idea, a tool is modeled as a serial connection of line segments with torsional springs and dampers, and an energy functional describing deflection behavior is defined in terms of a user’s movement and the angular displacement of neighboring joints. Thus, the deflection behavior is represented by the joint rotation of each line segment. In the same manner, the proposed interface system can have each serial linkage joint actively rotated using an actuator. Therefore, flexibility of a rod can be represented as the rotation speed and phase difference of neighboring joints. The proposed interface system operates as follows. The user grasps the bottom of the interface, and the position tracker follows the motion of the user. When displacement is detected, the trajectory of each joint is generated based on predefined mechanical characteristics, creating a joint trajectory for the proposed interface generated based on the user’s displacement. 3.2. Active Contour Model To control each element of the line segments, an active contour model (also known as the snake algorithm) [39] was employed as the baseline algorithm. The main purpose of a typical active contour model is to extract the contour of an object placed in an image. The model develops an energy-minimizing spline to determine the contour of the object. The total energy of the active contour model can be expressed as: ∗ Esnake =

Z

Eint (v(s)) + Eimage (v(s)) + Econ (v(s))ds

(1)

where v(s) is the position of each element and s represents space parameters (e.g., the pixel position in image coordinates). The energy includes internal energy (Eint ) and external energy, which derives from the image features (Eimage ) and the external constraint energy (Econ ). The internal energy imposes piecewise smoothness among neighboring snake points. This can be calculated as: Eint (v(s)) = (α(s)|vs (s)|2 + β(s)|vss (s)|2 )/2

(2)

where the subscript ‘s’ represents a derivative with respect to ‘s’, and α and β represent the weight of each derivative term. The first term represents the distance between neighboring elements, and the second term represents the curvature between elements. Depending on the weights, the spline acts like a membrane or thin plate. The external energy is responsible for pulling each element of the spline to the contour of the image. This energy was chosen since it relates to image contours such as lines and edges. The position of spline elements at the local minimum of the energy functional satisfy the Euler-Lagrange equation:     ∂ ∂2 ∂v ∂2 v α(s) − 2 β(s) 2 − ∇ Eext = 0 ∂s ∂s ∂s ∂s

(3)

Via iterative minimization, the snake elements gradually move towards the positions of the local minima of the energy functional. The minimization process is performed using an implicit Euler method as follows:     ∂2 v i ∂vi ∂ ∂vi ∂2 ∂2 v i µ i 2 + γi = α (s) − 2 β i (s) 2 − ∇ Eext (i ) (4) ∂t ∂s i ∂s ∂t ∂s ∂s where γi is the time step size and µi is an inertial coefficient added to impose dynamic effects during the process. Equation (4) can be solved by matrix inversions as described below

Symmetry 2018, 10, 57

5 of 13

vt+1 = B−1 4diag(u)vt + diag(gdt − 2u)vt−1 − 2dt2 ∇ Eext





where v = {v0 , . . . , v N −1 }, u = {µ0 , . . . , µ N −1 }, g = {γ0 , . . . , γ N −1 }   x0 · · · 0   .. .. diag(x) =  ...  . . 0 · · · x N −1 B = (2diag(u) + diag(g)dt)I − 2dt2 A  T A = "A0T , . . . , A TN −1 i −th element z }| { Ai = 0 · · · β i−1 (αi − 2β i−1 − 2β i ) (−αi+1 − αi + β i−1 + 4β i + β i+1 )

(5)

#

(αi+1 − 2β i − 2β i+1 )

β i +1

··· 0

As a result, the matrix A that determines shape of the spline becomes a pentagonal matrix. If the internal energy parameters to constant, the matrix is also bisymmetric. Due to Symmetry 2018, 10, x FORare PEERset REVIEW 5 of its 14 sparse and symmetric properties, the energy of each element is effectively minimized and the number of the spline ⋯ 0 ⋮ ⋱ ⋮ ( ) = elements is easily modified. 0 ⋯ (2

( )+

= 3.3. Proposed Method =

,…,

( ) ) −2





Our aim is to=visualize the behavior of a rod with various physical properties both virtually and i-th element physically as described in the previous section. Figure 2) is a −block diagram describing the proposed ( −2 ( ) − 2 ) (− − + +4 + 2 −2 ⋯ 0 0 ⋯ method. It is comprised of a position tracker that follows a user’s motion, and a computational As a result, the matrix that determines shape of the spline becomes a pentagonal matrix. If algorithm using an active model toconstant, represent the rod’s while the internal energycontour parameters are set to the matrix is alsobehavior bisymmetric. Due tofollowing its sparse the user’s and symmetric properties, the energy of each element is effectively minimized and the number of the movement. The proposed method includes both graphics and linkage hardware, a geometric constraint spline elements is easily modified. to maintain the distance between neighboring elements as constant. To satisfy this constraint, we 3.3.the Proposed Methodin other coordinates to decouple the distance from the effect of the energy parameterized contour functional. In this spline modeled inwith spherical coordinates, and formed Ourpaper, aim is tothe visualize theis behavior of a rod various physical properties bothis virtually andas a series of as described in the (Figure previous section. 2 is a block diagram the proposed line segmentsphysically with constant length 3). In Figure this configuration, thedescribing parameter value is defined as method. It is comprised of a position tracker that follows a user’s motion, and a computational the angular displacement of previous nodes. Each line segment is parameterized as: algorithm using an active contour model to represent the rod’s behavior while following the user’s movement. The proposed method includes both graphics and linkage hardware, h i T a geometric constraint to maintain thei distance between neighboring elements as constant. To satisfy this Pi (θi , φi , t) = Pi−1 + Ri−1 li cos φi sin θi li sin φi li cos φi cos θi , i=2 ∼ constraint, we parameterized the h contour in other coordinates to decouple the distance from i T the effect functional. In this paper, the spline is modeled in spherical coordinates, and is formed P1 (ofθithe , φenergy P + Rdev l1 cos φ1 sin θ1 l1 sin φ1 l1 cos φ1 cos θ1 i , t )of= as a series line dev segments with constant length (Figure 3). In this configuration, the parameter value i −1 i as ∏ the angular displacement of previous nodes. Each line segment is parameterized as: Ri−is1defined = Rdev k =1 rot x ( φk ) · roty ( θk ), li = constant ( , , )= cos sin sin cos cos + , =2~

N (6)

where i is the number node, parameters ( , , of cos and sin thesin cos θ cos ) =the spline + i and φi are the angles of(6)rotation about i . the x-axis and the y-axis of each(orientation ∏ )⋅ ( ) , Ri= = −1 Based onwhere this iconfiguration, internal energy the applied and changeareisthe expressed in Equation (7), as angles of rotation is the number of the spline node, and via the parameters about the x-axis and the y-axis of each orientation  " . # " # 2  in Equation (7), as Based on this configuration, internal energy via the applied change is expressed θi−1  θi Eint ( P(θi , φi )) (=, 21  β i1 − ) = − φi φi−1 2 (7)

β i = constant, N1, … , = i = 1,, . . . , = Articulated Line Cursor based on Active Contour Model (Parameters: Θ, Φ,

)

Mechanical Response of Rod Position Tracker

Position/Orientation Displacement

Energy Minimization Process

Internal Energy

External Energy

Figure 2. Block diagram of the proposed method.

Figure 2. Block diagram of the proposed method.

(7)

Symmetry 2018, 10, 57

6 of 13

Symmetry 2018, 10, x FOR PEER REVIEW

6 of 14

Figure 3. Parameterization thespline splinein in spherical spherical coordinates used in in thethe proposed method. Figure 3. Parameterization ofofthe coordinates used proposed method.

Since the proposed algorithm imposes the constant length constraint, the internal energy

Since the proposed algorithm imposes the constant length constraint, the internal energy (Equation (2)) derived from the first derivative becomes zero. Since the spline is parameterized by (Equation from thebecomes first derivative becomesThe zero. Since the spline angles,(2)) the derived second derivative a first derivative. external energy, which isis parameterized capable of by angles, the second derivative becomes a first derivative. The externalconnections energy, which is capable representing the physical behavior of a rod, is modelled as spring-damper between each of representing the physical of a rod, modelled spring-damper connections between each of the neighboring line behavior segments (Figure 4).isThese energyasfunctions were expressed as: ) + ( , )4).=These energy ( , functions , ) of the neighboring line segments (Figure were( expressed as: 1 1 ) =( P (θi ,( φ)i )) + E(damper ) ( P (θi , φi )) Erod ( P(θi , φi )) =( E, spring iT h 2 2 where  2  Espring ( P(θi , φi )) = 1 k si1 (θi ) 11 k si (φi )2 , = 1, … , 2 2 ( , ) = , i = 1, . . . , N where h 2 . iT 2   Edamper ( P(θi , φi )) = 1 c θ 2 1 c φ. 2

(8)

(8)

di i damping characteristics of each line i 2 dispring 2 and where ks and cd are coefficients that determine the segment. The external energy functional increases with the angular displacement from the previous where ks and cd areand coefficients that determine the Therefore, spring and damping characteristics of each line spline element, as its angular velocity increases. every spline element becomes aligned segment. The external energy functional increases with angular displacement from the previous with previous line segments having varying stiffness andthe damping properties. spline element, andaas its angular velocity Therefore, spline element becomes aligned Applying user’s positional input increases. to the proposed splineevery requires changing the linear and displacements by varying parameter of theand spline elementproperties. (Figure 5). These displacement with rotation previous line segments having the varying stiffness damping values affect parameter values input of firsttospline element which through the Applying a user’s positional the proposed splinepropagates requires changing the spline linear element and rotation during the energy minimization process with the rotation displacement is correspond to orientation displacements by varying the parameter of the spline element (Figure 5). These displacement values input. When a new orientation input occurs, the parametric variation is calculated based on angle affect parameter values of first spline element which propagates through the spline element during the between the line segment and axes of the orientation (Figure 5a). It can be calculated as:

energy minimization process with the rotation displacement is correspond to orientation input. When = ( − ) a new orientation input occurs, the parametric variation is calculated based on angle between the line . segment and axes of the orientation (Figure It can be calculated as: = 5a). − arctan .

t +1 t P1dev

t +1 R


T

(9)

P0 ) dev ( P1 − ! . t + 1 = − arccos P dθ1 = −arctan t+1t 1dev.x t P1dev.z ! where represents the first line segment vector in the input device coordinate of the t +1 t P1dev.y subsequent step. In the case of linear dφ1displacement, = −arccos the parametric variation is calculated by using 1 the total derivative relation between the Cartesian andLspherical coordinates (Equation (10)). As

=

(9)

described in Figure 5b from the linear displacement of the base element, the variation of parameter t+1 occurs. wherevalues t P1dev represents the first line segment vector in the input device coordinate of the subsequent As acase result, the spline with the proposed method operates in by Figure 6. the The total step. In the of linear displacement, the parametric variationasisillustrated calculated using translation and rotation of the base element caused by positional input induces an angular derivative relation between the Cartesian and spherical coordinates (Equation (10)). As described in displacement of the first element according to Equations (9) and (10). These angular displacements Figure 5b from the linear displacement of the base element, the variation of parameter values occurs. As a result, the spline with the proposed method operates as illustrated in Figure 6. The translation and rotation of the base element caused by positional input induces an angular displacement of the first element according to Equations (9) and (10). These angular displacements affect the change of

SymmetrySymmetry 2018, 10,2018, 57 10, x FOR PEER REVIEW Symmetry 2018, 10, x FOR PEER REVIEW

7 7ofof1413 7 of 14

affect the change of internal and external energies of entire line segments (Equations (6)–(9)). During internalenergy andSymmetry external ofand entire line segments (Equations (6)–(9)). During energy minimization, minimization, the spline operates according to segments the defined physical parameters, and 2018, energies 10, x FOR PEER REVIEW 7 of 14 affect the change of internal external energies of entire line (Equations (6)–(9)). During continues until the entire energy functional becomes zero. As a result, it reacts as a passively-moving the spline operates according to the defined physical parameters, and continues until the entire energy energy minimization, the spline operates according to the defined physical parameters, and affect the change of internal and external energies of entire line segments (Equations (6)–(9)). During object withuntil mechanical Note that a single position) is required properties. to control functional becomes zero. As aproperties. result,functional it reacts as aonly passively-moving object with mechanical continues the entire energy becomes zero. As ainput result,(base it reacts as a passively-moving energy minimization, the spline operates according to the defined physical parameters, and the rest of the joints. The entire operation of the proposed method is also available the object with mechanical properties. Note that only a single input (base position) required to control Note that only a single input (base position) required restis the joints. Theinentire continues until the entire energy functionalisbecomes zero.to Ascontrol a result, the it reacts asof a passively-moving the rest of the joints. The entire operation of the proposed method is also available in the Supplementary Materials. operation of the with proposed method is also available the Supplementary Materials. object mechanical properties. Note that onlyin a single input (base position) is required to control Supplementary the rest of Materials. the joints. The entiresinoperation the proposed cos method sin of cos sin  is also  available in the − sin " Materials. # " − # sin cos sin cos Supplementary dx − sin φL1 1sin θ1 cosL1φ1 −− cos θ1L1sin φ1  =− (10) dθ1  cos sin sin cos sin cos sin == (10) dy   (10) sin θ cos θ − − 1 1 0 − cos dφ1 0 − Lsin L1 coscos φ1 cos φcos 1 1 0 − dz = (10) cos cos sin cos 0 − cos cos

Figure 4. Structural diagram of the external energy functional; spring and damping constraints are

Figure 4. Structural diagram energyfunctional; functional; spring damping constraints are Figure 4. Structural diagramofofthe the external external energy spring andand damping constraints are Figure 4. Structural applied. diagram of the external energy functional; spring and damping constraints are applied. applied. applied.

Figure 5. Displacement of links due to rotation (a)(a) and translation (b)(b) ofofthe element; the Figure 5. Displacement of links to rotation (a) and translation (b) of the base element; the green Figure 5. Displacement ofdue links due to rotation and translation thebase base element; thegreen green solid line line indicates the the displacement thethe base element, and blue dashed lines the solid indicates displacement of base element, andthe the blue dashed linesindicate indicate the solid line indicates the displacement ofofthe base element, and the blue dashed lines indicate the Figure 5. Displacement ofcaused links due to base rotation (a) and translation (b) of the base element; the green movement the elements the displacement. movement of the elements caused by the base displacement. movement of the of elements caused byby the base displacement.

solid line indicates the displacement of the base element, and the blue dashed lines indicate the movement of the elements caused by the base displacement.

Figure 6. Response of spline with proposed model due user’sposition positioninput. input.The Theblue blueline line Figure 6. Response of spline with proposed model due to to thetheuser’s

Figure 6. Response of spline with proposed model due line to the user’s input. blue line indicates elements of the spline, and the green dashed indicates its position position where the The internal indicates elements of the spline, and the green dashed line indicates its position where the internal indicates elements of the spline, and the green dashed line indicates its position where the internal and external becomes zero. with (a) Initial state;model (b) Displacement occurred by user’s Figureenergy 6. Response of spline proposed due to the user’s position input. position The blueinput line (Equations (9) and (10)) of induces external energy node line 1 and 2 (Equation (8)); where (c,d) Internal and indicates elements the spline, and the greenon dashed indicates its position the internal external energy are propagated to succeeding nodes. (Equations (7) and (8)); (e,f) During the energy minimizing process, entire nodes move until internal and external energy becomes zero. (Equation (5)); (h) Final state.

Symmetry 2018, 10, 57

and external energy becomes zero. (a) Initial state; (b) characteristics with (Equations (9) and (10)) inducesstiffness externaland energy on dampin Symmetry 2018, 10, x FOR PEER REVIEW 4. Analysis external energy are propagatedenergy to succeeding nodes. coefficient an process, entire nodes move untilthis internal mentioned above, pap constructed, compri and external energyminimizing becomes zero. (a)As Initial state; (b) Displacement occu (5)); (h) Final state. characteristics with the joint angles (Equations (9) and (10)) induces external energy on 1 andelement 2 (Equo Asnode the base 8 of 13

and nodes. damping energy coeff external energy are propagated stiffness to succeeding (Equations and [40], which is(7) capab Analysis energy coefficient and positional inp minimizing process, entire nodes move until internal and external ener be applied to real-tim comprised of (5)); (h) Final state. trajectories on node the mentioned above, this paper proposes a c and external energy becomes zero. (a)As Initial state; constructed, (b) Displacement occurred byfour user’s posi

Symmetry 2018, 10, x FOR PEER REVIEW 4.

4. Analysis

As node the joint base element the spline w recorded trajectory characteristics with the angles of of multiple line In sew (Equations (9) and (10)) induces external energy on 1 and 2 (Equation (8)); (c,d) 4. Analysis As mentioned above, this paper proposes a computational algorithm to represent flexible [40], which is capable of tracking 3D At first, we exat and nodes. damping energy(7) coefficients. inves external energy are propagatedstiffness to succeeding (Equations and (8)); (e,f)To During characteristics with the joint angles of multiple line segments and can becoefficient achieved by modulating beand applied to real-time applications, photon latency. The minimizing process, nodes move until internal and external energy becomes zero. energy positional input on the node re As entire mentioned above, this paper proposes a computational al trajectories on the coefficients to un the stiffness and damping energy coefficients. To characteristics investigate itswith feasibility, the effect of changing (5)); (h) Final state. position input to the constructed, comprised of four nodes, with each node the joint angles of multiple line segments and can b recorded trajectory was ofits an arbitra the energy coefficient and positional input on the node responses was observed. Initially, the spline next time step. As the base element of the spline was moved byIta is u stiffness and damping energy coefficients. To investigate feasibili At first, we examined the compu Analysis was constructed, comprised of four4.nodes, with each node modelled as 30 mm long with a mass of spline element. The [40], which is capable of tracking 3D positions and or energy coefficient and positional input on the node responses was obs photon The computation element is represent be applied tocaptured real-time applications, we used ato record 0.2 kg. As the base element of the spline As wasmentioned moved byabove, a user,this its position bylatency. an HTC constructed, comprised ofwas four nodes, with each node modelled asrepr 30tim m paper proposes a computational algorithm position input to the time taken to ca number of the splin trajectories on the coefficients to understand the effe Vive [40], which is capable of tracking 3D positions and orientations. Although the proposed method As the base element of the spline was moved by a user, its position w characteristics with joint angles of multiple line segments and can be achieved by m next time step. Itoforientations. isspline measured unde elements recorded trajectory was an arbitrary circular motio can be applied to real-time applications, weand used a recorded input to compare theofeffect of [40], which isuser’s capable of tracking 3D positions and Altho stiffness damping energy coefficients. To investigate its feasibility, the effect ofis spline element. The average comput computational reso At first, we examined the computation time for tt the joint trajectories on the coefficients to understand the effect energy only. beand applied toofreal-time applications, we used awas recorded user’s input to energy coefficient positional input coefficient on the nodemodulation responses observed. Initially, element is represented in Table 1. Th without interfering photon latency. The computation time is measured fr trajectories on space, the coefficients tonode understand energy The recorded trajectory was of an arbitrary circular motion of infour 3D and at 100the Hz. constructed, comprised nodes, withwas eachsampled modelled as effect 30 mmoflong withcoe am number oftaken themotion spline element only, position input to the time to calculate the subse recorded trajectory was of an arbitrary circular in 3D space, ana As the base element of the was moved by to a user, its position At first, we examined the computation time for the spline proposed method obtain a low was captured by of spline elements is 50. Therefore, it Table 1. Average next timetime step. It orientations. isthe measured under conditi At wefrom examined the computation timeAlthough for thevarious proposed meth [40], which is capable offirst, tracking 3D positions and the propose motion-to-photon latency. The computation time is measured the that algorithm computational resources and reliab according to var spline element. The average computation time accord photonapplications, latency. The computation time isuser’s measured the time that applied toto real-time we used a recorded input to compare the eff receives a new position input to thebe time taken calculate the subsequent parameter values of the from processor for thet without interfering low motion-to-ph element is represented in Table 1. The computation position input to the time taken to calculate the subsequent parameter trajectories on the coefficients understand the effect of energy coefficient modulat entire spline for next time step. It is measured under various condition such as type of trajectories and of the motion spline various element only,and andwas the sampled maximum next time Itnumber is input measured under condition such as type a recorded trajectory was of step. an arbitrary circular in 3Dnumber space, number of spline element. The average computation time according to trajectories and Table 1. Average computation time of spline elements is 50. Therefore, it can be shown spline element. The average computation time according to input first,computation we examinedtime the computation time method for the proposed method obtaintraj ath lo of spline element is represented in Table At 1. The of the proposed is affected according to variation of number Number of computational resources and reliably reflects the us element is represented in Table 1. The computation time of the propo photon latency. The computation time is measured from the time that the algorithm r by the number of the spline element only, and the maximum computation time is 2.5low ms when for thethe computation. processor Element without interfering motion-to-photon latency. the spline element only, and the maximum computation position input tonumber the timeoftaken to calculate the subsequent parameter Spline values of the tim en number of spline elements is 50. Therefore, it can be shown that the proposed method does not require Circular ofisspline elements is 50. Therefore, it cansuch be shown that proposedan next time step. It measured under various condition as type ofthe trajectories large computational resources and reliably reflects the user’s input to virtual the environment system Table 1. Average computation and(239 standard Points) computational resources and reflects thetime user’s input todevi vir spline element. The average computation time reliably according to input trajectories and num Number of of spline 4elements without interfering low motion-to-photon latency. according to variation of number without interfering low motion-to-photon latency. element is represented in Table 1. The computation time of the proposed method is a 5 Spline Element processor for the computation. number of the spline element only, and the maximum computation time is 2.56ms whe Table 1. Average computation time and standard deviation of the proposed method in each time Table 1. Average time andthe standard deviation of the propos of spline elements is 50. Therefore, it computation can be shown that proposed method does noR Circular Linear step according to variation of number of spline elementsaccording and 3D trajectory. Weof used an Intel i5-3570elements and 3D 7trajectory to variation number of spline computational resources and reliably reflects the user’s(239 input to virtual the environ Points) (227 Points) 4 194 ±843 μs ( processor for the computation. Number of processor for the computation. without interfering low motion-to-photon latency. 5 224 ±935 μs Spline Element 6 251 10 ± 24 μs Circular Linear Rectangular Circular Linear Rectangular Shaking Star Table 1. Average(309 computation and standard deviation of the each 7 proposed 286 ± 21inμs (239 Points) (227 Points) Points) time(629 Points) (291 (239 Points) (227Points) Points) (309method Points) (6 20 Number of according to variation of number of spline elements and 3D 8trajectory. We used an Int 326 ± 32 μs 30μs 4 194 ± 43 μs 201 ± 46 Number of Spline processor for the computation. Spline Element ± 32 Element 40 5 224 ±935 μs 235356 ± 45 μsμs ± 24 50 6 251 ±1024 μs Shaking 260389 ± 38 μsμs Circular Linear Rectangular 20 769 ± 34 ± 21 298 (239 Points) (227 Points) Points) 4 194Points) ±743 μs (309 201286 ± 46 μsμs (629 194 ± ±3137μsμsμs(29 1 Number of 30 1257 ±μs 108 The modulatio 835 μs 193 326 ±µs32 332± ±46 51 μs μs2 4 194 ± 43 µs 201 ± 46 µs 194 ± 31 5 µs 199 224 ± 47±µs 235± ±3945 μsμs 229 Spline Element229 ± 46 µs 1823 ±μs 102 μs2 5 224 ± 35 µs 235 ± 45 µs 232 251 ± 41±µs µs spring constants 924 μs 232 356 ±40 32 366 μs wit 6 260± ±4938 μsμs 264 ± ±4243 6 251 ± 24 µs 260 ± 38 µs 264 ± 42 µs 262 ± 4210 µs 261 ± 39 µs 2463 ±μs 121 μs3 spring constant. 389 ±5024 396 μs The 7 286 ± 21 μs 297 298± ±3937 μsμs 307 ± ±4736 7 286 ± 21 µs 298 ± 37 µs 307 ± 47 µs 306 ± 50 µs µs Figure 7. These figu 20 769 ± 34 μs 789 ± 62 μs 326 ±µs 32 332±±±56 51µs μs 199 332±±47 47μs μs 19 3 4 194 201 μsμs 338 194 31 μs 8 326 ± 32 µs 332 ± 51 µs 332 ± 47 ±8 µs43 μs 329 ±± 3946 The modulation of these coeffi1 1257 ±µs 108 1248 ± μs 65 9 356 ± 32 µs 366 ± 43 µs 369 ± 39 ±9 µs35 μs 367 ±± 4045 µs ±±±44 356 ±30 32 366 43 μs μs232 369 39 μsμs 23 3 5 224 235 μsμs 366 229 46 μs ±±41 spring constants with a fixed dampin 10 389 ± 24 µs 396 ± 36 µs 403 ± 46 µs 403 ± 54 µs 398 ± 44 µs 40 1823 ± 102 μs 1839 ± 102 μs 10 389 ± 24 μs 396 ± 36 μs 403 ± 46 μs 41 6 251 ± 24 μs 260 ± 38 μs 264 ± 42 μs 262 ± 42 μs 26 constant. The responses of the 20 769 ± 34 µs 789 ± 62 µs 787 ± 62 µs 789 ± 6050 µs spring 7852463 ± 59±µs 121 μs 2493 ± 131 μs 2021 μs 769 ± 34 μs 307 789±±47 62 μs 306 787 ±50 62μs μs 29 72 298 30 1257 ± 108 µs 1248 ± 65 7 µs 1258 286 ± 72±µs 1255 ±±6937µsμs Figure 1258 7. ± These 63 μs µs figures±show the tra 30 1257 ± 108 1248 1258± ±3972μsμs 33 12 326 ± µs 32 μs 1846 332 51 μs μs 1829 332 ± ±47 40 1823 ± 102 µs 1839 ± 1028µs 1834 ± 116 ±± 121 µs ± 9465μs µsμs 329 The modulation of±±these coefficients was 40 1823 ± 102 1839 ±140 102 1834 116 18 50 2463 ± 121 µs 2493 ± 1319µs 2495 ± 117 ±± 130 µs µsμs 356 ± µs 32 μs 2496 366 43 μs μs 2493 369 39 μs 367 ±±40 μsμsperfo 36 spring constants with a fixed damping by 2463 ± 121 2493 131 μs 2495 117 10 389 50 ± 24 μs 396 ± 36 μs μs 403 ±±46 μs 403constant, ±±54 μsμsor 24 39 spring constant. The responses of the spline while fol 20 34 μs either 789by ± 62 μs 787 the ± 62spring μs 789 ± 60 μs 78 The modulation of these coefficients was performed in 769 two±ways, modulating These figures the trajectories of the125 ba The modulation of7.these coefficients performed inμs two wa 30 1257 ± 108 Figure μs 1248 ± 65 μs 1258show ± was 72 μs 1255 ± 69 constants with a fixed damping constant, or by modulating the damping constant with a fixed spring constants a fixed damping constant, modulating 40 spring1823 ± 102 with μs 1839 ± 102 μs 1834 ± 116 μsor by 1846 ± 121 μs the 182 constant. The responses of the spline while following the recorded trajectory are displayed in Figure 7. constant. responses spline while 50 spring2463 ± 121The μs 2493 ± 131 of μsthe2495 ± 117 μs following 2496 ± 130the μsrecord 249 These figures show the trajectories of the base element (blue7.solid line), parametric values of the of spline Figure These figures show the trajectories the base element (blue elements as defined in spherical coordinates in (a), the trajectories of each spline element in Cartesian The modulation of these coefficients was performed in two ways, either by m coordinates in (b), and in 3D space inspring (c), respectively. Forathe soft rod, with a smallorspring coefficient, constants with fixed damping constant, by modulating the damping constan a comparatively large angular displacement and phase delay were observed via the amplitudes spring constant. The responses of the spline while following of theeach recorded trajectory ar node compared to the base node. Therefore, the entire spline had large deflection, growing larger Figure 7. These figures show theatrajectories of the base element (blue solid line), para when nearing the rear of the nodes. For the stiff rod, however, the response was the opposite. Since

of the spline elements as defined in spherical coordinates in (a), the trajectories of each spline element in Cartesian coordinates in (b), and in 3D space in (c), respectively. For the soft rod, with a small spring coefficient, a comparatively large angular displacement and phase delay were observed via the amplitudes of each node compared to the base node. Therefore, the entire spline had a large Symmetry 2018, 10, 57 9 of 13 deflection, growing larger when nearing the rear of the nodes. For the stiff rod, however, the response was the opposite. Since it rapidly adapted to the angular displacement of previous elements, it it rapidly the angular displacement of previous it showed a relatively smaller showed a adapted relativelytosmaller angular displacement and phaseelements, delay. Figure 8 shows the response of angular displacement and phase delay. Figure 8 shows the response of the joints based on damping the joints based on damping coefficient modulation. The modulation of damping did not affect the coefficient modulation.orThe modulation damping not affect the angular displacement or phase angular displacement phase delay, butofthe smallerdid damping coefficients caused ripples during the delay, but the smaller damping coefficients caused ripples during the process. In contrast, for high process. In contrast, for high values of the damping coefficient, the joint angles converged to their values of the without dampingripples. coefficient, the joint angles converged to their rest position without ripples. rest position In this this section, proposed method to create various physical rod In section, we weanalyzed analyzedthe thepossibility possibilityofofthe the proposed method to create various physical characteristics. To verify this, an analysis of the response generated by energy coefficient modulation rod characteristics. To verify this, an analysis of the response generated by energy coefficient was performed. This showed the modulation of the spring effects the angular modulation was performed. Thisthat showed that the modulation of thecoefficient spring coefficient effects the displacement and phase delay among neighboring nodes, while that of the damping coefficient effects angular displacement and phase delay among neighboring nodes, while that of the damping the ripples effects duringthe the ripples response. Therefore, a varietyTherefore, of responses can be obtained by modulating these coefficient during the response. a variety of responses can be obtained energy coefficients. by modulating these energy coefficients.

Figure 7. Joint trajectories of a spline with spring coefficient modulation, with the damping coefficient Figure 7. Joint trajectories of a spline with spring coefficient modulation, with the damping coefficient set low (top) and high (bottom). The trajectories are represented (a,d) as parametric values; (b,e) in set low (top) and high (bottom). The trajectories are represented (a,d) as parametric values; (b,e) in Cartesian coordinates; and (c,f) in 3D space. Cartesian coordinates; and (c,f) in 3D space.

Symmetry 2018, 10, 57

10 of 13

Symmetry 2018, 10, x FOR PEER REVIEW

10 of 14

Figure 8. Joint trajectories of a spline with damping coefficient modulation, with the damping Figure 8. Joint trajectories of a spline with damping coefficient modulation, with the damping coefficient set low (top) and high (bottom). The trajectories are represented (a,d) as parametric values; coefficient set low (top) and high (bottom). The trajectories are represented (a,d) as parametric values; (b,e) in Cartesian coordinates; and (c,f) in 3D space. (b,e) in Cartesian coordinates; and (c,f) in 3D space.

5. Application 5. Application Thus far, we described a novel method to imitate the behavior of a deformable rod with certain Thus far, we described a novel method to imitate the behavior of a deformable rod with certain mechanical properties based on an active contour model. The proposed method can be applied to mechanical properties based on an active contour model. The proposed method can be applied various interactive areas, including immersive 3D interaction, especially in augmented reality to various interactive areas, including immersive 3D interaction, especially in augmented reality scenarios. Since the proposed interface can change its mechanical properties dynamically, it can be scenarios. Since the proposed interface can change its mechanical properties dynamically, it can be used by a variety of physical tools. This scenario can be applied to games, such as sword fighting, used by a variety of physical tools. This scenario can be applied to games, such as sword fighting, ribbon gymnastics, and other scenarios which deform its physical shape by directly reflecting the ribbon gymnastics, and other scenarios which deform its physical shape by directly reflecting the user’s position input. Furthermore, by representing the physical behavior in response to the other user’s position input. Furthermore, by representing the physical behavior in response to the other user’s position input, the proposed method can be applied to training scenarios conveying dynamic user’s position input, the proposed method can be applied to training scenarios conveying dynamic characteristics of the rod. The proposed method can also be extended to various interaction scenarios characteristics of the rod. The proposed method can also be extended to various interaction scenarios by defining additional energy functions according to a variety of task objects. For example, by by defining additional energy functions according to a variety of task objects. For example, by defining defining an additional energy function to provide geometric information of a virtual object that a user an additional energy function to provide geometric information of a virtual object that a user contacts, contacts, the interface can lead the users to think that they are touching a virtual object with a the interface can lead the users to think that they are touching a virtual object with a deformable deformable tool from the dynamic movement of each link. It can enhance the presence and tool from the dynamic movement of each link. It can enhance the presence and performance of performance of 3D manipulation tasks in augmented reality by providing enhanced spatial 3D manipulation tasks in augmented reality by providing enhanced spatial information of virtual information of virtual objects [23]. objects [23]. The proposed approach could also be applied to control graphical objects. The proposed model The proposed approach could also be applied to control graphical objects. The proposed model can generate trajectories of all spline elements while only measuring base positions. Hence, the can generate trajectories of all spline elements while only measuring base positions. Hence, the method can be applied for the intuitive control of a graphic element that is composed of a series of method can be applied for the intuitive control of a graphic element that is composed of a series of line segments. For example, it could enhance the process of animation authoring by offering intuitive line segments. For example, it could enhance the process of animation authoring by offering intuitive character control using the animator’s gestures. Note that the trajectory generation of 3D objects that character control using the animator’s gestures. Note that the trajectory generation of 3D objects that are composed of multiple joints requires a great deal of time and cognitive load on the part of the animator [41]. In fact, a previous effort by Shiratori et al. developed an efficient trajectory generation method based on an animator’s 3D gestures to resolve such issues [42]. Placed in the same context,

Symmetry 2018, 10, 57

11 of 13

are composed of multiple joints requires a great deal of time and cognitive load on the part of the animator [41]. In fact, a previous effort by Shiratori et al. developed an efficient trajectory generation method based on an animator’s 3D gestures to resolve such issues [42]. Placed in the same context, the method proposed herein can contribute to reducing the time and cost of animating tasks in that the phase and amplitude of each joint is determined only by the base position measurement, implying that an animator needs only to move a 3D positioning device around to simultaneously control the position of multiple joints, regardless of the number of joints. 6. Conclusions In this paper, we proposed a novel shape-changing interface system to enhance the immersive experience in virtual reality environments. This interface is formed with a serially-linked interface, and its control algorithm is a modified active-contour model. It enables the representation of a tool’s mechanical properties that users handle in virtual and physical environments. By setting the external energy to the spring and damper constraints of each spline element, the proposed method can reflect the physical properties of a rod. One of the main advantages of our approach is that the model can generate joint trajectories of a serially-linked system that consist of multiple joints, based only on the position measurement of the base node, which is normally held by the user’s hand. It can also simulate a variety of flexible objects, such as a fencing sword or a whip, depending on the spring-damper parameters of the proposed algorithm. The proposed method can be applied to not only 3D interaction scenarios, but also to graphic element control, such as during the animation of 3D characters. In addition, it can be applied to other interaction scenarios by defining additional external energy functionals with respect to a user’s intention. To validate the possibility of various response generation, we analyze the behavior of the spline element by modulating the external energy coefficient according to a user’s arbitrary position input. This showed that the proposed method can provide various responses depending on the external energy coefficients. In the future, we plan to systematically relate the parameter space to the user’s perception by conducting user tests. Supplementary Materials: The following are available online at http://www.mdpi.com/2073-8994/10/3/57/s1, demonstrative video of the proposed method. Acknowledgments: This research project was supported by the Sports Promotion Fund of Seoul Olympic Sports Promotion Foundation from Ministry of Culture, Sports and Tourism. Author Contributions: Seung-Chan Kim, and Dong-Soo Kwon conceived and designed the analysis; Byung-Kil Han performed the analysis; and Byung-Kil Han and Seung-Chan Kim wrote the paper. Conflicts of Interest: The authors declare no conflict of interest.

References 1. 2. 3. 4.

5. 6.

Hoffman, D.M.; Girshick, A.R.; Akeley, K.; Banks, M.S. Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. J. Vis. 2008, 8, 33. [CrossRef] [PubMed] Robles-De-La-Torre, G. The importance of the sense of touch in virtual and real environments. IEEE Multimed. 2006, 13, 24–30. [CrossRef] Laycock, S.; Day, A. The haptic rendering of polygonal models involving deformable tools. In Proceedings of the EUROHAPTICS, Dublin, Ireland, 6–9 July 2003. Frohlich, B.; Tramberend, H.; Beers, A.; Agrawala, M.; Baraff, D. Physically-based manipulation on the responsive workbench. In Proceedings of the 2000 IEEE Virtual Reality, New Brunswick, NJ, USA, 22 March 2000; IEEE: New York, NY, USA, 2000; pp. 5–11. Salisbury, K.; Conti, F.; Barbagli, F. Haptic rendering: Introductory concepts. IEEE Comput. Graph. Appl. 2004, 24, 24–32. [CrossRef] [PubMed] Salisbury, K.; Brock, D.; Massie, T.; Swarup, N.; Zilles, C. Haptic rendering: Programming touch interaction with virtual objects. In Proceedings of the 1995 Symposium on Interactive 3D Graphics, Monterey, CA, USA, 9–12 April 1995; ACM: New York, NY, USA, 1995; pp. 123–130.

Symmetry 2018, 10, 57

7. 8.

9. 10.

11. 12. 13.

14. 15.

16. 17.

18.

19.

20.

21.

22.

23.

24.

25. 26.

12 of 13

Prattichizzo, D.; Pacchierotti, C.; Rosati, G. Cutaneous force feedback as a sensory subtraction technique in haptics. IEEE Trans. Haptics 2012, 5, 289–300. [CrossRef] [PubMed] Poupyrev, I.; Nashida, T.; Okabe, M. Actuation and tangible user interfaces: The vaucanson duck, robots, and shape displays. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, Baton Rouge, LA, USA, 15–17 February 2007; ACM: New York, NY, USA, 2007; pp. 205–212. Sato, M.; Hirata, Y.; Kawarada, H. Space interface device for artificial reality—spidar. Syst. Comput. Jpn. 1992, 23, 44–54. [CrossRef] Massie, T.H.; Salisbury, J.K. The phantom haptic interface: A device for probing virtual objects. In Proceedings of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Chicago, IL, USA, 20 November 1994; pp. 295–300. Novint Technologies. Available online: http://www.novint.com/index.php/novintfalcon (accessed on 30 November 2017). Ho, C.-H.; Basdogan, C.; Srinivasan, M.A. Haptic rendering: Point-and ray-based interactions. In Proceedings of the Second PHANToM Users Group Workshop, Dedham, MA, USA, 19–22 October 1997. Zilles, C.B.; Salisbury, J.K. A constraint-based god-object method for haptic display. In Proceedings of the 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems 95 ‘Human Robot Interaction and Cooperative Robots’, Pittsburgh, PA, USA, 5–9 August 1995; IEEE: New York, NY, USA, 1995; pp. 146–151. Laycock, S.D.; Day, A. A survey of haptic rendering techniques. In Computer Graphics Forum; Wiley Online Library: Hoboken, NJ, USA, 2007; pp. 50–65. Rosenberg, L.B. Virtual fixtures: Perceptual tools for telerobotic manipulation. In Proceedings of the 1993 IEEE Virtual Reality Annual International Symposium, Seattle, WA, USA, 18–22 September 1993; IEEE: New York, NY, USA, 1993; pp. 76–82. Abbott, J.; Marayong, P.; Okamura, A. Haptic virtual fixtures for robot-assisted manipulation. In Robotics Research; Springer: Berlin/Heidelberg, Germany, 2007; pp. 49–64. Kim, H.; Kim, M.; Lee, W. Hapthimble: A wearable haptic device towards usable virtual touch screen. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, 7–12 May 2016; ACM: New York, NY, USA, 2016; pp. 3694–3705. Minamizawa, K.; Fukamachi, S.; Kajimoto, H.; Kawakami, N.; Tachi, S. Gravity grabber: Wearable haptic display to present virtual mass sensation. In Proceedings of the ACM SIGGRAPH 2007 Emerging Technologies, San Diego, CA, USA, 5–9 August 2007; ACM: New York, NY, USA, 2007; p. 8. Benko, H.; Holz, C.; Sinclair, M.; Ofek, E. Normaltouch and texturetouch: High-fidelity 3d haptic shape rendering on handheld virtual reality controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, 16–19 October 2016; ACM: New York, NY, USA, 2016; pp. 717–728. Winfree, K.N.; Romano, J.M.; Gewirtz, J.; Kuchenbecker, K.J. Control of a high fidelity ungrounded torque feedback device: The itorqu 2.1. In Proceedings of the 2010 IEEE International Conference on Robotics and Automation (ICRA), Anchorage, AK, USA, 3–7 May 2010; IEEE: New York, NY, USA, 2010; pp. 1347–1352. Badshah, A.; Gupta, S.; Morris, D.; Patel, S.; Tan, D. Gyrotab: A handheld device that provides reactive torque feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, USA, 5–10 May 2012; ACM: New York, NY, USA, 2012; pp. 3153–3156. Rasmussen, M.K.; Pedersen, E.W.; Petersen, M.G.; Hornbæk, K. Shape-changing interfaces: A review of the design space and open research questions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, USA, 5–10 May 2012; ACM: New York, NY, USA, 2012; pp. 735–744. Iwata, H.; Yano, H.; Nakaizumi, F.; Kawamura, R. Project feelex: Adding haptic surface to graphics. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA, 12–17 August 2001; ACM: New York, NY, USA, 2001; pp. 469–476. Follmer, S.; Leithinger, D.; Olwal, A.; Hogge, A.; Ishii, H. Inform: Dynamic physical affordances and constraints through shape and object actuation. In Proceedings of the UIST, St Andrews, UK, 8–11 October 2013; pp. 417–426. Bordegoni, M.; Ferrise, F.; Covarrubias, M.; Antolini, M. Geodesic spline interface for haptic curve rendering. IEEE Trans. Haptics 2011, 4, 111–121. [CrossRef] [PubMed] Klare, S.; Peer, A. The formable object: A 24-degree-of-freedom shape-rendering interface. IEEE/ASME Trans. Mechatron. 2015, 20, 1360–1371. [CrossRef]

Symmetry 2018, 10, 57

27.

28.

29.

30.

31. 32. 33. 34. 35. 36.

37.

38.

39. 40. 41. 42.

13 of 13

Katsumoto, Y.; Tokuhisa, S.; Inakage, M. Ninja track: Design of electronic toy variable in shape and flexibility. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, Barcelona, Spain, 10–13 February 2013; ACM: New York, NY, USA, 2013; pp. 17–24. Nakagaki, K.; Dementyev, A.; Follmer, S.; Paradiso, J.A.; Ishii, H. Chainform: A linear integrated modular hardware system for shape changing interfaces. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, 16–19 October 2016; ACM: New York, NY, USA, 2016; pp. 87–96. Nakagaki, K.; Follmer, S.; Ishii, H. Lineform: Actuated curve interfaces for display, interaction, and constraint. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, NC, USA, 11–15 November 2015; ACM: New York, NY, USA, 2015; pp. 333–339. Nakagaki, K.; Inamura, C.; Totaro, P.; Shihipar, T.; Akikyama, C.; Shuang, Y.; Ishii, H. Linked-stick: Conveying a physical experience using a shape-shifting stick. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, Seoul, Korea, 18–23 April 2015; ACM: New York, NY, USA, 2015; pp. 1609–1614. Abrash, M. What VR Could, Should, and Almost Certainly Will Be within Two Years; Steam Dev Days: Seattle, WA, USA, 2014. Gibson, S.F.; Mirtich, B. A Survey of Deformable Modeling in Computer Graphics; Technical Report; Mitsubishi Electric Research Laboratories: Cambridge, MA, USA, 1997. Guessasma, S.; Hamdi, A.; Lourdin, D. Linear modelling of biopolymer systems and related mechanical properties. Carbohydr. Polym. 2009, 76, 381–388. [CrossRef] Huang, J.; Ong, S.-K.; Nee, A.Y. Real-time finite element structural analysis in augmented reality. Adv. Eng. Softw. 2015, 87, 43–56. [CrossRef] Guessasma, S.; Benseddiq, N.; Lourdin, D. Effective young’s modulus of biopolymer composites with imperfect interface. Int. J. Solids Struct. 2010, 47, 2436–2444. [CrossRef] Park, Y.-W.; Park, J.; Nam, T.-J. The trial of bendi in a coffeehouse: Use of a shape-changing device for a tactile-visual phone conversation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Korea, 18–23 April 2015; ACM: New York, NY, USA, 2015; pp. 2181–2190. Togler, J.; Hemmert, F.; Wettach, R. Living interfaces: The thrifty faucet. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction, Cambridge, UK, 16–18 February 2009; ACM: New York, NY, USA, 2009; pp. 43–44. Park, J.; Park, Y.-W.; Nam, T.-J. Wrigglo: Shape-changing peripheral for interpersonal mobile communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, ON, Canada, 26 April–1 May 2014; ACM: New York, NY, USA, 2014; pp. 3973–3976. Kass, M.; Witkin, A.; Terzopoulos, D. Snakes: Active contour models. Int. J. Comput. Vision 1988, 1, 321–331. [CrossRef] HTC. Htc Vive. Available online: https://www.vive.com (accessed on 5 April 2016). Jacobson, A.; Panozzo, D.; Glauser, O.; Pradalier, C.; Hilliges, O.; Sorkine-Hornung, O. Tangible and modular input device for character articulation. ACM Trans. Graph. TOG 2014, 33, 82. Shiratori, T.; Mahler, M.; Trezevant, W.; Hodgins, J.K. Expressing animated performances through puppeteering. In Proceedings of the 2013 IEEE Symposium on 3D User Interfaces (3DUI), Orlando, FL, USA, 16–17 March 2013; IEEE: New York, NY, USA, 2013; pp. 59–66. © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).