Closed-Loop Control in an Autonomous Bio-hybrid Robot System Based on Binocular Neuronal Input Jiaqi V. Huang and Holger G. Krapp() Department of Bioengineering, Imperial College London, London SW7 2AZ, UK
[email protected]
Abstract. In this paper, we describe the implementation of a closed-loop control architecture on a bio-hybrid robotic system. The control loop uses the spiking activity from two motion-sensitive H1-cells recorded in both halves of the blowfly’s brain as visual feedback signals that are sent to an ARM processor, programmed to establish a brain machine interface. The resulting output controls the movements of the robot which, in turn, generates optic flow that modifies the activity of the H1-cells. Instead of being inhibited by front-to-back optic flow would the robot move forward in a straight line, the closed-loop system autonomously produces an oscillatory trajectory, alternatingly stimulating both H1-cells with back-to-front optic flow. The spike rate information of each cell is then used to control the speed of each robot wheel, on average driving the robot in the forward direction. Our extracellular recordings from the two cells show similar spike rate oscillation frequencies and amplitude, but opposite phases. From our experiments we derive parameters relevant for the future implementation of collision avoidance capabilities. Finally, we discuss a control algorithm that combines positive and negative feedback to drive the robot. Keywords: Motion vision · Brain machine interface · Blowfly · Closed-loop control · Autonomous
1
Introduction
The blowfly’s ability to perform agile flight manoeuvres [1], using robust reflexes, such as gaze control [2] and collision avoidance [3], make it a suitable model system for opto-motor research. We use a bio-hybrid system to investigate the H1-cells of the blowfly, Calliphora vicina, in a state of locomotion stimulating non-visual sensory systems (e.g. the halteres), in order to explore visual processing and multisensory integration in a closedloop condition, which can be created by a robot. The research expands on a previous study of visual stabilization of a robot using H1-cell activity in closed-loop [4] and extracellular neural recordings on a mobile platform [5]. A similar implementation of insect sensory control of a wheeled robot was performed using mainly the olfactory system of the silk-moth [6]. Other relevant closed loop control projects are also of importance, such as collision avoidance based on a well characterized cell in locust [7]. Previous work in fruit flies has shown that locomotor state affects the processing of motion vision in individual neurons [8], so we expect to find a difference in response upon movement of the platform. Integration of visual and haltere input has © Springer International Publishing Switzerland 2015 S.P. Wilson et al. (Eds.): Living Machines 2015, LNAI 9222, pp. 164–174, 2015. DOI: 10.1007/978-3-319-22979-9_17
Closed-Loop Control in an Autonomous Bio-hybrid Robot System
165
also been shown to be non-linear in the neck motor system of the blowfly [9] and a mobile platform for recordings will enable further investigation of such gating mechanisms dependent on locomotion. Following on from previous research, where a mobile extracellular recording platform was designed [10] and a potential control architecture was proposed [11], here we describe a closed loop control implementation on a bio-hybrid robot system that involves the signals of both H1-cells during forward movement of the robot. In addition we accumulate data to enable collision avoidance in future applications of the system. The H1-cell is one of the lobula plate tangential cells (LPTCs) located in the third optic lobe. There are about 60 identified directional-selective interneurons in the lobula plate [12]. So-called heterolateral tangential cells convey visual motion information from one lobula plate to its contralateral counterpart. The other LPTCs connect directly or via descending neurons, to the various motor systems supporting gaze control, flight stabilization and collision avoidance [3, 13]. The H1-cell is a heterolateral spiking interneuron sensitive to horizontal back-to-front motion over the ipsilateral eye, the cell’s preferred direction (PD), which increases its spike rate. Front-to-back motion in its null direction (ND) inhibits spiking in the H1-cell [14]. If there is no visual motion at all, the H1-cell generates action potentials at a low spontaneous rate of 10-30 spikes per second. Its signals are comparatively easy to measure by means of extracellular recordings, and has been studied for decades [15]. Compared to other LPTCs, the H1-cell has a large receptive field. It’s azimuthal sensitivity to visual motion ranges from -15° (contralateral) to 135° (ipsilateral), and from +45° to -45° elevation above and below the eye equator of the animal [14]. Previous research has shown the temporal frequency tuning of the H1-cell under wide field stimulations, e.g. for a constant spatial frequency (black and white vertical stripe pattern) [10] or a combination of different spatial frequencies (lab environment) [11]. Due to the contrast dependence of its responses, H1-cell signals are believed to decrease as the distance to visual objects increases, which has important implications for using the H1-cell as motion vision sensors in a closed-loop robotic system that avoids collision with objects in the surroundings [16]. As mentioned above, spiking in the H1-cell increases upon horizontal visual motion from back-to-front and is inhibited by front-to-back motion. The challenge is to control the forward movement of a robot with a biological sensor that is sensitive to back-to-front motion. If a forward facing blowfly is placed on top of a robot driving forward, both H1-cells will be inhibited. If the blowfly is facing backwards, during motion, both H1-cells will be excited, but the frontal area of the robot would be out of the receptive field of the H1-cells. It was discussed in previous work [11], one possible way to control the robot by H1-cell activity would be to implement a preprogrammed sinusoidal trajectory. Instead of inhibiting both H1-cells in straight forward motion, the sinusoidal oscillating robot trajectory would stimulate the H1-cells one after the other, thus, producing feedback signals for the control loop. But there are still several unknown parameters that need to be determined before launching such an experiment, including: the optimum turning radius and turning frequency of the preprogrammed sinusoidal robot trajectory.
166
J.V. Huang and H.G. Krapp
In this paper, we are introducing a primitive closed-loop control method which generates an autonomous forward oscillation for the bio-hybrid robot system, in which more detailed information is gathered for the development of a closed-loop control system that supports collision avoidance, e.g. turning radius and turning frequency of the robot.
2
Methods
2.1
Bio-hybrid Robot System
The bio-hybrid robot system consists of three subunits: i) the blowfly, ii) the mobile extracellular recording platform, iii) a two-wheeled robot (Pololu© m3pi), as shown in figure 1.
Fig. 1. The assembly of the bio-hybrid robot system. The mobile extracellular recording platform is mounted on top of the m3pi robot. The blowfly is positioned in the center of the recording platform. The whole bio-hybrid robot system weights 587 grams. The inset in the top left corner shows a rear view of the opened blowfly head capsule with two tungsten recording electrodes placed in the left and right lobula plate, respectively.
Closed-Loop Control in an Autonomous Bio-hybrid Robot System
167
The mobile extracellular recording platform is similar to the version described previously [7]. It consists of an aluminium chassis holding three parts in position: i) the blowfly, ii) the tungsten recording electrodes and micromanipulators to position them, and iii) custom-designed high-gain amplifiers (Figure 1). A number of features were found to improve experiments. These include: a) re-routing the PCB for better signalto-noise ratio and to eliminate electrodes cross talk between channels, b) lifting up the electrodes for a better view of blowfly’s brain during electrode placement, c) blowflies are mounted from the front, protecting the electrodes from damage during frequent removal, d) improved electrical shielding of the recording platform. The m3pi robot contains an ARM processor (NXP© LPC1768) which is now programmed with firmware, driving three modules: i) the ADC module, for digitizing bi-lateral H1-cell action potentials, sampled at 5 KS/s each side, ii) the H1-cell model which converts the spike rates into wheel speed control commands, iii) the UART module, for sending a control command to the robot, that generates PWM voltages to control the DC motors, driving the wheels. 2.2
Blowfly Preparation
Female blowflies, Calliphora vicina, from 4-11 days old were used in the experiments. The fly’s legs and proboscis were cut off and the wounds were sealed with bee wax to reduce any movements which could degrade the quality of the recordings. Wings were immobilized by a small droplet of bee wax on the wing hinges. The head of the fly was placed in between two pins of a custom-made fly holder while adjusting its orientation with reference to the ‘pseudopupil methods’ [17] before being waxed to the pins. The back of the head capsule was exposed by bending down the thorax and fixing it to the pins. Holes were cut into the back of the head capsule under optical magnification using a stereo microscope (Stemi 2000, Zeiss©) on either side. Fat and muscle tissue were cleared to get access to the lobula plate and physiological Ringer solution (for recipe see Karmeier et al. [18]) was added to the brain frequently to prevent desiccation. Tungsten electrodes (3 MΩ tungsten electrodes, UEWSHGSE3N1M, FHC Inc., Bowdoin, ME, USA) were used for extracellular recording from H1-cells. The placement of the electrode was adjusted to approach the H1-cell by using a micromanipulator. Recordings of acceptable quality had signal-noise-ratios of > 2:1, i.e. the amplitude of the recorded H1-cell spikes was at least 2 times higher than the largest amplitudes of the background noise. The signals were amplified by a nominal gain of 10,000 with band pass filter from 300 Hz to 3000 Hz. A copy of the signal was sampled by a data acquisition board (NI USB-6215, National Instruments Corporation, Austin, TX, USA) at a rate of 20KS/s. 2.3
Control Algorithm Design
The H1-cell model is derived from the inverse function of the temporal frequency tuning curve obtained in the lab environment [11], which transfers a spike rate (input signal) into an angular velocity (output signal). The curve of the model is simplified to a straight line for the range from 50 Hz to 300 Hz at H1-cell spike rate, and keeps the zone below 50 Hz as dead zone, shown in figure 2. The simplified model is
168
J.V. Huang and H.G. Krapp
applied for primitive investigation so that the gain can be easily configured to adjust the stability of the system, in addition to reducing processing time in the ARM processor to keep the control loop delay as short as possible.
Fig. 2. The H1-cell model. [Left] the H1-cell velocity tuning curve in the lab environment [11], the blue curve is the simplified straight line approximation of the tuning curve. [Right] the inverse function of the simplified tuning curve on the left plot, which is programmed in an ARM processor on-board of the robot.
Fig. 3. The block diagram of the bio-hybrid robot system
On top of the monotonic H1-cell inverse function model, the system is capable of eliciting self-oscillation with the control loop designed as figure 3. The control loop of the bio-hybrid robot system can be broken down into two sub-loops based on two
Closed-Loop Control in an Autonomous Bio-hybrid Robot System
169
H1-cells, which influence each e other. Specifically, the increment of the spike ratee on the left H1-cell will increasse the speed of the left robot wheel, which will generatte a clockwise yaw motion on the robot. This yaw motion inhibits the left H1-cell and excites the right H1-cell, and vice-versa for the right H1-cell. The way in which the m that an increase in spike rate of a given H1-cell pprocontrol loops are coupled means vides negative feedback on its own activity, but positive feedback to the H1-cell in the contralateral part of the braiin.
3
Results
3.1
Robot Trajectory
After providing an initial sttimulation to one eye of the blowfly, in order to pass thee 50 Hz threshold in activity to o start movement, the wheel on one side drives the roobot away from the movement seen s by the fly. The resulting turn produces motion acrross the other eye, stimulating the contralateral H1-cell, and this sequence repeats aas a series of oscillations, gradu ually driving the robot forwards. The overall motion of the robot was forward, with a little bias to the right wall of the tunnel. (Video liink: https://www.youtube.com/w watch?v=yNZAB3YrplY) From the overhead video-footage of the experiments, the orientation and posittion of the robot relative to the center-line c in between the walls of the tunnel was foundd by analyzing individual framees using a program written in Python OpenCV. 90° is defined as the viewing directiion of the fly being aligned with the center-line of the ttunnel. The coordinates of the robot center and the robot’s rear were recorded from eeach frame. Those coordinates were w connected to reconstruct the trajectory of the robott on a frame-by-frame basis, plo otted as green (robot central trajectory) and red (robot poosterior trajectory) curves in fiigure 4, respectively.
Fig. 4. The robot trajectory. Th he experimental arena was a tunnel with vertical stripes (gratinngs) attached to the walls (spatial wavelength: w 30 mm or 16.2° from initial point). The green curvve is the trajectory of robot center; the t red curve is the trajectory of the robot’s rear.
170
J.V. Huang and H.G. Krapp
The orientation of the rob bot for each frame was derived from the robot’s central and rear points via an arctangentt function, where the 0° of the polar coordinate is definedd on the right of the video framees. The oscillation frequency of the robot was found too be around 5 Hz, with amplitude of 5° to 10° change in orientation with each cycle (Figure 5).
Fig. 5. Robot orientation plot. Data were derived from the frame-by-frame video analysis. 900° in orientation refers to an orientatiion of the robot that is aligned with the center line of the tunnel.
3.2
H1-cell Responses
H1-cell activities were reco orded during the autonomous closed-loop control, shownn in figure 6. The bilateral H1-cell exttracellular recording is performed on the fast moving robbot. The recording quality was acceptable a by means of the following criteria: i) the signnalnoise-ratios of each cell aree both over 2:1, which means a single threshold was suufficient for successful spike detecting, d ii) the inter-spike interval rates of each cell did not exceed 400 Hz at any time, which means the recordings were obtained from sinngle neurons with sensible refraactory periods, iii), the peak spike rate, spontaneous sppike rate and horizontal motion--induced excitation/inhibition properties indicate the sinngle neuron recordings were obttained from (the left and right) H1-cells. Double thresholds were used here for spike sorting to accurately detect the occcurrence time of the peak of eaach action potential. The peak is found by searching for the minimum value between tw wo consecutive negative threshold crossings and two cconsecutive positive threshold crossings. The top threshold is set at 2.5V, where the reference potential is, and the bottom threshold line is set at around 60% of the averrage action potential peak. Two threshold lines are located at the top and bottom edgee of 6 the magenta area in figure 6.
Closed-Loop Control in an Autonomous Bio-hybrid Robot System
171
Fig. 6. Recording of bi-lateral H1-cell activities during autonomous closed loop control, plootted from 0 to 5 second. [Top] thee blue raster is the occurrence of the left H1-cell action potenntial, the magenta curve is the interr-spike interval (ISI) rate, and the yellow curve is the Gausssiansmoothened spike rate. [Upperr] the neuron activity of the left H1-cell, the purple area indiccates the threshold values for spike detection. [Middle] the comparison of two spike rates from eeach n activity of the right H1-cell, the purple area indicatess the side of H1-cell. [Lower] the neuronal threshold values for spike deteection. [Bottom] the red raster is the occurrence of the right H1cell action potential, the mageenta curve is the ISI rate, the cyan curve is the Gaussian sm moothened spike rate.
172
J.V. Huang and H.G. Krapp
The oscillations were initiated when the H1-cell spike rate reaches 50 Hz. The reason of the vanishing of the oscillation is not clear from the plot yet. The oscillation frequency is around 5 Hz, counted from figure 6, which matches well with the result from previous video processing, as figure 5. The peak spike rates during the oscillation are less than 300 Hz, which fits well with the H1-cell model.
4
Discussion
4.1
Control Loop Analysis
As the results show, under closed-loop conditions, bi-lateral H1-cell activity drives the robot forward in an oscillatory trajectory. Coincidently, in previous studies it was observed that blowflies do produce sinusoidal trajectories when flying through a narrow tunnel [19]. It was concluded that they generated alternating phases of rapid saccadic turns followed by drift phases during which they assess relative distance to the walls based on translation-induced optic flow. The horizontal systems (HS) lobula plate tangential cells which respond to front-to-back horizontal motion were suggested to be involved in the task that enables the flies to steer away from the closed wall. HS-cells are output cells of the lobula plate and connect to both the neck and the flight motor systems [20]. It is unclear whether or not the H1-cell response characteristic plays a cardinal role in case of distance estimation based on HS-cell activities, but the integration of signals from both eyes certainly does. In case of our robot experiment, the integration of both H1-cell signals may also play an important role. This experiment showed that the gain configuration used did not produce divergent oscillations. It is unclear as yet how equilibrium of oscillation is achieved, and what determines the oscillation frequency of the robot orientation at around 5 Hz. Further experimentation is needed to explore those details. Although there is uncertainty surrounding the question at how the oscillation vanishes, there is a way to bypass this issue by tuning the curve of the H1-cell model down, when reducing the dead zone spike rate from 50 Hz to 25 Hz. The spontaneous activity would then sustain the oscillation for all times – at least in an environment with striped patterns. 4.2
Towards Collision Avoidance
Theoretically, the system should achieve collision avoidance by only using the two H1-cells themselves. But from the trajectory, we can see the robot system is not yet capable of doing that. Ultimately it generates a bias in oscillation towards the right hand side of the wall. Two things need to be considered here. First is the balance (mass distribution) of the robot, which has not been taken care of yet in the robot platform. Certain components could be upgraded. The friction on each wheel is required to be identical to avoid drift. Alternatively, an advanced robot with feedback of wheel speed could overcome this problem.
Closed-Loop Control in an Autonomous Bio-hybrid Robot System
173
Second is the neuronal activity from both H1-cells, which needs to be normalized, due to the variance of the neuronal response caused by the neuron’s asymmetric structure or firing rate adaptation. Two 360° yaw rotations in opposite direction would have to be performed right after powering up the robot, to measure the peak spike rate of each H1-cell for subsequent normalization of the dynamic output range generated by the left and right cells. H1-cells are quite sensitive to rotation. As we can see, the bio-hybrid system can achieve 5 Hz oscillation frequency during forward movement and the full swing of the robot orientation in the oscillation is around 5° to 10°. It is possible to calculate the robot tuning radius based on the 5 Hz turning frequency and the 5° to 10° peak to peak angle of robot orientation oscillation. These parameters will guide us when developing and implementing a collision avoidance control algorithm including a preprogrammed trajectory as proposed in the previous paper [11].
5
Conclusion
This work presents the first time an autonomous closed-loop control implemented on a bio-hybrid robot system using simultaneous recordings of bilateral blowfly H1-cells, which can drive the robot forwards based on visual feedback information. The data collected with the system will be used to optimize the platform performance to achieve a collision-free trajectory in arbitrarily structured environments. Acknowledgments. We’d like to thank Ben Hardcastle and Peter Swart for proof reading and discussion of this paper. And thanks to Gary Jones who provided professional support on fabrication of mechanical components. Also appreciate Yilin Wang, Martina Wicklein and Léandre Varennes-Phillit for all the help and experience sharing on the work presented. This work was partially supported by US AFOSR/EOARD grant FA8655-09-1-3083 to HGK.
References 1. Bomphrey, R.J., Walker, S.M., Taylor, G.K.: The Typical Flight Performance of Blowflies: Measuring the Normal Performance Envelope of Calliphora vicina Using a Novel Corner-Cube Arena. PLoS One 4, e7852 (2009) 2. Hengstenberg, R.: Gaze control in the blowfly Calliphora: a multisensory, two-stage integration process. Semin. Neurosci. 3, 19–29 (1991) 3. Lindemann, J.P., Weiss, H., Möller, R., Egelhaaf, M.: Saccadic flight strategy facilitates collision avoidance: closed-loop performance of a cyberfly. Biol. Cybern. 98, 213–227 (2008) 4. Ejaz, N., Peterson, K.D., Krapp, H.G.: An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces. J. Vis. Exp. (2011) 5. Lewen, G.D., Bialek, W., de Ruyter van Steveninck, R.R.: Neural coding of naturalistic motion stimuli. Netw. Bristol. Engl. 12, 317–329 (2008) 6. Minegishi, R., Takashima, A., Kurabayashi, D., Kanzaki, R.: Construction of a brain–machine hybrid system to evaluate adaptability of an insect. Robot. Auton. Syst. 60, 692–699 (2012)
174
J.V. Huang and H.G. Krapp
7. Reid, R. Harrison, R.J.K.: Wireless Telemetry of In-Flight Collision Avoidance Neural Signals in Insects 26 (2010) 8. Maimon, G., Straw, A.D., Dickinson, M.H.: Active flight increases the gain of visual motion processing in Drosophila. Nat. Neurosci. 13, 393–399 (2010) 9. Huston, S.J., Krapp, H.G.: Nonlinear Integration of Visual and Haltere Inputs in Fly Neck Motor Neurons. J. Neurosci. 29, 13097–13105 (2009) 10. Huang, J.V., Krapp, H.G.: Miniaturized electrophysiology platform for fly-robot interface to study multisensory integration. In: Lepora, N.F., Mura, A., Krapp, H.G., Verschure, P.F., Prescott, T.J. (eds.) Living Machines 2013. LNCS, vol. 8064, pp. 119–130. Springer, Heidelberg (2013) 11. Huang, J.V., Krapp, H.G.: A predictive model for closed-loop collision avoidance in a fly-robotic interface. In: Duff, A., Lepora, N.F., Mura, A., Prescott, T.J., Verschure, P.F. (eds.) Living Machines 2014. LNCS, vol. 8608, pp. 130–141. Springer, Heidelberg (2014) 12. Borst, A., Haag, J.: Neural networks in the cockpit of the fly. J. Comp. Physiol. A 188, 419–437 (2002) 13. Lindemann, J.P., Kern, R., van Hateren, J.H., Ritter, H., Egelhaaf, M.: On the Computations Analyzing Natural Optic Flow: Quantitative Model Analysis of the Blowfly Motion Vision Pathway. J. Neurosci. 25, 6435–6448 (2005) 14. Krapp, H.G., Hengstenberg, R., Egelhaaf, M.: Binocular Contributions to Optic Flow Processing in the Fly Visual System. J. Neurophysiol. 85, 724–734 (2001) 15. Hausen, K.: Functional characterization and anatomical identification of motion sensitive neurons in the lobula plate of the blowfly Calliphora erythrocephala. Z Naturforsch, 629–33 (1976) 16. Egelhaaf, M., Kern, R., Lindemann, J.P.: Motion as a source of environmental information: a fresh view on biological motion computation by insect brains. Front. Neural Circuits. 8, 127 (2014) 17. Franceschini, N.: Pupil and pseudopupil in the compound eye of drosophila. In: Wehner, R. (ed.) Information Processing in the Visual Systems of Anthropods, pp. 75–82. Springer, Berlin Heidelberg (1972) 18. Karmeier, K., Tabor, R., Egelhaaf, M., Krapp, H.G.: Early visual experience and the receptive-field organization of optic flow processing interneurons in the fly motion pathway. Vis. Neurosci. 18, 1–8 (2001) 19. Kern, R., Boeddeker, N., Dittmar, L., Egelhaaf, M.: Blowfly flight characteristics are shaped by environmental features and controlled by optic flow information. J. Exp. Biol. 215, 2501–2514 (2012) 20. Taylor, G.K., Krapp, H.G.: Sensory systems and flight stability: what do insects measure and why?. In: Casas, J., Simpson, S.J. (eds.) Advances in Insect Physiology, pp. 231–316. Academic Press (2007)
