178
Measurement Automation Monitoring, May 2016, no. 05, vol. 62, ISSN 2450-2855
Tomasz MAŃKOWSKI, Jakub TOMCZYŃSKI, Piotr KACZMAREK POZNAN UNIVERSITY OF TECHNOLOGY, INSTITUTE OF CONTROL AND INFORMATION ENGINEERING 3A Piotrowo St., 60-965 Poznań, Poland
A test bench setup with programmable trajectory for IMU-based dataglove development Abstract This paper presents a test bench created during development of an IMUbased dataglove. The system allows gathering synchronised hand phantom posture and trajectory, together with inertial sensor data for evaluation of posture recognition algorithms. The setup includes an inertial measurement system, with IMU modules designed to be mounted on human hand or a phantom, together with proper embedded firmware and PC software. The system uses a collaborative Universal Robots UR3 robot arm for generation of trajectories and as a source of ground truth data. The program used to control the robot together with a method of gathered data synchronisation are described. The test bench has been successfully used during development of a joint angles estimating algorithm. Keywords: dataglove, test bench, IMU, MARG, Universal Robots, UR3.
Fig. 1. Overall system architecture diagram
1. Introduction
Lastly, a custom Inertial Measurement System (IMS) was proposed. IMS is connected with a PC via High Speed USB in order to provide a sufficient bandwidth to transfer raw IMU data. IMS consists of a Dataglove MainBoard (MB) which is the main hub unit collecting IMU data, and multiple IMU Boards (IB), each equipped with an independent MARG sensor. Moreover, MB is equipped with an additional MARG sensor allowing finger position estimation relative to hand palm. MB is also capable of generating synchronization signal for the UR3 robot. Ground-truth data acquisition of finger joint position is possible using potentiometer position readout via ADC.
In recent decade, augmented and virtual reality has gained significant momentum in both research and entertainment industry. While the topic of visual feedback, including virtual reality devices like Oculus Rift, is constantly evolving, the matter of reliable and novel input devices can still be considered underdeveloped. Many recent solutions, such as Microsoft Kinect or Leap Motion rely on the use of depth cameras for human body pose acquisition. Using optical sensors means vulnerability to difficult lighting conditions and requirement of line-of-sight between the sensor and user. Precision is also limited by the sensor resolution, especially when analysing a large scene. Another way of implementing a Human-Machine Interface (HMI) is a dataglove. Dataglove is a wearable device, in the form of a five-finger glove, which can read posture of each individual hand finger. These solutions are resistant to most environmental interferences. Development of datagloves has been a subject of research for many groups. Types of sensors used in various projects include fiber optic sensors [1], bend sensors [2] or Inertial Measurement Units (IMU) [3, 4]. An extensive survey on dataglove applications can be found in [5]. In this paper, a test bench setup for IMU-based dataglove system is presented. It features a description of implementation of IMU data collection architecture with the maximum number of 17 9-DOF (Degrees of Freedom) MARG (Magnetometer, Angular Rate and Gravity) sensors. Moreover, the designed test bench enables acquisition of ground-truth data for joint angles of a phantom hand and hand base linear and angular position and velocities. Thanks to cooperative robot utilization, the test bench allows programming a precise movement trajectory and easy phantom hand joint adjustment.
2. System architecture A general diagram of the test bench setup is presented in Fig. 1. The whole system consists of 3 main parts. A PC class computer with dedicated software was chosen as a main acquisition unit, as it can easily serve as a bridge between other parts of the setup. The second part of the setup is a Universal Robots UR3 cooperative robot which serves as both movement trajectory ground-truth, and can be used to generate specific and precise movement of the base IMU unit. As UR devices are equipped with an easy to use TCP/IP interface, position and velocity data is transferred to a PC using a LAN Ethernet connection.
3. Inertial measurement system 3.1. Hardware IMS is based on 9-DOF MARG sensors LSM9DS0 by STMicroelectronics. This unit was chosen as one of the cheapest solutions available on the market, offering measurement of triaxial acceleration, angular rate and magnetic field in one integrated chip. MB is equipped with one LSM9DS0 sensor and has 6 connectors which allow connecting 16 additional external IMU Boards. Size of IBs was minimized in order to enable their placement on human hand phalanges, resulting in PCB dimensions of 12.5 by 12.5 mm. IBs are connected with MB using SPI interface. Connection between MB and IB was designed to limit the number of data lines. 3-wire SPI standard was chosen and IB PCB was designed as a passthrough for Chip Select (CS) lines. As LSM9DS0 CS lines are divided between Gyroscope (G), and Accelerometer and Magnetometer (AM), each IB connector is fitted with a line to choose between G and AM. Moreover, 3 Board Select (BS) lines are available in each connector. A custom designed circuit based on 7400-series ICs enables passing G/AM and clock signal (SCK) to a chosen IB or group of IBs by setting selected BS low. For board select circuit schematics see Fig. 2.
Fig. 2. Board select circuit schematic
Measurement Automation Monitoring, May 2016, no. 05, vol. 62, ISSN 2450-2855
MB is controlled by the STM32F405RG microcontroller by STMicroelectronics. Communication with a PC is carried out by FT232HL chip by FTDI, which serves as a USB⇔SPI bridge operating at 10.5 MHz.
3.2. Firmware IMS firmware was written using STM32F4 Standard Peripheral Library in C language. All readouts are triggered using hardware timer, generating interrupts at 100 Hz. Due to incompatibilities in STM’s SPI peripheral 3-wire mode, which generated unnecessary SCK pulses, software GPIO-based SPI was implemented for communication with MARG sensors. As communication with particular MARGs is performed in short bursts, no significant performance hit was caused by using software SPI. The system uses double buffering - data is fed into the first buffer. When data transfer is triggered by PC software, the write buffer is switched and the previous buffer contents are sent to the USB bridge chip using DMA. The frame header contains a number of samples in current packet, and may also contain additional data, such as potentiometer readouts or robot synchronisation data (see 4.1.). Data transfer integrity is ensured by means of CRC checksum calculated per-frame by hardware peripheral in the microcontroller. The firmware supports basic commands received from the host, which start and stop data acquisition.
4. Robotic arm A flexible, collaborative robotic arm - UR3 by Universal Robots was used as a way to perform given trajectories and acquire kinematic data of the system. Choosing a collaborative robot for the task gives an additional benefit of being able to put the robot in the freedrive mode, where the operator is able to alter robot configuration by grabbing its arm and moving it, even during program execution. The robot controller operates using a proprietary programming language interpreter - URScript. The set of commands available allows all basic and advanced robotic operations, such as executing various types of movement and rotations, as well as acquiring information about its current kinematic and dynamic state. Together with ability to communicate via standard TCP/IP interface, the robot makes an excellent ground truth trajectory data source.
179
connection with the PC. The script polls the TCP/IP socket in search of a server with designated IP address. The process repeats until the connection is established. Depending on the type of experiment, the routine also puts the robot into the freedrive mode, or starts to perform a given trajectory. Then the robots goes into an infinite loop and waits for a synchronization signal. Each time the state of robot Tool Input 0 changes, the second script is executed. If the connection has been properly established, the program reads current arm pose, speed, and state of the synchronisation inputs. Then the script assembles a TCP packet and sends it to the PC.
5. PC acquisition software Acquisition of data from various sources required custom PC software. The whole software used by the test bench is written using Qt 5 framework, and with maximum portability between various operating systems in mind. The application was written in a modular way, with a supervising module responsible for controlling the experiment process and passing data between particular modules. Various types of experiments can be performed, with experiment and device data stored in XML files. The software functionality can be extended through additional modules, which provide support for new sensors, display and computational modules. Firstly, the software starts a TCP/IP server socket for a UR3 robot to connect, establishes the USB connection with MB and exchanges the initialisation data. After the experiment is fired, it starts to flush the MB data buffer, which also triggers a robot synchronization. New position and velocity data is acquired via the TCP/IP socket. All the received data is marked with a time stamp and unique source id, and redirected into the recording module. The data is written to a CSV file, for further analysis in other environments such as MATLAB or GNU Octave. Thanks to synchronization data, a correlation of the signal generated by MB and acquired from the robot can be run in order to calculate the offset between the inertial data and the robot position data.
6. Experimental evaluation The test bench was successfully used in a series of experiments during development and assessment of algorithms estimating joint angles. A 3D-printed kinematic structure with two parallel rotational joints, simulating simplified human finger with three phalanges, was prepared. The experimental setup is shown in Fig. 3.
4.1. Synchronisation Acquisition of data from various sources poses a problem of proper sample synchronisation. Using TCP/IP for posture data transfer further complicates the task, as it does not provide determinate time constraints. To minimize influence of delays introduced by various communication channels, an additional synchronisation channel was added. The IMS’s MB generates two synchronisation clocks, precise and coarse, which are fed into the robot tool input. A precise signal is toggled at each acquisition of inertial data (100 Hz), a coarse signal is toggled at each tenth acquisition (10 Hz). Signals generated by the MB have to be converted to meet the robot I/O standards. This was achieved using a voltage converter based on operational amplifiers in comparator configuration. The robot controller loop works at a frequency that guarantees 16.0 ms maximum delay of input signal propagation to the software, in the case of robot tool inputs. However, the performed tests showed that the propagation time never exceeded the sampling period (10.0 ms).
4.2. URScript The robot data acquisition software consists of two subprograms written in URScript. The first, run at startup, establishes a TCP/IP
Fig. 3. The test bench during test run
A series of test runs was performed, during which the robot executed various types of movement. After recording, the coarse synchronisation signal acquired from both IMS and robot controller was used to verify proper data synchronisation. Only below 5% of recordings showed any offset, which was easily
180
Measurement Automation Monitoring, May 2016, no. 05, vol. 62, ISSN 2450-2855
corrected in postprocessing through maximising cross-correlation between the recorded signals. Fig. 4. presents the experimental results for roll axis ration measured by the IMU gyroscope and UR3 robot. Residuals analysis shows that errors are caused by the gyroscope accuracy, not a system lag.
Part C: Applications and Reviews, IEEE Transactions on, vol.38, no.4, pp.461-482, 2008.
_____________________________________________________ Received: 11.02.2016
Paper reviewed
Tomasz MAŃKOWSKI, MSc Tomasz Mańkowski received an MSc degree in Electrical Engineering, in 2014 from Poznań University of Technology. His highly awarded master thesis has been a motor of his further research. He is an PhD student and professor assistant at Poznań University of Technology at Faculty of Electrical Engineering. His scope of interest include biosignal amplifiers, biosignal processing and cybernetics.
e-mail:
[email protected] Jakub TOMCZYŃSKI, MSc In 2014 Jakub Tomczyński has graduated Poznań University of Technology gaining an MSc degree in Electrical Engineering. He has received a numerous awards for his master thesis. Today he is continuing his research as an PhD student at Poznań University of Technology at Faculty of Electrical Engineering. His research include machine learning and cybernetics.
Fig. 4. Roll axis angular rate and residuals measured by base IMU and UR3 e-mail:
[email protected]
Although all the recorded data was valid, it is worth noting that the proximity of the robot metal structure and magnets present in the robot drives affect magnetometer readouts in a noticeable way. Additional magnetometer calibration is recommended after mounting IMS on the robot for minimisation of this effect.
7. References [1] da Silva A.F.; Goncalves A.F.; Mendes P.M.; Correia J.H.: FBG Sensing Glove for Monitoring Hand Posture. Sensors Journal, IEEE, vol.11, no.10, pp.2442-2448, 2011. [2] http://www.cyberglovesystems.com/cyberglove-ii/ [3] http://synertial.com/products/ [4] Santaera G.; Luberto E.; Serio A.; Gabiccini M.; Bicchi A.: Low-cost, fast and accurate reconstruction of robotic and human postures via IMU measurements. In Robotics and Automation (ICRA), 2015 IEEE International Conference on, pp.2728-2735, 2015. [5] Dipietro L.; Sabatini A.M.; Dario P.: A Survey of Glove-Based Systems and Their Applications. In Systems, Man, and Cybernetics,
Piotr KACZMAREK, PhD Piotr Kaczmarek received the M.Sc. degree in Electrical Engineering from the Poznań University of Technology, in 2003 and the PhD degree in biocybernetics from the same University in 2009. His research work includes modeling of biological systems and FES system control. Since 2003 he has been employed by Institute of Control and Information Engineering at Poznań University of Technology.
e-mail:
[email protected]
Accepted: 04.04.2016
