Haptic Manipulation of Microspheres Using Optical Tweezers İbrahim Büküşoğlu*
Cagatay Basdogan┼
Alper Kiraz+
Adnan Kurt║
(*,┼)College of Engineering, Koc University, Istanbul Turkey (+, )College of Arts and Sciences, Koc University, Istanbul Turkey ║
ABSTRACT We report the manipulation of glass microspheres having a diameter of 3-10 µm using optical tweezers and with haptic feedback. We detect the position of a microsphere manipulated in a fluid bed using a CCD camera and calculate the forces acting on it due to the optical trap and viscous drag. We calculate the optical forces between the laser beam and the manipulated particle using a mass-spring-damper model. For this purpose, we calibrated the optical trap and used image processing and curve fitting techniques to evaluate the coefficients of the mass-spring-damper model. The drag force is calculated using the velocity of the sphere and the viscous damping coefficient of the fluid. We then use a potential field approach to generate a collision-free path for the manipulated microsphere among other spheres and display the optical trapping and drag forces and the forces due the artificial potential field to a user of the system via a haptic device for better manipulation and steering. We have observed performance improvements over manual control in our preliminary manipulation experiments.
wavelength develop an electric dipole moment in response to the light’s electric field. This leads to a gradient force which attracts the particle to the beam focus with a magnitude proportional to the intensity gradient of the laser beam. Larger objects act as lenses, causing to a change in the momentum of the incident photons. This results in an effective force which draws the particle towards the higher flux of photons near the focus [1]. For the inverted geometry depicted in Fig. 1, the particle reaches equilibrium along the axis of the laser beam (z-axis) by the gradient, scattering, and the gravitational forces. In the following sections we will primarily discuss the motion of the trapped particle in the plane perpendicular to the z-axis. The scattering and gravitational forces have no contribution on this lateral motion when the particle is in equilibrium along the axis of laser beam.
z
CR Categories: I.4.8 [Image Processing and Computer Vision]: Scene Analysis---object recognition; H.5.2 [Information Interfaces and Presentation]: User Interfaces---haptic I/O; I.2.9 [Artificial Intelligence]: Robotics---operator interfaces, commercial robots and applications; I.6.3 [Simulation and Modeling]: Applications.
r Fgrad
Keywords: optical tweezers, haptics, path planning, image-based micro manipulation.
1.
INTRODUCTION
Optical trapping is very attractive as it provides non-contact trapping forces that can manipulate objects spanning from single atoms to microscopic particles [1,2]. Single-beam optical traps, i.e. optical tweezers (Figure 1), were first introduced in 1986 [2,3]. Optical tweezers can trap objects as small as 5 nm and exert forces exceeding 100 pN [4]. Trapping objects using optical tweezers relies on a highly focused laser beam (Figure 1). A laser beam exerts gradient and scattering forces on a small polarizable object [5]. Small objects with sizes comparable to laser *e-mail:
[email protected] † e-mail:
[email protected] ┴ e-mail:
[email protected] ▀ e-mail:
[email protected]
Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems 2006 March 25 - 26, Alexandria, Virginia, USA 1-4244-0226-3/06/$20.00 ©2006 IEEE
r Fscat r G
Glass microsphere
Coverslip Laser Beam
Figure 1. Optical tweezers: Optical trapping using a single laser beam.
Optical tweezers are highly attractive for investigating biological and macromolecular systems since the scale of the applied forces are highly small. Optical tweezers have been used to probe the viscoelastic properties of single biopolymers (such as DNA), cell membranes, aggregated protein fibres (such as actin), gels of such fibres in the cytoskeleton, and composite structures (such as chromatin and chromosomes). They have also been used to characterize the forces exerted by molecular motors such as myosin, kinesin, processive enzymes and ribosomes [1].
361
An important challenge in manipulating micro and nano scale objects with optical tweezers is its control. The potential field of the laser beam acts like a nonlinear spring. When an external force is applied to a micrometer-sized particle trapped in the optical trap, the particle is displaced from the focus of laser beam in a manner similar to an object that is attached to a mechanical spring and exposed to an external force. Moreover, the drag force around the particle acts like a damper. Hence, optimum control is crucial for precise transportation and positioning of the particles by using small scaled forces. To this end, haptic control appears to be a natural choice. The drag forces and trapping forces acting on the particle can be displayed to a user via a haptic device during the manipulation. This way, manipulation tasks can be performed faster without the loss of control. In addition, artificial force fields can be used during the steering of a trapped particle for better haptic guidance and to prevent the trapping of the other particles. In this study, we discuss our optical trapping set-up, the preliminary experiments that are designed to characterize the movement of a trapped particle in a fluid and the steering control of the trapped particles using a haptic device. 2.
3.
MODELING PARTICLE MOTION IN AN OPTICAL TRAP
The trapping potential depends on the properties of the laser beam and the trapped particle, as well as the refraction index of the surrounding medium. The approximate potential energy of the particle in an optical trap can be calculated by [5]:
U = U 0e
⎛ x2 y2 − ln( 2 ) ⎜ 2 + 2 ⎜a b ⎝
⎞ ⎟ ⎟ ⎠
(1)
where U0 is the well depth, x and y are the axes perpendicular to the laser path, and a and b are the beam waist dimensions. The trapping force exerted on the particle by the laser field along the x-axis is given by the gradient of the potential field as
Fgradx =
− 2 ln(2)U 0 ( X P − X L ) a2
e
⎛ ( X − X )2 ⎞ −ln(2)⎜ P 2 L ⎟ ⎜ ⎟ a ⎝ ⎠
(2)
SET-UP
The set-up for optical trapping is shown in Fig. 2. The beam of a continuous wave green laser (Crysta Laser CRL-GCL-025-L, λ = 532 nm) with an output power of 25 mW is sent through a 6x magnifying telescope into an inverted microscope (Nikon TE 2000-U). After being reflected off a dichroic mirror (Chroma Filters Q570LP), the laser beam is focused on the sample by a high numerical aperture (NA = 1.4, 60x) microscope objective. Transmission microscopy images are captured by a CCD camera. By using an intermediate magnification module, a total magnification of 90x is achieved. A red pass filter (Chroma filters HQ610/75) is used to filter out stray laser light. The sample is scanned by a three-dimensional piezoelectric scanner (Piezosystem Jena Tritor 102-SG) working in the closed loop control (scanning resolution is 2 nm). An aqueous solution of glass beads (Polysciences Inc.) with diameters ranging from 3-10 µm are used in the experiments. The solution was kept between a coverslip and a thick glass with a separation of ~100 µm. White light
where, XP and XL are the positions of the particle and laser beam respectively (the laser position is fixed, XL = 0, in our application). In reality, theoretical prediction of trapping forces is not easy, but a number of emprical methods have been developed to determine the trapping forces, each with its own advantages and disadvantages [see the review in 7]. Considering the drag force on the particle, the equation of motion of the particle can be written as
mX&& P = −∇ U − b( X& P − X& S )
XS
Sample Three-dimensional piezoelectric scanner
Xp
XL = 0
Output
Input
Oil 60x NA=1.4
(3)
kNL
Telescope Dichroic mirror
l=532 nm, 25 mW Nd:YAG Laser
Red pass filter 1.5x intermediate magnification module
CCD camera
Figure 2. Our experimental setup for optical trapping microscale particles.
362
S
P b
Figure 3. The schematic diagram of the system dynamics.
Here, m is the mass of the trapped particle, XS is the position of the piezoelectric scanner, and b( X& P − X& S ) represents the drag force acting on the particle moving in a fluid.
4.
EXPERIMENTS WITH MICROSPHERES
In order to make the haptic control possible, we first need to calculate the coefficients b, U0 and m. To calculate these coefficients, we applied a sinusoidal displacement input to the scanner at a frequency of 1 Hz (see Figure 1) and captured the motion of a trapped microsphere using a video imaging system at 25 Hz. We then calculated the position of the sphere from the captured video frames using the Image Processing Toolbox of Matlab (see Figure 4).
Figure 6. Fitted vs original data
5.
MANIPULATION EXPERIMENTS WITH HAPTIC FEEDBACK
We fitted separate sinusoidal waves to the recorded positions of the scanner and sphere to estimated the velocity of the scanner as well as the velocity and acceleration of the sphere through differentiation of the position data (Figure 5). This approach is more robust than the numerical calculation of velocities and accelarations from the position data using the finite difference method directly.
A haptic device (Omni, Sensable Tech.) was connected to the piezo scanner such that the scanner displacement is controlled by the movement of the haptic device (Figure 7). We first programmed the haptic device to act like a typical force feedback joystick. The feedback forces opposite to the movement of the scanner were calculated using a virtual spring between the center position of the scanner and the current position of the haptic interface point (HIP) and reflected to the user for the manipulation control. We observed that even joystick type of control enhances the manipulation skills of the user. This mode of manipulation is used for controlling the movements of the stage and locking the laser to the particle.
Figure 5. Input (scanner position) and output (particle position) data after sinusoidal curve fitting. The phase shift in the output response is an indication of the viscoelastic forces acting on the particle.
Figure 7. The set-up for manipulating microspheres using optical tweezers and haptic feedback.
Figure 4. Frames captured by the video imaging system are used to determine the position of a trapped microsphere, XP, and the scanner, XS. We used one of the untrapped spheres to determine the position of the scanner. One pixel is 0.1756 microns in pictures.
The position, velocity, and accelaration data was then inserted into the dynamical equation and the unknown coefficients were calculated using the least squares curve fitting technique (Figure 6)
To simplify the manipulation of particles, we neglect the force interactions between the laser beam and the trapped particle and we assume that the focus point of the laser beam and the center of the particle is always coincident during the manipulations. Hence, the movements of the haptic interface point represents the scaled movements of the laser focus and particle. We then implemented a path planning algorithm based on a potential field approach to compute the collision-free path of the particle for a given goal point. In this approach, obstacles and the goal are represented by repulsive and attractive potential fields respectively. The negative
363
gradient of the total potential gives the force applied on the particle. To implement the potential field approach, we first take a picture of the manipulation field using the camera. From the picture, the location and the radius of all spheres in the field are calculated using the standard image processing techniques (Figure 8). Then, the potential field approach is implemented to define the collision free path of the particle to be manipulated (Figure 9). Due to the potential field, there are attractive and repulsive forces acting on the particle. The attractive force is a result of a linear spring between the particle and the goal, pulling the particle towards the goal point. The repulsive force is modeled as a nonlinear spring between the particle and the obstacle. The spring force increases quadratically as the distance between the particle and the obstacle is reduced linearly (Figure 10). A simulated model of the manipulation field (Figure 11) was constructed from the camera image and displayed to the user in real-time to help him/her be aware of the particle position with respect to the goal and the other particles in the field. Since the field of view of the camera changes continuously as the particle is manipulated, the simulated environment helps the user keep the big picture in mind. As the user manipulates the particle in the virtual world using the haptic device, the laser beam manipulates the particle in the real world. Three modes of steering were tested and the forces acting on the user, the particle position, and the task completion time were recorded in all modes and then compared with each other to investigate the role of haptics in optical manipulation of micro particles. These manipulation modes are steering 1) without haptic feedback (WF), 2) with drag force as a haptic feedback (DF), and 3) with the summation of drag force and force due to potential field as a haptic feedback (DPF).
Figure 9. A potential field approach was used to calculate the collision-free path of the manipulated sphere.
Figure 10. Attractive and repulsive forces acting on the manipulated sphere due to an artificial potential field.
Figure 8. Picture of the manipulation field after image processing
Figure 11. The simulation environment for path planning and real-time manipulation.
364
The steering paths for the three manipulation modes and the ideal path calculated by using the potential field are displayed in Figure 12. The root mean square error (RMSE), task completion time, and the work done by the user to complete the task are tabulated in Table 1 for the three manipulation modes. The figure and table shows that the best and worst performances were obtained in DPF and WF modes respectively. In the DF mode, the user must work against the drag forces. In the DPF mode, no work was done by the user and the haptic device guides the user to steer the particle. The work done in the WF mode is simply equal to zero since no haptic feedback is present.
DPF DF WF path
250
200
manipulation to investigate the role of haptics in steering of micro particles. The results of the experiments show that the summation of artificially generated path forces and the drag forces displayed to the user enable the user to manipulate the trapped particle more accurately and efficiently. In the next phase of the project, the forces between the laser beam and the particle will also be displayed to the user via the haptic device to help him/her adjust the velocity of the haptic interface point which in turn controls the velocity of the piezo stage. This is important for keeping the particle in the trap. This force will be calculated using the parameters determined by the characterization experiments. The user will control the focus point of the laser beam via the haptic device and the particle will move according to the equilibrium between the drag force and the spring force due to the effect of laser potential. In our work, we only focused on the in-plane movements of particles. However, a movement in the z direction (along the laser beam) is also possible by changing the position of the scanner which will be further investigated in the future along with the strategies for automated manipulation.
150 REFERENCES
100
50
[1] D. G. Grier, “A revolution in optical manipulation,” Nature, 424, 810– 816 (2003). 0
50
100
150
200
250
[2] N. Schlosser, G. Reymond, I.E. Protsenko and P. Grangier, “Subpoissonian loading of single atoms in a microscopic dipole trap”, Nature 411, pp. 1024, 2001.
300
Figure 12. The desired path and the paths followed by the particle in three different manipulation modes.
[3] Ashkin, J. M. Dziedzic, J. E. Bjorkholm, and S. Chu, “Observation of a single-beam gradient force optical trap for dielectric particles,” Opt. Lett. 10, pp. 288–290, 1986. [4] Ashkin, A. “History of optical trapping and manipulation of smallneutral particle, atoms, and molecules.”, IEEE J. Sel. Top. Quantum Elec. 6, pp. 841–856, 2000.
Table 1. Comparison of three different manipulation modes
6.
Measures
DPF
DF
WF
RMSE
2.7653
5.3751
6.2832
Time (sec)
6.2
6.2
9.3
Work
-75.229
19.273
0
[5] Rohrbach, A. & Stelzer, E. H. K. “Trapping forces, force constants, and potential depths for dielectricspheres in the presence of spherical aberrations.”, Appl. Opt. 41, pp. 2494–2507, 2002. [6] Lee, Yong-Gu, Lyons, Kevin W., LeBrun, Thomas W., ‘‘Virtual Environment for Manipulating Microscopic Particles with Optical Tweezers’’, Journal of Research of NIST, Vol. 108, No 4, 2003. [7] Fällman, Erik, Schedin, Staffan, Jana, Jass, Andersson, Magnus, Bernt, Eric U., Axner, Ove, “Optical tweezers based force measurement system for quantitating binding interactions: system design and application for the study of bacterial adhesion”, Biosensors and Bioelectronics, Vol. 19, pp. 1429–1437, 2004. [8] Visscher, K.; Gross, S.P.; Block, S.M., “Construction of multiple-beam optical traps with nanometer-resolution position sensing Selected Topics in Quantum Electronics”, IEEE Journal of Selected Topics in Quantum Electronics, Vol. 2, Issue 4, pp. 1066 – 1076, 1996.
CONCLUSIONS
We have developed a set-up for haptic manipulation of microscale objects using optical tweezers. The haptic feedback enables the user to control the movement of the trapped object in a more efficient and accurate manner. First, we programmed the haptic device to act like a force feedback joystick for locking the laser to the particle. After locking, the particle is assumed to be glued to the laser beam, which is controlled by the user via the haptic device. We experimented with three modes of
365
