Aug 9, 1999 ... ecutlon apphcatlon ?led under 37 CFR. Comput. & Graphics ..... U.S. Patent. 0a.
30, 2001. Sheet 2 0f 9. US 6,310,605 B1. FIG. 2a. I. I. a l. 90/'. ______ ... 204.
K204. I] Pr0gram1. [I ProgramZ. File Edit. File Edit. 4. 7. \/\Z24. 272 i. /. [7 / \. 2 70
..... 1 is a block diagram of a system for controlling a force feedback ...
US006310605B1
(12)
United States Patent
(10) Patent N0.: (45) Date of Patent:
Rosenberg et a1.
(54)
FORCE FEEDBACK INTERFACE WITH
*Oct. 30, 2001
FOREIGN PATENT DOCUMENTS
SELECTIVE DISTURBANCE FILTER
(75)
US 6,310,605 B1
Inventors: Louis B. Rosenberg, Pleasanton; Dean
0265011181
4/1988 (EP) '
0626634 A2
11/1994 (EP) '
gjsghang, Mountain View, both of CA
(List continued on next page.) OTHER PUBLICATIONS
(73) Assignee: Immersion Corporation, San Jose, CA (Us)
GotoW, J .K., et al., “Perception of Mechanical Properties at the Man—Machine Interface,” IEEE 1987, pp. 688—689.
(*) Notice?
This_ Pawnt lssufid on a Continued PTOS-
Atkinston, William D. et al, “Computing With Feeling,”
ecutlon apphcatlon ?led under 37 CFR 1.53(d), and is subject to the tWenty year patent term provisions of 35 U.S.C.
Comput. & Graphics, vol. 2, No. 2—E, pp. 97—103. (List continued on next page.)
154(a)(2).
Primary Examiner—Jeffery Brier
Subject to any disclaimer, the term of this patent is extended or adjusted under 35
(74) Attorney, A gent, 0r Firm—James R. Riegel
U.S.C. 154(b) by 0 days. _
_
_
(57) _
_
ABSTRACT
A force feedback interface and method providing a selective
Thl_s Patent 15 Sublect to a termmal dls' Clalmer'
disturbance ?lter for providing selective reduction or elimi nation of displayed disturbances associated With output force sensations. A force feedback interface device is con
(21) APPL NOJ 09/370,948 (22) Filed Aug 9 1999 .
.
nected to a host computer that displays a graphical environ ment. The interface device includes a user manipulatable
,
object, a sensor for detecting movement of the user object, and an actuator to apply output forces to the user object. A
Related US. Application Data
microprocessor outputs controlling force signals to the
.................................................... .. G06F 3/033
actuator, receives sensor signals from the sensors and reports locative data to the host computer indicative of the move ment of the user object. The host computer updates a position or a displayed usepcontrorred graphical Object in
(52)
345/157; 345/161
the graphical environment based on the reported data. The
(58)
Field Of Search ................................... .. 345/156, 157,
microprocessor implements a Selective disturbance ?lter for
(63)
Continuation of application No. 08/839,249, ?led on Apr. 14, 1997, now Pat. No. 6,020,876.
(51)
Int. Cl.
(56)
7
345/158> 161> 184; 463/30> 38; 244/223;
modifying the locative data reported to the host computer
434/45; 318/568~1> 568-11 _
When the output force Would cause a disturbance to the user-controlled graphical object, the disturbance occurring
References Clted Us PATENT DOCUMENTS
When an output force sensation affects the position of the user object such that the host computer Would display the user controlled graphical object in an undesired location or
1213; ,
,
$011 am e
~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ 3415/24; a .
With an undesired motion in the graphical environment.
........................... ..
(List continued on next page.)
43 Claims, 9 Drawing Sheets
APPROPRlATE
COMMANU 13 A FlLTIZH coo/MAW:
cum 1 \NUF
OPERATlON As PRHHOUSLY
comncww
315 RU‘OW mn/mvE DNA 10 HUM cuww LR
NORMALLV
M) w
MBLE
ACTWE FlLTIZRlS!
RF mm NODWED LouAm/E DATA
,
TO HOST CF>NPUT£R
32”
US 6,310,605 B1 Page 2
US. PATENT DOCUMENTS 4,436,188
3/1984 Jones .................................. .. 188/378
5,755,577 5,767,839
5/1998 Gillio ................................. .. 434/262 6/1998 Rosenberg ......................... .. 345/161
577697640
6/1998 Jacobus et at ____________________ " 434/262
4,477,043
10/1984 Repperger ---- --
244/223
5,781,172
7/1998 Engeletal.
345/164
4,632,341
12/1986 Repperger et a1-
244/230
5,790,108
8/1998 Salcudean etal.
345/184
4,679,331
7/1987 Koontz ----- --
4,689,449
8/1987
4,782,327
R9599 - - - - - -
11/1988 Kley et a1
33/551
5,802,353
9/1998 Avila etal.
395/500
- - - - -- 200/6
5,805,140
9/1998
345/161
340/365 P
5,808,601
9/1998 Lean etal.
60/393
5,825,308
10/1998 Rosenberg ..
..... .. 341/20
12/1998 Penraeb etal.
318/56817
Rosenberg etal.
4,800,721
1/1989 Cemenska et a1-
4,803,413 4,861,269
2/1989 Kendig et a1 8/1989 Meenen, Jr.
-- 318/648
5,844,392
434/45
578577986
1/1999 Moriyasu
4,868,549 4,896,554
9/1989 A?inito et al 1/1990 Culver ....... ..
~- 340/710 74/471 XY
5,877,748 578897672
3/1999 Redlich ..... .. 3/1999 Schuler et at
4,897,582
1/1990 Otten et a1
5,007,300
4/1991
5,044,956
Siva --------- ~-
11/1991 Repperger
5,065,145 5,076,517 5,088,055
11/1991 Purcell ------ -12/1991 Ferranti et a12/1992 Oyama ------ --
5,103,404
4/1992 McIntosh
5,107,080
4/1992
5,107,262 5,116,051
4/1992 (39991 et a1 5/1992 Moncrief et al
5,128,671
7/1992 Thomasalr
Rosen ........ ..
318/135
5,914,705
6/1999 Johnson et al. .
5,944,151
8/1999 Jakobs etal.
. 188/267.1
434/45
5,956,484
9/1999 Rosenberg et al.
395/200.33
5,959,613
9/1999 Rosenberg etal.
345/161
340/706 - 244/228 -- 364/560
5,990,869 5,999,168 6,001,014
11/1999 Kubica etal. 12/1999 Rosenberg etal. 12/1999 Ogata etal.
318/568-22
6,004,134
12/1999 Marens etal. ..
345/163 . 345/161 463/37 434/45
.. 200/6A
670057551
12/1999
_ 345/161
341/22
6,028,593
2/2000 Rosenberg etal.
345/156
- 273/448 B
6,088,017
7/2000 Tremblay etal.
345/156
341/20 .. 414/5
6,111,577
8/2000 Zillesetal. ........................ .. 345/355
5,143,505
9/1992 Burdea et al. .
9/1992 Hollis, Jr. et al. .
. 395/275
5,184,319
2/1993
.. 364/806
Osborne et at
FOREIGN PATENT DOCUMENTS
5,185,561
2/1993 Good etal.
.. 318/432
W09502801
1/1995 (W0)
5,186,629
2/1993
.. 434/114
W09520788
8/1995
5,193,963
3/1993 McAffee etal.
.... .. 414/5
W09532459
5,195,179
3/1993
5,223,776
6/1993 Radke et al.
5,264,768
5,296,871
11/1993
Rohen ....... .. Tokunaga ..... ..
345/163
244/175
5,146,566
Kramer ........ ..
601/49 .345/163 _ 364/188
74/471 XY
9/1991 Behensky et a1
5,062,594
345/145
(W0)
11/1995 (W0)
.. 395/161
.. 318/5681
Gregory et al.
OTHER PUBLICATIONS
318/561
3/1994 Paley .............................. .. 345/157 X
_
“
_ _
Rosenberg Louls B- et a1» Perceptual Decomposmon of
5,298,890
3/1994 Kanamaru et al. ................ .. 345/157
Virtual Haptic Surfaces,” IEEE Symposium on Research
5,298,918 *
3/1994 Yen-Chen et al. .
Frontiers in Virtual Reality, Oct. 1993.
5,396,266
Burdea Ct 8.1. ......................... .. 3/1995 Brlmh'all ............................ .. 345/161
Jones, “A perceptual analysis of stiffness”, Research, 79:150_156, 1990~
Performance Rosenbergea-wsing 1“ Graphlcal UserFeedbacktojinhance Interfaces > WWWaCmOrg/
.. 345/163
5,451,924 2:222:22?
9/1995 22222 Massimino 2322225221.: et al. ................... ............ .. 340/407.1 2127222
5,512,919
4/1996 Araki ........... ..
5,513,100
4/1996 Parker et al. ...................... .. 364/167
5,565,887
10/1996 McCambridge et 8.1. ......................... et al. .. 345/145
.. 345/156
2:222:82 12/1996 1211222 Armstrong 22222225221:":1111111111111111: 2221121 34100
575897828
Brain
SigChi/Chi96/pr0C99dingS, Apr- 1996
Ramstein, Christophe, “Combining Haptic and Braille Tech nologies: SIGGAPH, Design 1996,Issues PP- 37—44 and
Study”, Assets :96,
Payee e ab ‘evaluation, of a Fejdback (Haptic) Computer Pointing Device in Zero Gravity, DSC—v0l. 58,
5,589,854
12/1996 Tsai .................................... .. 345/161
5,591,924
1/1997 Hilton ........................... .. 73/862043
ASME 1996, pp. 547—553.
5,596,347
1/1997 Robertson 9t a1~
Rosenberg, Louis, “The Use of Virtual Fixtures to Enhance
~~ 345/145
glosellbertgl """""""""""""" ,
,
576297594
assle
e
a .
PIOC- 0f the ASME Dynamics Systems and Control Diva
Operator Performance in Time Delayed Teleoperation,”
..................... ..
5/1997 Jacobus ~~~~~ N
-
318568“
Armstrong Lab., CreW Systems Directorate, 1993, pp. 1—45.
5,631,861
5/1997 Kramer .............................. .. 364/406
Ramsteln et al» “The Pantographi A Large Workspace
5,642,469
6/1997 Hannaford et al. .................. .. 395/99
Haptic Device for a MulIim0dalH1lmaH—C0mPuI@rInterac
5,643,087
7/1997 Marcus et al. .
tion,” Cornputer—Hurnan Interaction, CHI ’94.
463/38
Kurita ................................ ..
5,666,138 5’691’898 5,701,140
et al_, “Design of a Four Degree_Of_FreedOm
9/1997 Culver ................................ .. 345/161 11/1997 Rosenberg et a1‘ " 364/190 12/1997 Rosenberg et al. ................ .. 345/156 Chen etal.
FOrCe_Re?eCting Manipulandum With a Speci?ed Force/ Torque Workspace”, Proc. of 1991 IEEE Int’l Conf. on Robotics and Automation IEEE 1991 1488_1493
5,709,219
1/1998
5,714,978
2/1998 Yamanaka et at
5,721,566
2/1998 Rosenberg et al. ................ .. 345/161
IEEE 0—7803—1363—1, 1993, pp. 287—292.
5,731,804
3/1998 Rosenberg Svancarek Ct...... 8.1. .. ............ .. .. 345/156 X
in Ouh_yOung, MolecularMing, Docking,” “Using aIEEE Manipulator CH2555_1, for Force 1988, pp‘
5,734,373
3/1998 Rosenberg et al. ................ .. 345/161
1824_1829'
5,736,978
4/1998
577397811
4/1998 Rosenberg et at
__ 345/161
Wlney III, Calvin, “Cornputer Simulated Visual and Tactile
5,742,278 * 5,754,023
4/1998 Chen et al. ........................ .. 345/156 5/1998 Roston et al. ..................... .. 318/561
Feedback as an Aid to Manipulation and Vehicle Control,” MIT, 1981, pp. 1—79.
Haser et al.
........................ .. 128/782
__ 345/157
....................... ..
345/173
’
’
app"
"
IWata, Hiroo, “Pen—based Haptic Virtual Environment”,
_
_
_
_
_
US 6,310,605 B1 Page 3
Howe, Robert D. et al., “Task Performance With a DeXtrous
Adelstein et al., “Design and Implementation of a Force
Teleoperated Hand System,” Proceedings of SPIE, vol.
Re?ecting Manipulandum,” NASA Ames Research Center,
1833, 1992, pp. 1—9. Kotoku, T. et al., “Environment Modeling for the Interactive Display used in Telerobotic Systems”, IROS ’91, IEEE Cat. No. 91Th0375—6, 1991, pp. 999—1004. Batter et al., “Grope—1: A Computer Display to the Sense of Feel,” Proc. IFIP Congress 1971, pp. 759—763.
Ouh—young, Ming, “Force Display in Molecular Docking,” Univ. of North Carolina at Chapel Hill, 1990, pp. 1—12, 66—85.
Tan, H. et al., “Manual Resolution of Compliance When Work and Force Cues are MinimiZed,” DSC—vol. 49, ASME 1993, pp. 99—104.
Russo, Massimo, “The Design and Implementation of a
Three Degree—of—Freedom Force Output Joystick,” 1990, pp. 1—40.
Rosenberg, Louis, “Virtual Haptic Overlays Enhance Per formance in Telepresence Tasks,” Stanford Univ., 1994. Adachi, Y., et al., “Sensory Evaluation of Virtual Haptic Push—Buttons,” Technical Research Center, SuZuki Motor Corp., 1994. Su, Augustine et al., “The Virtual Panel Architecture: A 3D Gesture Framework” IEEE 4—7803—1363—1, 1993, pp. 388—393.
Buttolo, Pietro et al., “Pen—Based Force Display for Preci sion Manipulation in Virtual Environments,” IEEE 0—8186—7084—3, 1995, pp. 217—224. Ellis, RB et al., “Design and Evaluation of a High—Perfor mance Prototype Planar Haptic Interface,” DSC—vol. 49, ASME 1993, pp. 55—64. Burdea, Grigore et al, “Distributed Virtual Force Feedback,” IEEE Workshop on Force Display in Virtual Environments and its Application to Robotic Teleoperation, 1993, pp. 25—44. Hasser, Christopher, “Tactile Feedback for a Force—Re?ect
ing Haptic Display,” Univ. of Dayton, 1995, pp. 1—65. Rosenberg et al., “Commercially Viable Force Feedback Controller for Individuals With Neuromotor Disabilities,”
Armstrong Lab., CreW Systems Directorate, 1996, pp. 1—33. Repperger, D.W., “Active Force Re?ection Devices in Tele
operation,” IEEE Control Systems, 0272—1708, 1991, pp. 52—55.
1992, pp. 1—24.
Minsky et al., “Feeling and Seeing: Issues in Force Display”, Association for Computing Machinery, 1990, pp. 235—242. Tan et al., “Human Factors for the Design of Force—Re?ect
ing Haptic Interfaces,” ASME WAM 1994, pp. 1—11. Fischer et al., “Speci?cation and Design of Input Devices for Teleoperation”, IEEE Conference on Robotics, 1990, pp. 540—545. Schmult et al., “Application Areas for a Force—Feedback
Joystick”, ASME 1993, pp. 47—54. Hannaford, B. et al., “Force—Feedback Cursor Control,” NASA Tech Brief, vol. 13, No. 11, Item #21, 1976, pp. 1—4. Kelley, A]. et al., “On the Development of a Force—Feed back Mouse and Its Integration into a Graphical User Interface,” Symp. on Haptic Interfaces for Virtual Environ
ment and Teleoperator Systems, 1994 Int’l Mech. Eng. Congress and Exhibition, 1994, pp. 1—8. Rosenberg, “Perceptual Design of a Virtual Rigid Surface
Contact”, Center for Design Research, Stanford University,
Armstrong Laboratory, CreW Systems Directorate, Apr. 1993, pp. 1—42. Kelley, A]. et al., “MagicMouse: Tactile and Kinesthetic Feedback in the Human Computer Interface using an Elec
tromagnetically Actuated Input/Output Device,” Dept. of Electrical Engineering, Univ. of British Columbia, 1993, pp. 1—27.
Kilpatrick, Paul, “The Use of a Kinesthetic Supplement in an Interactive Graphics System,” Univ. of N. Carolina at
Chapel Hill, Computer Science, 1976, pp. 1—175. Hirota, K. et al., “Development of Surface Display,” Dept. of Mechano—Informatics, Univ. of Tokyo, IEEE 0—7803—1363—1/93, pp. 256—262. IWata, H. et al., “Arti?cial Reality With Force—feedback:
Development of Desktop Virtual Space With Compact Mas ter Manipulator,” Computer Graphics, vol. 24, No. 4, 1990, pp. 165—170.
Wiker, Steven et al., “Development of Tactile Mice for Blind Access to Computers: Importance of Stimulation Locus,
Akamatsu, M. et al., “Multimodal Mouse: A Mouse—Type Device With Tactile and Force Display,” Presence vol. 3, Winter 1994, pp. 73—80. Munch, Stefan et al., “Intelligent Control for Haptic Dis
Object SiZe, and Vibrotactile Display Resolution,” Proc. of the Human Factors Society 35”1 Annual Meeting, 1991, pp.
plays,” Eurographics ’96, vol. 15, No. 3, 1996, pp.
Force Display,” Computer Science Dept., Univ. of N. Caro lina at Chapel Hill, 1989, pp. 1—14.
C—217—C226. Hannaford, B. et al., “Performance Evaluation of a SiX—AXis
Generalized Force—Re?ecting Teleoperator,” 0018—9472, 1991, pp. 620—623, 631—633.
IEEE
Colgate et al., “Implementation of Stiff Virtual Walls in
708—712.
Ouh—young, “Creating an Illusion of Feel: Control Issues in
HayWard, Vincent et al., “Design and Multi—Objective Opti miZation of a Linkage for a Haptic Interface,” Advances in
Robot Kinematics and Computationed Geometry, 1994, pp. 359—368.
Force—Re?ecting Interfaces,” NorthWestern Univ., IL, Sep. 22, 1993.
* cited by examiner
U.S. Patent
0a. 30, 2001
Sheet 1 0f 9
US 6,310,605 B1
[/12
HOST COMPUTER SYSTEM
OH _7
E FMU
_1.igmm
E M _AL
29
|-r .
m0 TORE_UUYH_DTR.1|;
-O M__
EmSAP_EC
V U n pA]
Ya0w"m5"omcWPgoARCRE\_TmMP3“mm"Em"
2 _7
CmPS._S
lI| 4 _‘
_LH.
_AUl
1% n F. CF
0_ Ro3 0 2lI
S% _ 48.L
R _ _
0_ SSS
7n-. L
w _NI3 w .-M G
_
_ _
_
_.EJ
AwEmF_FN|_TA
"|l|_:
_
U.S. Patent
0a. 30, 2001
Sheet 2 0f 9
FIG. 2a I I
70x
90/‘
______ ___
a l
I 88b
2% 80
98
FIG. 2b
US 6,310,605 B1
U.S. Patent
0a. 30, 2001
Sheet 3 0f 9
F103
US 6,310,605 B1
U.S. Patent
0a. 30, 2001
Sheet 4 0f 9
US 6,310,605 B1
34
FORCE FEEDBACK PERIPHERAL _
_ Wu T 2 O P _
VIM ||
AUnNl 6CR__
72
oCT|,\, 700o/D\,OW
COMPUTER
NPRA0
A
A4O D
“ w M cR _0M m =_FS _L_OS
TNET A0“_
AN.“ W 070 E T
__R0 C
m _s _
wun w n _8
I
