Dec 15, 2005 - face unit to the retrieval processing unit, image data stored in the image information storing unit is re
USO0RE42185E
(19) United States (12) Reissued Patent
(10) Patent Number: US RE42,185 E (45) Date of Reissued Patent: Mar. 1, 2011
Sekiguchi et a]. (54)
5,819,286 A
IMAGE RETRIEVAL SYSTEM AND IMAGE RETRIEVAL METHOD
7,006,714 B2 *
(75) Inventors: Shunichi Sekiguchi, Tokyo (JP); Yoshimi Isu, Tokyo (JP); Hirofumi NishikaWa, Tokyo (JP); Yoshihisa Yamada, Tokyo (JP); Kohtaro Asai,
2002/0105541 A1 *
2/2006 Kasutani 8/2002
EP GB JP
Mitsubishi Denki Kabushiki Kaisha,
(Continued) Sung et al., “Iconic pictorial retrieval using multiple attributes, ” Systems, Man, and Cybernetics, 1998, 1998 IEEE International Conference, San Diego, CA, USA,, Oct.
Related US. Patent Documents
11, 14, 1998, New York, NY, USA, IEEE, US, vol. 5, pp. 458941594, Oct. 11, 1998, XP010311152.
Reissue of:
Patent No.:
6,665,442
Issued:
Dec. 16, 2003
Appl. No.:
09/773,570
Filed:
Feb. 2, 2001
(Continued) Primary ExamineriSamir A Ahmed Assistant ExamineriAli Bayat (74) Attorney, Agent, or FirmiBirch, Stewart, Kolasch & Birch, LLP.
US. Applications:
(30)
Continuation of application No. PCT/JP00/08547, ?led on Dec. 1, 2000.
(51)
(57)
Foreign Application Priority Data
Dec. 2, 1999
(JP)
11/1998 10/1998 8/1990
OTHER PUBLICATIONS
(21) Appl. No.: 11/300,693 Dec. 15, 2005 (22) Filed:
(63)
382/305
Endou et al. .............. .. 345/738
0878767 A 0872803 A1 2-219391 A
Tokyo (JP)
(64)
Yan et a1. ............... .. 707/104.1 Takahashiet a1. .... .. 375/24001
FOREIGN PATENT DOCUMENTS
Tokyo (JP) (73) Assignee:
10/1998 Yang et al.
6,502,105 B1 * 12/2002 6,792,043 B1 * 9/2004
ABSTRACT
When a retrieval condition of an attribute list is input from a user interface unit to a retrieval processing unit, the attribute list stored in an attribute list storing unit is retrieved in the
......................................... .. 11-343256
Int. Cl.
(2006.01)
retrieval processing unit. Thereafter, attribute information
(52)
US. Cl. ...................... .. 382/224; 382/190; 382/195;
played on a displaying unit. Thereafter, When a retrieval con
(58)
Field of Classi?cation Search ................ .. 382/232,
G06K 9/62
conforming to the retrieval condition is output to and dis
dition of the similarity retrieval is input from the user inter face unit to the retrieval processing unit, image data stored in the image information storing unit is retrieved in the retrieval processing unit, and speci?c image data relating to a charac teristic descriptor set conforming to the retrieval condition is selected in the retrieval processing unit. Thereafter, the spe ci?c image data is output to and displayed on the displaying
382/232; 382/305
382/224, 217, 305, 236, 190; 348/231.99; 707/104.1, 3, 6, 2, 100, 102 See application ?le for complete search history.
(56)
References Cited
unit. U.S. PATENT DOCUMENTS
5,465,353 A
11/1995 Hull et al.
28 Claims, 17 Drawing Sheets
, --------------- --
-
I
1
g
5
:
_-
____ __ _
SERVER
IMAGE DATA BASE PRODUCING UNIT
7
k
1 I
ATTRIBUTE DATA
|
ATTRIBUTE LIST
|
PRODUCING UNIT
5
l
6 CHARACI'ERISTIC
m. E
CHARACTERIS'I'IC
__
- DESCRIPI'OR sEI'
:
EXTRACIZNG uurr
PRODUCI‘NG UNIT
I
3
4 ____
{
--- ................. --.,
I I
\__
8
l
IMAGE RETR'EWNG UNIT
:
I 2 | 5 ll
"
STORING UN"
RErRIEvAI.
-
INs'rRucI'IoN
:
___________ __
RErRIEvAI.
-
l
: 10
s
l
RETRIEVAL PROCBSING
l
l I I
INSTRUCTION
: I
5 __
:
__ ___
_ _ _ __
____I - _ '7
i 9 IMAGE DATA
CLIENT
_ '\
11
l
S
:
|
s
|
USER INTERFACE
DISPLAYING
|
UNIT
UNIT
l
I I
I
I
\ ________________ _ _ _¢
US RE42,185 E Page 2
FOREIGN PATENT DOCUMENTS JP JP JP JP JP JP JP JP TW TW TW WO
3-270388 6168277 9-270006 09282324 10091634 10124655 10-320400 11224266 84112558 85104458 85112595 WO-98/21688
A A
A
12/1991 6/1994 10/1997 10/1997 4/1998 5/1998 12/1998 8/1999 11/1995 4/1996 10/1996 5/1998
Shibata, Masahiro, Temporal Video Segmentation for Scene Description, Transactions of the Institute of Electronics,
Information and Communication Engineers, Apr. 25, 1999, vol. J82*D*11,No. 4, pp. 7324742. Yoshimi Isu et al., “XML Kijutsu wo mochiiru Contents
Kensaku Jikken System no Kouchiku”, Proceedings of
Information System Society Meeting in 1999, the Institute of Electronics, Information and Communication Engineers (Sep. 7, 1999), p. 132, Fig. 1.
JauiYuen Chen et al., “ViBE: a new paradigm for Video database browsing and search, ” ContentiBased Access of
“Synthesis and Evalution of the Image Database with Fully Automated Keyword Extraction by State Transition Model and Scene Description Language”, Institute of Electronic Information and Communication Engineers ofJapan, Dill vol. J79iDill No. 4, pp. 4764483, Apr. 1996.
Image and Video Libraries, 1998, Proceedings, IEEE Work shop on Santa Barbara, CA, USA, Jun. 21, 1998, Los Alami tos, CA, USA, IEEE Comput. Soc., US, 1998, pp. 964100,
“A critical evaluation of mage and Video indexing techniques in the compressed domain”, M.K. Mandal et al., Image
XP010293858.
ViSiOI’l Computing, 17, 5134529, 1999.
A
OTHER PUBLICATIONS
Hatano, Kenji, New Generation Database Technology, Authoring and Retrieval of Video Scenes by Multiilevel Sel fmrganizing Maps, Transactions of the Institute of Informa
Ono et al., IEICE, DAII vol. J79, No. 4, pp. 476483 (1996)
w/partial English Translation.
tion Processing Society of Japan, Apr. 15, 1998, VOl. 39, No. 4, pp. 933942.
* cited by examiner
US. Patent
Mar. 1, 2011
Sheet 1 0f 17
US RE42,185 E
FIG.1 (PRIOR ART) 101
102
S
S
PREPARATION UNIT
103
RETRIEVAL TOOL
104
A
IMAGE
S
CONCEPTION —»
KEYWORD
EXTRACTING UNIT
9 ,
I;
108
Eo
AREA
7
[4,] E
DIVIDING
E3 5;
UNIT
-
SCENE DESCRIPTION V
KEYWORD
DESCRIBING UNIT
i E I: 5
~
109
I
105
U9D
1; 0 106 ~ DESCRIPTION
107 ~
g; m 2
USER
PARAMETER INPUT N112
111~
US. Patent
Mar. 1, 2011
US RE42,185 E
Sheet 3 0f 17
FIG?» START
1 I
3T1.~
5T2
~
I
?’??g?gg" 0F IMAGE
EXTRACTION OF IMAGE
(TEXT INFORMATION) I
CHARACI‘ERJSTIC VALUE
~ 8T4
r PRODUCTION OF
PRODUCTION OF
A'I'I‘RIB UTE LIST
ggggagrrg?'ggc
I
1
REGISTRATION OF
CHARACTERISTIC
A'I'I‘RIBUTE LIST
DESCRIWOR SET
~
5T5
REGISTRATION OF ST3 ~
END
I CALCULATION OF AVERAGE ~ ST15_1
MAGNITUDE OF MOTION I
THRESHOLD PROCESSING
~ST15-2
II
PRODUCI‘ ION OF ACTIVITY OF MOTION DISTRIBUTION
l
r» STlS-3
~ ST6
US. Patent
Mar. 1, 2011
Sheet 5 or 17
US RE42,185 E
WA21C.
_
_
_
+4:41!
QUE
MOA U I
/uoqmdlz
Am8muzw2niM25
US. Patent
Mar. 1, 2011
Sheet 7 or 17
US RE42,185 E
FIG.8 SHORT RUN
MIDDLE RUN
O 0 LONG RUN
DCUInI 0R
DCVIn'I
L
avg std (DISPERSION FROM AVERAGE) > TIME
US. Patent
Mar. 1, 2011
Sheet 8 or 17
US RE42,185 E
MPEG-1 CD
A: alWideoMail ¥Shun 5&5 meeting. mpeg
10.5
47.5
50.2
55
CHARACI‘ERISTIC
_
VALUE SET EXTRACI‘ED
FROM VIDEO BIT
fPDG'
STREAM
dUDC>
DEFINITION FILE FOR DESCRIBING CHARACTERISTIC VALUES
(CHARACTERISTIC VALUE DESCRIPTION DATA CLASS) (DTD)
US. Patent
Mar. 1, 2011
US RE42,185 E
Sheet 10 0f17
I INPUT OF RETRIEVAL CONDITION
~ST18
FIG. 12
I ST19 CONFORMITY JUDGMENT OF ATTRIBUTE
INCONFORMITY
CONFORMITY TRANSFER OF CONFORMITY RESULT TO DISPLAYING UNIT
DISPLAY OF RETRIEVAL RESULT
sT23
~ ST2O
ST22 REPRODUCTION INSTRUCTION
INSTRUCTION OF REPRODUCTION/SIMILARTT Y RETRIEVAL
ST24
RETRIEVAL INSTRUCTION
S
SELECTION OF DISPLAYED IMAGE
SELECTION OF SIMILAR
RETRIEVED IMAGE 5
ST26
V
REPRODUCTION AND DISPLAY OF SELECTED IMAGE
ST27 CONFORMITY JUDGMENT OF CHARACTERISTIC DESCRIPTOR INCONFORMITY CONFORMITY TRANSFER OF CONFORMITY RESULT TO DISPLAYING UNIT
néM?
ST29
(M:LENGTH OF LIST OF CHARACTERISTIC DESCRIPTOR
SETS) YES
5
ST25
~ ST28
US. Patent
Mar. 1, 2011
Sheet 11 0f 17
Ta, min, Ta, maxzTHRESHOLD VALUES
OF CHARACTER DESCRIPTOR "a" (Ta, min
