Journal of Electronic Imaging 17(1), 013008 (Jan–Mar 2008)
Feature extraction and clustering for the computer-aided reconstruction of strip-cut shredded documents Anna Ukovich Giovanni Ramponi Dipartimento di Elettrotecnica, Elettronica ed Informatica University of Trieste v. A. Valerio 10 I 34127 Trieste, Italy E-mail:
[email protected]
Abstract. We propose a solution for the computer-aided reconstruction of strip-cut shredded documents. First of all, the visual content of the strips is automatically extracted and represented by a number of numerical features. Usually, the pieces of different pages have been mixed. A grouping of the strips belonging to a same page is thus realized by means of a clustering operator, to ease the successive matching performed by a human operator with the help of a computer. © 2008 SPIE and IS&T. 关DOI: 10.1117/1.2898551兴
1 Introduction The reconstruction of shredded or torn documents is a problem that may arise in the forensics and investigative science fields. Image processing techniques give the opportunity for computer-based reassembly. Typical documents that one may need to reconstruct are office documents, such as printed text documents, or handwritten notebook paper. In general, the different pages that have been shredded may have different font format, ink color, and page layout. In Fig. 1 some remnants are shown. For computer-aided reconstruction, the strips need to be first acquired by means of a scanner, then segmented from the background.1 At this point, features describing their visual content are extracted. The visual content needs to be extracted, because the piece contour shape commonly used in the analogous problem of automatic jigsaw puzzle assembly is not very useful here, since the strips have approximately all the same shape 共long and narrow rectangular strips兲. Once the strips are described by the features, the search for matching pieces is the search for pieces that are similar in the feature space. In this work, we focus on stripcut shredded documents for which the cutting direction is orthogonal to the text direction. The work is organized as follows. In Sec. 2, the state of the art is presented. In Sec. 3, the numerical quantities that encode the strip visual content are described. In Sec. 4, the algorithm for grouping the strips into pages is introduced, and in Sec. 5, the experimental results are reported. Paper 07047R received Mar. 29, 2007; revised manuscript received Aug. 14, 2007; accepted for publication Sep. 12, 2007; published online Apr. 2, 2008. This paper is a revision of a paper presented at the SPIE conference on Image Processing: Algorithms and Systems, Neural Networks, and Machine Learning, January 2006, San Jose, California. The paper presented there appears 共unrefereed兲 in SPIE Proceedings Vol. 6064. 1017-9909/2008/17共1兲/013008/13/$25.00 © 2008 SPIE and IS&T.
Journal of Electronic Imaging
2
Previous Work
In the scientific literature only a few works examine the recovery of shredded documents. In the contribution of Brassil,2 the appearance of the shredded remnants has been described. However, the work does not concern the reconstruction from the shredded remnants, but it analyzes the possibility of recognizing the used shredder device on the base of the profile of the cut. The work of De Smet, Bock, and Philips3 addresses the reconstruction of shredded printed documents: first, the blank strips are automatically identified and removed, then for each strip the similarity to the other 共N − 1兲 strips is calculated using three different features in sequence, in such a way to successively reduce the search space. The first feature is related to the positions of the top and bottom margins of text contained in the strips: only the strips with similar margin feature values 共a threshold is defined兲 are retained. Then the position of the lines of text is extracted and the first 10% of the most similar strips are retained 共with a minimum of ten strips兲. Finally, the RGB color on the strip border is compared using the Euclidean distance 共either in a 1-pixel width or averaging in a 3 ⫻ 3 or 5 ⫻ 5 neighborhood兲. The most similar strip is virtually glued to the starting one, and the process is iterated until the page is reconstructed. Glassner4 also addresses the reconstruction of shredded documents. He defines several features, all based on the pixel RGB or gray values on the two vertical strip borders. For matching the pieces, he utilizes a greedy algorithm 共highest confidence first兲. Moreover, prior to the matching, printed strips are grouped into overlapping sets on the base of four features, and the matching is conducted on these sets to speed up the computation in cases of large datasets. The four features are one dimensional and the values limiting each bin 共set兲 are empirically defined for each feature individually. The idea of the grouping 共called sorting in the paper兲 is the same idea motivating the clustering operation we propose. However, in our case, the clustering is unsupervised 共the user does not need to define the bins by hand兲, and it is able to work with multiple features at the same time. Although the literature is poor in the specific field of shredded document reconstruction, related fields exist. These are the jigsaw puzzle assembly and the reconstruc-
013008-1
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering…
Fig. 1 Strip samples: from left to right, strips 1 and 2 come from the same page, like 3 and 4, and 5 and 6.
tion from fragments of works of art. Computer vision methods for the solution of jigsaw puzzles have been proposed since 1963.5 Recently, jigsaw puzzle approaches have been adopted in the field of archaeology and art conservation, for Journal of Electronic Imaging
the computer-aided reconstruction of 2- and 3-D fragmented objects.6–8 Most of the solutions proposed for the assembly of jigsaw puzzles define a model for the piece contour, and perform the matching based on the contour shape.5–12 In some works, this problem is called “apictorial jigsaw puzzle.” Moreover, there are some papers that address the problem of reconstruction of documents that have been torn by hand.13,14 The problem is very similar to the 2-D assembly of works of art, because the piece shape is irregular. Finally, the works of Memon and Pal15 and Shanmugasundaram and Memon16 deal with the reassembling of fragmented computer files, containing respectively images15 or text.16 In the case of shredded document reconstruction, where all the puzzle pieces have approximately the same rectangular shape, the contour shape does not provide the necessary information for the matching. Matching based on the content of the piece should thus be investigated. Color information, together with contour shape information, is used in Refs. 17 and 18 to improve the automatic puzzle solver. Toyama et al.19 study the problem of reconstructing jigsaw puzzles with rectangular pieces looking at the black and white sequence along the contour of each piece. Sagiroglu and Ercil20 propose a reconstruction of 2-D fragments based on inpainting and texture synthesis techniques: these techniques are used for guessing the possible continuation to a piece border, and the piece that has the highest correlation to the guessed continuation is selected to continue building the puzzle. In Shanmugasundaram and Memon,16 the feature for determining the similarity among fragmented file pieces is exactly the pixel content. The problem is very similar to the shredded document reconstruction: in the latter, the visual content information on the border of the strip 共instead of at the end of the file fragment兲 should be analyzed to find the matches, and pieces of different pages 共instead of files兲 are mixed. However, in shredded document reconstruction, the problem is complicated by the fact that noise is introduced on the strip border by the digitalization or by the shredder device itself. When considering more general features, such as page layout and handwriting style, the algorithms developed in the document analysis field cannot simply be applied to the strips. For example, the problem of finding the positions of text lines has been addressed in the document layout analysis field.21 In general, the problem is more difficult in the shredded remnants case, because in the remnants not the whole line but just some characters of a line are present. Moreover, the strips can be torn or folded after disposal, and therefore the paper background can be very noisy. These are the reasons why the methodology for extracting line positions proposed in Ref. 21 can hardly be applied to the shredded remnant case. Concerning the handwriting style description, the studies done in the fields of handwriting recognition, writer identification, and handwriting classification can help in choosing suitable features. However, we must recall that a remnant usually does not contain entire words, but at most a couple of characters for each line of text. This is very poor information for a good description of the writing style. Moreover, while in the case of printed text the layout of the remnants is expected to be quite the same for the remnants that were adjacent in the original
013008-2
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering…
document, it is more difficult to have such a regular arrangement in handwritten notebook remnants, especially with nonsquared paper. The problem of assembling more than one puzzle at once 共as is the case of shredded document reconstruction of many pages whose strips have been mixed兲 is considered by some authors, in particular the ones involving works of art reconstruction. Leitao and Stolfi7 present experiments on five intermixed works of art, using a local matching strategy. The same happens in Papaodysseus et al.,6 but also in this case the matching is local, with user interaction. Wolfson et al.22 present results on two intermixed puzzles, but the assembly strategy they use does not differ from the one-puzzle one. The largest dataset reconstructed by Memon and Pal15 contained 24 images and 264 fragments. In this case, the assembly strategy for the intermixed puzzle starts from the first fragment of each file 共identified by the header兲 and reconstructs the N 共with N known from the total number of pieces containing the header兲 puzzles in parallel or in a sequence. In conclusion, none of these authors tackle the problem of an intermixed puzzle with a strategy that differs significantly from the one-puzzle solution. This work describes a number of features specially designed to characterize the strip’s visual content. Moreover, it proposes a computer-aided solution for the shredded document reconstruction problem, which includes an approach for dealing with the intermixed puzzle problem. A clustering algorithm groups the pieces belonging originally to different pages. The matching is then performed in the subsets with the help of a human operator, who iteratively picks up a strip, queries the database as in an image retrieval task,23 and chooses the correct matching strips among the most similar ones returned by the computer. 3 Describing the Pieces A number of low-level features are presented in the following. They are used to describe the content of printed or notebook paper or both, as summarized in Table 1. They are specifically designed to take into account the real-world shreds peculiarities.1,24 3.1 Line Spacing The line spacing feature LS is useful to recognize when two or more remnants come from different documents. We developed an algorithm that starts from the the luminance values of the remnant image. The horizontal projection profile25 of the strip region is calculated. For each row x, the horizontal projection 共see an example in Fig. 2兲 is defined as: Q
1 x p共x兲 = 兺 F共j,x兲, Qx j=1
共1兲
where F共j , x兲 is the gray value of the pixel in position 共j , x兲 in the image, and Qx is the strip width in pixels for row x. The projection is a 1-D periodic signal, whose period is the line spacing. The periodicity is extracted using the discrete Fourier transform 共DFT兲 of the projection P共f兲. See Fig. 3 for some examples. The first major peak detected in the Journal of Electronic Imaging
Table 1 List of features presented in the work and their suitability for printed and handwritten text. Feature
Description
Printed
Handwritten
LS
Line spacing
X
NLINE
Number of lines
X
FIRSTL
First line position
X
LASTL
Last line position
X
MA
Presence of a marker
X
HORIZ
Horizontal edge energy
X
X
VERT
Vertical edge energy
X
X
DIAGO1
Diagonal edge energy
X
X
DIAGO2
Diagonal edge energy
X
X
ISOTR
Isotropic edge energy
X
X
FGY
Text ink color
X
X
FGCb
Text ink color
X
X
FGCr
Text ink color
X
X
BGY
Paper color
X
X
BGCb
Paper color
X
X
BGCr
Paper color
X
X
SP
Squared paper
X
transformed domain corresponds to the fundamental frequency, and is equal to 1 / LS. The amplitude and frequency of the successive harmonics of P共f兲 are related to the font type, since they characterize the gray-level transitions between a white space and a text line. In Fig. 3, the harmonics from the second to the seventh can be recognized. These harmonics could be extracted as the local maxima positions in the signal 兩P共f兲兩. 3.2 Document Layout In our approach, we use four features for describing the document layout: number of lines of text, position of the first line of text, position of the last line, and line positions. The position of the first line of text is usually dependent on the way the text is formatted; it should give some useful information in distinguishing different documents, but not different pages of the same document. The position of the last line of text is important to find out pages in which a section finishes, or to distinguish among different text formats. The number of lines of text can be an important feature if paragraph interruptions are present in a page, or to distinguish between remnants positioned in the left or in the right part of the same page. Indeed, for the remnants corresponding to the right part of a page of text, it is very likely that there are missing lines, because the text may end before reaching the right margin of the page. In contiguous
013008-3
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering…
Fig. 2 A strip 共bottom兲, its horizontal projection profile p共x兲 共top兲, and the corresponding signal vT共x兲 in which each peak represents a line 共center兲.
remnants, going from left to right, the number of text lines of the remnants is likely to decrease, with possible exceptions in the very left remnants if indentation or subheadings are present. The opposite happens for documents in languages with right to left writing 共e.g., Arabic or Hebrew兲.
The algorithm we propose relies on the previous estimation of the line spacing 共Sec. 3.1兲. Since the latter has been estimated, taking into account the information of the whole remnant via the DFT, the method is robust to the paper background noise.
Fig. 3 Magnitude of the DFT of the remnants in Fig. 1 in 关0 , / 2兴. The x axis represents the frequency, while the y axis is the DFT amplitude. From top to bottom: the first two signals have a fundamental frequency of 0.175 rad, and the corresponding strips 共Fig. 1兲 have single line spacing and font size 12; the third and fourth signals have a fundamental frequency of 0.202 rad, and the line spacing 共Fig. 1兲 is single with font size 10; the last two signals have fundamental frequency of 0.199 rad, and the line spacing 共Fig. 1兲 was single with a font size of 10, as the previous ones, but the font style was Courier instead of Times New Roman. Journal of Electronic Imaging
013008-4
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering…
To obtain the line positions, an approach similar to the concept of short-time Fourier transform 共STFT兲 is used. A sinusoidal wave is built with frequency equal to the line spacing LS, and length equal to one period. This is intended to model the appearance of one line of text and its surrounding background. Defining f 0 = 1 / LS, the wave is: w共t兲 =
再
sin共2 f 0t兲 if 0 ⬍ t ⬍ 2 otherwise
0
冎
.
共2兲
A signal v共x兲 is built for the whole horizontal projection profile p共x兲 共Eq. 共1兲兲, by calculating for each x the correlation between the sinusoidal wave and the vertical projection: v共x兲 =
兺 w共t − x兲p共t兲.
共3兲
A threshold is then applied to v共x兲, in such a way to consider only large positive peaks 共corresponding to the lines of text兲. The threshold value Th is experimentally determined. In general, the same threshold can be used for all the remnants if they have been acquired by the same scanner and with the same scanner setting. In this way, a new function vT is derived: vT共x兲 =
再
v共x兲 if v共x兲 ⬎ Th
0
otherwise
冎
.
共4兲
An example of vT共x兲 is shown in Fig. 2. The local maxima of vT共x兲 are extracted and saved in the vector ln. The positions are coded in a differential way: ln represents the difference in pixels between the position of the n’th line and the position of the 共n − 1兲 line, where the line number 0 is considered having position equal to 0. The differential coding is used in such a way that the line position value depends only on the previous line position, and not on the absolute position of the line in the strip, and there is no distinction between first and last lines in the computation of the distance among strips. Values of ln larger than the line spacing LS represent the start of a new paragraph. The output of the algorithm is represented by the array: 共5兲
L = l1,l2, . . . ,lN . The array elements li follow this relation: N
li 艋 共height − 1兲, 兺 i=1
共6兲
where height is the image height in pixels. The following other features can be extracted from L: • number of lines of text NLINE N LINE = L,
共7兲
where represents the cardinality of a set; • relative position of the first line of text FIRSTL FIRST L =
l1 ; height
Journal of Electronic Imaging
共8兲
• relative position of the last line of text lastL N
兺 li
LAST L =
i=1
height
.
共9兲
NLINE, FIRSTL, and LASTL are scalar numbers. LASTL and FIRSTL have values between 0 and 1. Since the text line positions are obtained by exploiting the periodicity of the line spacing, the algorithm is robust to cases in which part of the text is bold, underlined, or if italics is used. Observe for example Fig. 2: in the strip, the fifth line from the right has a bold font face, and the corresponding peak of vT共x兲 is larger than the other one, which means it is even easier to identify that line. The algorithm is also alphabet-independent, as long as the writing direction is orthogonal to the strip cut. 3.3 Presence of a Marker There is the possibility that a text marker was used on the documents we need to reconstruct. If this is the case, this is very useful information for finding matching remnants. The marker can be identified looking at the chrominance histograms of the remnants, for example in the YCbCr color space. If there is no marker, then the Cb and Cr channels present a single-mode, approximately Gaussian histogram 关see for example Fig. 4共a兲兴. This is the color of the background, since the ink amount is usually low and does not affect the histogram appearance. If the marker is present, the histogram will be bimodal, as for example in Fig. 4共b兲. The bimodality is detected by an index MA, MA = 兩mode共C兲 − mean共C兲兩,
共10兲
where C contains the pixel values of the remnant in either the Cb or Cr channel, mode共C兲 is the most frequent value in C, and mean共C兲 represents the mean value in the considered color channel. The index has low values, close to 0, if the histogram is unimodal, and high values if it is bimodal, and thus large areas are marked. Note that the marker index can also be used in the case of presence of stains on the paper that can be caused by a bad remnant preservation. 3.4 Text Edge Energy In the case of either printed or handwritten text, the text appearance is extracted by descriptors of edges related to the writing. The handwriting style in particular can have preferred directions of strikes. General texture descriptors,26 such as the gray-level cooccurrence matrix, the Fourier descriptors, and the invariant moments, are not able to capture the directionality of the writing, since they focus on more general properties such as coarseness and contrast. The choice goes thus on edge descriptors. The descriptor we designed is calculated by applying the Robinson operator25 in four directions, horizontal 共feature HORIZ兲, vertical 共feature VERT兲, two diagonals 共feature DIAGO1 and DIAGO2兲, and in one isotropic feature 共ISOTR兲, on the gray value strip image. The energy of the edges is evaluated into sub-blocks as the average of the
013008-5
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering…
Fig. 4 Histogram of the Cb channel of a strip 共a兲 without marker and 共b兲 one with marker.
squared edge values. The sub-blocks are arranged vertically. For one strip, the overall edge energy in one given direction is the average of the sub-blocks energy in that direction. 3.5 Paper/Ink Color Color features have mainly the aim of describing the kind of paper and the color of the ink. For the description of the ink color, a segmentation needs to be performed to divide the textual part from the background. Otsu thresholding27 is used for this. The ink color is a distinctive feature of the pen or pencil used for writing. There are a number of different color descriptors proposed in the literature.28,29 A suitable color descriptor for our problem does not need to include spatial information of the color distribution, because we just want to extract the overall paper color and the overall ink color 共we suppose that the same pen/pencil has been used on the same page兲. For this reason, descriptors such as color layout28 are not suitable. General color histograms can be used. However, the dimensionality of a color histogram feature is too high for good classification. Either the MPEG-7 scalable color28 could be used, where the histogram information is reduced to a small number of coefficients, or descriptors of the dominant color. Indeed, since the histogram represents either the background 共paper兲 or the foreground 共ink兲, it has approximately one dominant color. The descriptor we implemented is obtained by defining the paper 共ink兲 dominant color as the most frequent Y, Cb, Cr values in the paper 共ink兲 region. The resulting features are called FGY, FGCb, FGCr for the ink color 共foreground兲, and BGY, BGCb, BGCr for the paper color 共background兲. To capture the information on the paper pulp type, color correlograms30 could additionally be used on the segmented background, since they are able to describe the color-texture appearance. 3.6 Squared Paper A first important distinction that helps the reconstruction is between squared and blank notebook paper. The detection of squares and lines in the remnants is not trivial, due to the low contrast of the printing. The feature24 that we resort to Journal of Electronic Imaging
for this detection relies on the Hough transform.26 According to the transform,26 each pixel having coordinates 共x , y兲 in the original image is transformed into a curve in the Hough space 共 , 兲. A straight line in the original image corresponds to a set of sinusoidal curves in the Hough space 共one for each pixel that the straight line is made of兲 intersecting in the same point 共m , m兲. The detection of straight lines in the original image becomes the detection of peaks in the Hough 2-D histogram. A pattern of squares, positioned with any orientation in the image, produces in the Hough space some bright points with coordinate having just two possible values, 1 and 2, where abs共1 − 2兲 = 90. This happens of course because a squared pattern is nothing less than a set of straight lines orthogonal to each other. The other pixels in the original image that do not belong to any straight line pattern produce in the Hough space some curves that do not intersect with each other or intersect in random positions. This means they are sparse and they create low and dispersed values in the Hough accumulating matrix. In our detection process, a nonlinear operator is applied to the transformed data to separate two different contributions. If the Hough transform is performed from 1 to M deg 共for example 1 = 0 and M = 179兲 with a step equal to 1 deg, then the square pattern can be detected in the Hough transformed space H1 looking for the maxima values in the array g: g共i兲 = 兺 H1共i, j兲 + 兺 H1共i+90, j兲, j
j
共11兲
where i belongs to 兵1 , . . . , M/2其. A square pattern or a line pattern in the original image exists if and only if there is a strong peak in g共i兲. The square pattern is thus detected by means of an index, defined as: SP = max共g兲.
共12兲
In the case of the remnants of shredded documents, their typical shape could yield misleading results if the Hough transform is performed on the whole image at once. Indeed,
013008-6
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering…
since the image width is very small with respect to the image height, in the Hough space the contributions of in the horizontal direction are much less dispersed than the contributions of in the vertical direction, because in the horizontal direction just a few values are possible. For this reason the Hough transform is evaluated on image subblocks approximately square in shape in the vertical remnant direction. The typical size of the blocks is the strip width. The blocks do not overlap and they are positioned along the strip length. The contribution of each block gk共i兲 is averaged: g T共 i兲 =
1 兺 gk共i兲, NB k
共13兲
where NB is the total number of sub-blocks. The index SP is then obtained as in Eq. 共12兲 by substituting g with gT. 4 Grouping the Pages 4.1 Motivation The clustering approach used to group the pages is similar to the methodology a human being uses for reassembling a jigsaw puzzle. The pieces with a similar content, usually the color, are grouped together, and matches are first searched within these groups. Although the methodology is described here for strip-cut shredded document reconstruction, it could be also applied to cross-cut shredded document reconstruction, jigsaw puzzle assembly, and reconstruction from fragments. The computer-aided reconstruction can be modeled as an image retrieval process. A strip is selected by the human operator and it is used as a query for the image retrieval system. Once the strips most similar to the query are retrieved, the human operator selects among them the two correctly matching the query, if present, and virtually glues them. Since the retrieval is done on-line, its computation needs to be as fast as possible. An off-line grouping of the strips into clusters permits speeding up the on-line search, because the similar strips are searched within the cluster the query strip belongs to and not within the whole dataset of strips. In the case of perfect clustering 共the clusters correspond to the original pages兲, if the total number of strips is N and the number of strips in a cluster is K, then the complexity of the retrieval of one strip reduces from O共N兲 to O共K兲, which is a constant. If we suppose that the human operator needs to use as query all the strips for the complete reconstruction 共in practice he will need to use only half of the strips, because the other half will be already matched兲, then the overall complexity of the on-line part of the reconstruction is O共N2兲 without clustering and O共N兲 with clustering 关which comes from O共NK兲, with K being a constant兲. The same reasoning applies to the distances calculation used in the clustering. In the case where the hypothesis of perfect clustering does not hold, K is the average number of strips per cluster with, in general, K Ⰶ N, and K does not depend on N, so the same considerations made before are still valid for the first retrieval operation. Tables 2 and 3 show the computational complexity of each step in computer-aided reconstruction with and without clustering, respectively. Although the overall complexity is higher in the case of using the clustering, the compuJournal of Electronic Imaging
Table 2 Computer-aided reconstruction with clustering 共for the clustering it is assumed that the hierarchical algorithm described in Sec. 4 is used兲: estimated computational complexity. Step
Online/Offline
Complexity
Extract features for clustering
Offline
O共N兲
Clustering
Offline
O共N3兲
Extract features for retrieval
Offline
O共N兲
Distances calculation
Offline
O共N兲
Reconstruction by retrieval
Online
O共N兲
tational cost of the on-line part is reduced by a factor of N. Regarding the off-line part, which is automatic and does not require user feedback, the computational efficiency is not a strict requirement. The procedure for the computer-aided reconstruction is iterative. When the two correctly matched strips to a query strip are found, this strip is put in the set of “matched.” After all the “nonmatched” strips have been used as query, the remaining strips that do not belong to the “matched” set are used as the initial dataset for a second iteration. The user may choose to perform an additional clustering step or go directly to the retrieval step without any clustering. The procedure is repeated iteratively until the document is reconstructed. In particular, in the case of nonperfect clustering, the iterations successive to the first will be necessary, since there will be strips belonging to the same page put in different clusters. 4.2 How Many Clusters? The problem of finding the best number of clusters is usually referred to as the cluster validity problem.31 In the ideal shredded reconstruction problem, the choice of the number of clusters is trivial. Indeed, if the dataset contains N strips, and we suppose there are no missing pieces and we know the exact number of pieces n in which a page is cut 共i.e., the shredding device used is known兲, then the number of clusters we have to look for is equal to N / n if we want to group together the strips originally belonging to the same page. In the nonideal case, however, the following situations make the number of clusters vary. 1. Some of the features are likely to take different values when going from strips in the left portion of a Table 3 Computer-aided reconstruction without clustering: estimated computational complexity. Step
Online/Offline
Complexity
Extract features for retrieval
Offline
O共N兲
Distances calculation
Offline
O共N2兲
Reconstruction by retrieval
Online
O共N2兲
013008-7
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering…
page to those in the right portion; for example, the number of lines of text NLINE is usually decreasing. When considering these features, it may happen that the left- and right-side page strips are put in different clusters. 2. There are outlier strips, in particular close to the page margins: they are likely to be classified as singleton clusters. 3. There may arise the case of missing strips. In that case, the number of strips that form one page is smaller. 4. The number of strip/page n depends on the shredder used and on the paper size. In the case the former is unknown, but the page size is known, n can be calculated by measuring the strip width. The problem is more difficult if more than one shredder device has been used 共unless we measure the width of all the strips兲. The strip width, according to Brassil,2 can vary from 1 / 8 to 5 / 16 in. for common shredders. Note that the first three points would cause the expected number of clusters to increase with respect to the ideal case of N / n, while for the last point the problem is that n is not known or not unique. For these reasons, we prefer to choose the number of clusters according to some measures on the clustering results, instead of guessing a precise value. We are looking for natural clusters. In general, we would like to have compact clusters containing strips belonging to the same page. Since we want to find a compromise between the number of elements/cluster 共reasonably high兲 and compactness of the cluster, we define32 an interval where the optimal number of clusters has to be found. The interval is: T = 关k1共N/n兲,k2共N/n兲兴,
共14兲
where k1 and k2 are two constants 共for example, if we suppose that we have two to three clusters/page, then it would be k1 = 2 and k2 = 3兲. k1 varies from 1 to 2 共1 if a large variability in the dataset strip appearance is observed, 2 if strips are very similar to avoid the forming of one big cluster兲, and k2 from 2 to 5 or more 共2 for large variability, 5 for small variability兲. In the case n is not known, it can be replaced by its estimate ˜n obtained, for example, by taking the average of some sample strip width. Note that although the clustering is run only using the nonblank strips, the evaluation of the best number of clusters interval is done taking into account all the strips 共i.e., the total number of strips in the dataset兲. This allows us to avoid the introduction of further errors due to the fact that the amount of blank strips can vary according to the document type and style. 4.3 Algorithm The choice of the clustering algorithm32 is driven by the fact that we do not know a priori the number of clusters to be searched for 共see Sec. 4.2兲. We use an agglomerative hierarchical approach with step-wise optimization.32 It is an iterative procedure, which starts from each element being a single cluster; at each step, it fuses two clusters into one until all the elements are in one cluster. In this way, a tree called a dendrogram is built: the root represents a cluster Journal of Electronic Imaging
containing all the elements; splitting of branches corresponds to the splitting of one cluster in two different clusters; and the leaves are the original dataset elements. Once the tree is built, measures can be adopted to decide where to cut the tree and thus form the clusters. In our case, an interval of possible values for the expected number of clusters is defined, and we choose the best value 共according to measures on the hierarchical tree兲 in this interval. The definition of the interval is explained in Sec 4.2. To build the tree, two distances have to be defined. The first one is needed to calculate the distance between pieces, the second to calculate the distance between clusters 共to choose which pair to merge at each step兲. The first one takes as input the features values. If piece a and piece b have features Fa = f a1 , . . . , f aM and Fb = f b1 , . . . , f bM , respectively, then the distance between a and b is: d共a,b兲 = l共储Fa − Fb储兲,
共15兲
where l共 兲 is a distance measure, for example the Euclidean distance, the Manhattan distance, or the Mahalanobis distance. The features related to the document layout 共since they are length values in pixels兲 suggest the use of the Euclidean distance. Mahalanobis distance is to be preferred in the case where the features are strongly correlated.33 The choice of the features that could yield a good clustering depends on the specific dataset considered. For example, the background color feature will work well only if the original pages differ in the type of paper. If we do not have enough a-priori knowledge about the dataset characteristics, then an automatic criterion must be defined to choose the most relevant ones. The literature in this sense is wide. The criterion we adopt is very simple and is based on a correlation measure. If two features show to have a high correlation in the specific dataset examined, then only one of these features is considered for the clustering. Indeed, the other feature does not bring more information for characterizing the strips, and it could instead reduce the algorithm performance. Concerning the distance between two clusters, there are different methods to measure it, for example single linkage, complete linkage, average linkage, and Ward distance.31 We use the Ward or minimum variance method, because it creates clusters that are dense and well separated one from the other.33 In general, the Ward method is associated with the Euclidean distance for l共 兲 in Eq. 共15兲.33 The Ward method aims to merge at each step the two clusters that minimize the overall squared error of the clustering. The squared error for cluster k is:34 nk
e2k
M
共k兲 2 = 兺 兺 关f 共k兲 ij − m j 兴 ,
共16兲
i=1 j=1
where f 共k兲 ij is the value for feature j of strip i, nk is the number of strips in cluster k, M is the dimensionality of the feature space, and m共k兲 j is the centroid of cluster k, calculated as:
013008-8
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering… Table 4 Ground truth classes for the real-world dataset.
n
m共k兲 j
1 k = 兺 f 共k兲 . nk i=1 ij
共17兲 Class
The squared error for the entire clustering is: K
EK2 = 兺 e2k ,
共18兲
k=1
where K is the number of clusters at the current step. EK2 is minimum 共equal to 0兲 if each strip is one cluster. According to the Ward method, the pair of clusters p and q that minimizes ⌬E2pq, i.e., the change in EK caused by the merger of clusters p and q into cluster t, is chosen. Once all the iterations have been accomplished, the selection of the number of clusters is conducted in the interval defined in Sec. 4.2. The processing of merging clusters can be visualized in the dendrogram tree. The leaves are the dataset elements, while the root represents the cluster that contains all the elements. Branches go from the leaves to the root, successively merging from two into one. Each node 共connection of three branches兲 has a height hn corresponding to the effort the algorithm made in merging two clusters into one at step n. The leaves have height 0 and the height is increasing, going toward the root. In our case, hn = EK2 of Eq. 共18兲, with K = N − n 共indeed, at step 1 two elements are merged and we have K = N − 1 clusters, at step n, K = 1 cluster is formed兲. The heights hn are nondecreasing, with n = 1 , . . . , N − 1. The ratio between two successive values of the heights is defined as: sn =
hn+1 , hn
共19兲
with n = 1 , . . . , N − 2. If sn is high for a given n, this means the two clusters that have been merged at step 共n + 1兲 were likely to remain two separate clusters, thus the number of clusters should better stay at K = N − n. Since we are looking for natural clusters in the interval T defined in Eq. 共14兲, we have to look for local maxima of sn in the interval with K = N − n belonging to T. The optimal number of clusters is thus topt with: s共N−topt兲 = max s共N−t兲 . t苸T
共20兲
5 Experimental Results Experiments have been conducted to test the proposed computer-aided reconstruction. The software R 共see http:// www.r-project.org兲 has been used in the experiments for performing the hierarchical clustering. In Sec. 5.1, experiments on the discrimination power of the low-level features is shown using a real-world dataset. In Sec. 5.2, the advantage of performing the clustering is shown using a virtually shredded dataset. 5.1 Real-World Dataset With the real-world dataset, we want to test the discrimination power of the low-level features described in Sec. 3. The distribution of the strips in the feature space should Journal of Electronic Imaging
Freq
A
20
B
52
C
108
D
24
E
74
F
24
G
175
H
13
I
29
L
15
All
534
show that strips belonging to different documents have different feature values. The ground truth available is the classification of the strips into document classes, as shown in Table 4. There are ten different shredded documents in the dataset. Each document has been labeled by a letter. In the following, we refer to the ground truth as the classes. Some of the documents originally contained more than one page 共for example C and G兲. Several strips are missing. The set of features used in this experiment is made of the following numerical quantities: line spacing 共LS兲, number of lines of text 共NLINE兲, position of last line of text 共LASTL兲, position of first line of text 共FIRSTL兲, text edge energy in the vertical and horizontal direction 共VERT, HORIZ兲, dominant paper color in the YCbCr space 共BGY, BGCb, BGCr兲, and dominant ink color in the YCbCr space 共FGY, FGCb, FGCr兲. The dimensionality of the feature space is thus 12. To analyze the class separability, in Fig. 5 the average feature values for the ten classes is shown. With the exception of class F and class G, which contained documents with a very similar layout, the other classes are well separated. 5.2 Virtual Dataset The considered dataset contains ten pages 共the classes兲. The pages have been acquired by means of a scanner, and then virtually shredded, each page producing 34 strips. The dataset has thus N = 340 strips. The shredding process has been done virtually for the sake of simplicity. The difference with respect to a real shredding is that the strip borders are sharp cut and the border is a straight line instead of being slightly curved. Moreover, no border noise is present. Note, however, that the features used in the experiments have been defined and experimented on real-world shredded strips and have proved to be robust enough to the nonideal cutting process1 共see also Sec. 5.1兲. Since we want to group together strips belonging originally to the same page
013008-9
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering… Table 5 Characteristics of the virtually shredded dataset. Page number
Fig. 5 Feature distribution of the ground truth classes in the realworld dataset. Given a feature and a class, the size of the area corresponding to that feature in the figure represents the mean value of that feature 共compared to the maximum value in the dataset兲.
and we are not using the information about the strip border, we believe the experiments to be meaningful also with virtually shredded pieces. The characteristics of the different pages are shown in Table 5. In some pages, there are handwritten notes on the printed text. The notes have been made by using either a blue or a green pen. Some pages 共for example, pages 7, 8, and 9兲 are different pages of the same document, thus they have the same font style. The first two pages are printed e-mails, and they contain the footnote and the header. The features for the description of the strip visual content used in the experiments are nine: line spacing LS, number of line of text NLINE, position of the last line of text LASTL, position of the first line of text FIRSTL, edge energy in the vertical direction VERT, edge energy in the horizontal direction HORIZ, and ink color FGY, FGCb, FGCr. Other features were not selected because the documents did not have some characteristics, for example no marker was present and the paper type was the same for all the pages. In general, during the acquisition of the strips by the scanner, it is normal to notice some characteristics of the strips under analysis and to choose the features accordingly. The blank strips have been identified by the feature NLINE= 0, and they have been excluded from further Journal of Electronic Imaging
Document class
Document type
Appearance description
1
A
E-mail 1
Printed+ blue ink notes
2
A
E-mail 2
Printed+ green ink notes
3
B
Report 1
Printed+ blue ink notes
4
B
Report 1
Printed+ green ink notes
5
B
Report 1 共Bibliography兲
Printed
6
C
Paper
Printed
7
D
Thesis
Printed
8
D
Thesis
Printed
9
D
Thesis
Printed
10
E
Report 2
Printed+ blue ink notes
analysis. 82 blank strips were selected in this way. The classification result is good, since they are all blank, except three of them, which present some small part of font. The other 340− 82= 258 strips, classified as nonblank by NLINE⫽ 0, are all real nonblank strips. To perform the clustering, a feature selection is made. The features to be considered for the clustering are chosen according to the correlation coefficient measure 共see Sec. 4兲. If two features have a correlation value higher than 0.9,
Fig. 6 Hierarchical tree 共dendrogram兲 generated for the virtually shredded dataset. See Sec. 4.3 for details.
013008-10
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering… Table 6 Percentage of strips for which the first five most similar retrieved strips contain 0 matching strips, 1 matching strip, or both the 2 matching ones. Results are compared for the retrieval inside the whole dataset, inside the cluster, and inside the class.
Table 7 Retrieval with or without clustering: average positions in the retrieval of the first and second correctly matching strips. Strip number
No clustering
Clustering
0 matches
1 match
2 matches
Total
First strip
15
4
Search in the whole dataset
29%
41%
30%
100%
Second strip
46
11
Search inside the cluster
20%
45%
34%
100%
Search inside the class
17%
48%
35%
100%
only one is kept. The following features are thus selected: LS, NLINE, LASTL, FIRSTL, FGY, FGCr. The data are standardized before performing the clustering. The hierarchical clustering described in Sec. 4 is conducted, and the resulting dendrogram is shown in Fig. 6. k1 and k2 关Eq. 共14兲兴 values have been set to 1.5 and 3, respectively. n is known and it is equal to 34. The best number of clusters is selected according to Eq. 共14兲 in the interval 关15, 30兴. According to the measure in Eq. 共20兲, the index is maximum for topt = 23. The errors in the obtained clusters 共strips belonging to different pages put in the same clusters兲 are mostly for pages 8 and for pages 3 and 4, since most of the strips of these pairs of pages tend to be in the same cluster. Indeed, with the features used, it is very difficult to distinguish between pages 8 and 9 共see Fig. 7兲. Other features would have to be adopted, as the line positions, or a new feature can be defined, such as the new paragraph position. The strips with only a few handwritten notes tend to be in one cluster. Indeed, they are very different from the other strips of the same page. Experiments of retrieval have been conducted using the feature position of the lines of text L. Other features whose value varies inside a page 共not described in the present work兲 could have been used as well. For each strip, the Q = 5 most similar strips are retrieved and the presence of the two matching ones in these five is evaluated. The retrieval is conducted in three different cases: searching in the whole
database, searching inside the cluster the strip belongs to, or searching inside the class. The Euclidean distance has been used for measuring the similarity, because the L array values represent length in pixels and they are all in the same range. The results are shown in Table 6. In the case where no clustering is performed, the feature L is able to select a subset of five strips that contains at least a matching strip in 71% of the cases. If the clustering is first performed, this value increases to 80%. In the case of using the ground truth classification into pages, the value is 83%. The results can also be displayed as precision-recall measures, as usually done in image retrieval evaluation23 tasks. In Table 7, the retrieval using perfect clustering 共the clusters corresponding to the classes兲 is compared with the one obtained without clustering. The retrieval returns an ordering of the strips from the most similar to the least similar to a given one, the query. The most similar are displayed in the monitor closer to the query. At this point, it is the human operator that performs the matching, i.e., chooses, among the retrieved strip, the strips that were adjacent to the query in the original page. The results in Table 7, where the average positions of the two correctly matching strips after the retrieval are reported, show that the clustering eases the human operator task, by presenting the actual adjacent strips in a lower position in the retrieval 共closer to the query兲. Although the human time for matching will vary according to the person who is performing the matching, the results show that the clustering significantly improves the retrieval. The processing times on a Pentium 4 CPU at 2.8 GHz, 1-GB RAM are reported in Tables 8 and 9. Basically, as it has been explained in Sec. 1, it is important that the on-line part is fast, thus the two values for the task of distance
Fig. 7 From left to right: pages 3, 7, 8, and 9 of the virtually shredded dataset. Journal of Electronic Imaging
013008-11
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering… Table 8 Time of computation for the tasks involved in the computeraided reconstruction using clustering 共perfect clustering兲. Task
Time
Extract features for clustering 共LS, NLINE, LASTL, FIRSTL, L兲
1 min
Extract features for clustering 共VERT, HORIZ兲
3 min
Extract features for clustering 共FGY, FGCb, FGCr兲
3.4 min
Distance calculation for clustering
0.02 s
Clustering
0.11 s
Extract features for retrieval 共included in features for clustering L兲 Distance calculation and retrieval 共total兲
Acknowledgments This work has been supported by Andor s.r.l., Trieste, Italy. The authors would like to thank Alessandra Zacchigna and Gabriella Schoier for their precious suggestions and the reviewers for their accurate analysis of a previous version of this contribution. References
共included above兲 0.4 s
calculation and retrieval need to be compared: the clustering reduces this time from 4.2 to 0.4 s. The reduction will be even more significant in the case where a larger dataset is considered. Concerning the off-line tasks performed for the clustering, the total computational time 共from Table 8兲 is less than 7.5 min. 6 Conclusions We present a set of algorithms for the computer-aided reconstruction of strip-cut shredded documents. Numerical features are automatically extracted to describe the visual content of the strips. Strips originally belonging to the same page are grouped by a clustering algorithm. This allows us to speed up the on-line reconstruction performed by a human operator with the help of a computer. This strategy could possibly also be employed for assembling intermixed jigsaw puzzles or fragments of works of art. The online reconstruction has been modeled as an image retrieval process. Results both in terms of computational efficiency and efficacy are promising, and in the future, experiments on larger datasets will be conducted. Some of the presented techniques can also be used for reconstructing cross-cut shredded strips. Concerning the clustering part, it can be used without any modification. Concerning the features, the line spacing, the text edge energy, the paper/ink color, the squared paper, and the line positions can be extracted for cross-cut strips as well, and they are useful for the reconstruction. However, the docuTable 9 Time of computation for the tasks involved in the computeraided reconstruction without clustering. Task
Time
Extract features for retrieval 共L兲
1 min
Distance calculation and retrieval 共total兲
4.2 s
Journal of Electronic Imaging
ment layout features are less helpful, because they refer mainly to the overall page appearance. In general, the tools introduced in this work can be used for finding matching pieces in the horizontal direction, while for the vertical one, it would be necessary to include some additional features.
1. A. Ukovich and G. Ramponi, “System architecture for the digital recovery of shredded documents,” Proc. SPIE 5672, 1–11 共2005兲. 2. J. Brassil, “Tracing the source of a shredded document,” Tech. Rep. HP Labs 2002 Technical Reports 共2002兲. 3. P. D. Smet, J. D. Bock, and W. Philips, “Semiautomatic reconstruction of strip-shredded documents,” Proc. SPIE 5685, 239–248 共2005兲. 4. A. Glassner, “Putting the pieces together,” IEEE Comput. Graphics Appl. 22共3兲, 76–86 共2002兲. 5. H. Freeman and L. Garder, “Apictorial jigsaw puzzles: the computer solution of a problem in pattern recognition,” IEEE Trans. Electron. Comput. 13, 118–127 共1964兲. 6. C. Papaodysseus, T. Panagopoulos, M. Exarhos, C. Triantafillou, D. Fragoulis, and C. Doumas, “Contour-shape based reconstruction of fragmented, 1600 b.c. wall paintings,” IEEE Trans. Signal Process. 50共6兲, 1277–1288 共2002兲. 7. H. da Gama Leitao and J. Stolfi, “A multiscale method for the reassembly of two-dimensional fragmented objects,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 1239–1251 共2002兲. 8. G. Papaioannou, E. A. Karabassi, and T. Theoharis, “Virtual archaeologist: assembling the past,” IEEE Comput. Graphics Appl. 21共2兲, 53–59 共2001兲. 9. T. Altman, “Solving the jigsaw puzzle problem in linear time,” Appl. Arti. Intell. 3共4兲, 453–462 共1989兲. 10. R. Webster, P. LaFollette, and R. Stafford, “Isthmus critical points and solving jigsaw puzzles in computer vision,” IEEE Trans. Syst. Man Cybern. 22共5兲, 1271–1278 共1991兲. 11. D. Goldberg, C. Malon, and M. Bern, “A global approach to automatic solution of jigsaw puzzles,” in Proc. 18th Ann. ACM Symp. Computational Geometry, pp. 82–87 共2002兲. 12. G. Burdea and H. Wolfson, “Solving jigsaw puzzles by a robot,” Proc. IEEE Trans. Robotics Auto. 5共6兲, 752–764 共1989兲. 13. A. Biswas, P. Bhowmick, and B. Bhattacharya, “Reconstruction of torn documents using contour maps,” in Proc. IEEE Intl. Conf. Image Process. 2005 (ICIP ’05), III-517-20 共2005兲. 14. C. Solana, E. Justino, L. Oliveira, and Flavio Bortolozzi, “Document reconstruction based on feature matching,” in Proc. 18th Brazilian Symp. Computer Graph. Image Process. (SIBGRAPI 2005), pp. 163– 170 共2005兲. 15. N. Memon and A. Pal, “Automated reassembly of file fragmented images using greedy algorithms,” IEEE Trans. Image Process. 15共2兲, 385–393 共2006兲. 16. K. Shanmugasundaram and N. Memon, “Automatic reassembly of document fragments via context based statistical models,” in Proc. 19th Ann. Computer Security Appl. Conf., pp. 152–159 共2003兲. 17. D. Kosiba, P. Devaux, S. Balasubramanian, T. Gandhi, and R. Kasturi, “An automatic jigsaw puzzle solver,” in Proc. IAPR Intl. Conf. Patt. Recog. 1, 616–618 共1994兲. 18. M. Chung, M. Fleck, and D. Forsyth, “Jigsaw puzzle solver using shape and color,” in Proc. Intl. Conf. Signal Process. (ICSP ’98), pp. 877–880 共1998兲. 19. F. Toyama, Y. Fujiki, K. Shoji, and J. Miyamichi, “Assembly of puzzles using a genetic algorithm.,” in Proc. Intl. Conf. Patt. Recog. (ICPR), 4, 389–392 共2002兲. 20. M. Sagiroglu and A. Ercil, “A texture based matching approach for automated assembly of puzzles,” in Proc. Intl. Conf. Patt. Recog. (ICPR 2006) 3, 1036–1041 共2006兲. 21. B. Gatos, N. Papamarkos, and C. Chamzas, “Skew detection and text line position determination in digitized documents,” Pattern Recogn. 30共9兲, 1505–1519 共1997兲. 22. H. Wolfson, E. Schonberg, A. Kalvin, and Y. Lamdan, “Solving jigsaw puzzles by computer,” Ann. Operat. Res. 12共1兲, 51–64 共1988兲. 23. A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-based image retrieval at the end of the early years,” IEEE Trans. Pattern Anal. Mach. Intell. 22共12兲, 1349–1380 共2000兲.
013008-12
Jan–Mar 2008/Vol. 17(1)
Ukovich and Ramponi: Feature extraction and clustering… 24. A. Ukovich and G. Ramponi, “Features for the reconstruction of shredded notebook paper,” in IEEE Intl. Conf. Image Process. 05 (ICIP 05) 3, 93–96 共2005兲. 25. W. K. Pratt, Digital Image Processing, John Wiley and Sons, New York 共1991兲. 26. R. C. Gonzalez and R. E. Woods, Digital Image Processing, AddisonWesley, Reading, MA 共1992兲. 27. N. Otsu, “A threshold selection method from gray level histograms,” IEEE Trans. Syst. Man Cybern. 9, 62–66 共1979兲. 28. B. S. Manjunath, J. R. Ohm, V. V. Vasudevan, and A. Yamada, “Color and texture descriptors,” IEEE Trans. Circuits Syst. Video Technol. 11共6兲, 703–715 共2001兲. 29. Y. Rui, T. Huang, and S. Chang, “Image retrieval: current techniques, promising directions and open issues,” J. Visual Commun. Image Represent 10共4兲, 39–62 共1999兲. 30. J. Huang, S. Kumar, M. Mitra, W. J. Zhu, and R. Zabih, “Image indexing using color correlograms,” in Proc. IEEE Conf. Computer Vision Patt. Recog. (CVPR’97), pp. 762–768 共1997兲. 31. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, John Wiley and Sons Inc., New York 共2001兲. 32. A. Ukovich, A. Zacchigna, G. Ramponi, and G. Schoier, “Using clustering for document reconstruction,” Proc. SPIE 6064, 168–179 共2006兲. 33. K. V. Mardia, J. T. Kent, and J. M. Bibbi, Multivariate Analysis, Academic Press, San Diego, CA 共1979兲. 34. A. K. Jain and R. C. Dubes, Algorithms for Clustering Data, Prentice-Hall, Inc., Upper Saddle River, NJ 共1988兲.
Giovanni Ramponi received his degree in electronic engineering 共summa cum laude兲 in 1981; since 2000 he has been a professor of electronics at the Department of Electronics of the University of Trieste, Italy. His research interests include nonlinear digital signal processing, and in particular the enhancement and feature extraction in images and image sequences. He has been an associate editor 共AE兲 of the IEEE Signal Processing Letters and of the IEEE Transactions on Image Processing; presently is an AE of SPIE’s Journal of Electronic Imaging. He has participated in various EU and national research projects. He is the coinventor of various pending international patents and has published more than 140 papers in international journals and conference proceedings; and book chapters. He contributes to several undergraduate and graduate courses on digital signal processing.
Anna Ukovich obtained her MSc degree in electronic engineering 共summa cum laude兲 from the University of Trieste, Italy, in 2003 and her PhD degree from the same University in 2007. She has worked for one year at the Department of Security Technologies, Fraunhofer IPK, Berlin, Germany. She is currently a contract researcher at the University of Trieste, Italy. Her research interests include image and video processing for security applications.
Journal of Electronic Imaging
013008-13
Jan–Mar 2008/Vol. 17(1)
