Video Forgery Detection and Localization based on 3D ... - Google Sites

0 downloads 245 Views 2MB Size Report
ó We propose an effective video copy-move algorithm ó It extends our image ... ó A real-world case. D.Cozzolino, G.Po
GRIP

Image Processing Research Group

Video Forgery Detection and Localization based on 3D PatchMatch L. D’Amiano, D. Cozzolino, G. Poggi, L. Verdoliva University Federico II of Naples, ITALY

WeMuV – Torino 29 June 2015

From Image Forgery (a long tradition) Improving a dictator image Benito Mussolini 1942

Insurance Fraud “66 on trial” La Repubblica 2012

... to Video Forgery (growing)

The Varoufakis web case 2015

A copy-move video forgery FRAME:

65

66

67

68

69

250

Original

...

Fake

...

251

252

253

https://sites.google.com/site/rewindpolimi/downloads/datasets/videocopy-move-forgeries-dataset

254

About this work We propose an effective video copy-move algorithm It extends our image copy-move forgery detector (CMFD) Dense-field approach Rotation-invariant features Fast matching and post-processing

Numerical assessment on a publicly available database A real-world case

D.Cozzolino, G.Poggi, and L.Verdoliva, “Efficient dense-field copy-move forgery detection”, IEEE Transactions on Information Forensics and Security, to appear.

Main steps in CMFD Feature Extraction

List of features

Matching

[13,11, 32, ….,103] Patch

Feature vector

PostProcessing

Keypoint-vs-dense CMFD Keypoint-based

Dense-field

Only high-entropy points

Regular grid of points

FAST Loses «occlusive» copy-moves

SLOW Catches all types of copy-moves

Main qualifying points Feature Extraction

Rotation-invariant features: • Zernike moments on a polar grid

Matching

Fast matching with smooth matching field • Modified PatchMatch

PostProcessing

Fast and accurate post-processing • Dense linear fitting

PatchMatch A stochastic iterative fast matching algorithm Alternates propagation and random search steps Quick convergence to an approximate (but accurate) and smooth NN field

C. Barnes et al., “PatchMatch: a randomized correspondence algorithm for structural image editing,” ACM Transactions on Graphics, 2009.

Propagation in PatchMatch top neighbour

current match of top neighbour

left neighbour

candidates

current match of left neighbour

Zero-order predictors are used, good for pw-constant fields

Modified PatchMatch Copy-move with rotation and resizing induce linear (not constant) offset fields Modified propagation (includes 1st-order predictors) Allows working of compact features

Propagation in modified PatchMatch top neighbours current matches of top neighbours

left neighbours

candidates

current matches of left neighbours

First-order predictors allow one to deal with pw-linear fields

Some sample results

original image

forgery

[Christlein2012]

[Cozzolino2015]

TP V. Christlein et al., “An Evaluation of Popular Copy-Move Forgery Detection Approaches,” IEEE TIFS 2012.

TN

FN

FP

1

1

0.8

0.8

0.8

0.6 0.4 proposed Christlein2012 Bravo2011 Amerini2013

0.2

0 NC 100 90 80 70 60 50 40 30 20 JPEG-compression (QF)

avr F-measure

1

avr F-measure

avr F-measure

Comparison with state of the art 0.6 0.4 0.2

proposed Christlein2012 Bravo2011 Amerini2013

0 0.5 0.650.8 0.93 1 1.071.2 1.6 Resizing (Scale)

0.6 0.4 0.2

2

0 0° 4° 10° 30° 60° 90° Rotation (Angle)

More accurate than keypoint-based methods • Faster than comparable dense-based methods •

proposed Christlein2012 Bravo2011 Amerini2013

180°

Adaptation to video Same structure as image CMFD algorithm 3D patches Include temporal predictors in PatchMatch Temporal guard interval Post-processing includes the temporal direction

PatchMatch 3D

PatchMatch 3D current match of F.1 left neighbour

F.2

current match of top neighbour

F.3

top neighbour

F.4

F.5

F.6 candidates

Past neighbour

F.7 current match of past neighbour

left neighbour

F.8

F.N

Rewind dataset Expands the University of Surrey SULFA database 10 videos, approximately 300 frames, 320x240 pixels/f. Forgeries are created by copy-moving a portion of the video Copy-moves involve from 33 to 211 frames, with copied regions going from 630 (very small) to 30680 pixels Only rigid copy-moves are performed We add copy-moves with rotation / resizing for video #3

Detection Performance Uncompressed

QP=10

QP=20

CPU

TPR*

FPR*

TPR*

FPR*

TPR*

FPR*

(s)

Proposed

9

0

9

0

9

0

526

PM - RGB

9

0

9

0

9

0

800

Bestagini

10

3

9

1

8

1

154

(*) all numbers to be divided by 10, the number of videos

M. Bleyer, C. Rhemann, and C. Rother, “Patchmatch stereo - stereo matching with slanted support windows,” in British Machine Vision Conference, 2011. P. Bestagini, S. Milani, M. Tagliasacchi, and S. Tubaro, “Local tampering detection in video sequences,” in IEEE International Workshop on Multimedia Signal Processing, 2013.

Video #7 – wrong matches FRAME:

5

6

7

8

9

37

VIDEO FAKE

...

MATCHING

...

38

39

40

41

Video #7 – correct matches FRAME:

VIDEO FAKE

MATCHING SHOULD BE

5

6

7

8

9

37

...

...

38

39

40

41

Localization Performance Uncompressed

QP=10

QP=20

CPU

TPR

FPR

TPR

FPR

TPR

FPR

(s)

Proposed

48.77

0.02

46.10

0.12

29.41

0.06

526

PM - RGB

51.81

0.00

42.88

0.02

21.09

0.03

800

Localization Performance on video #3 Uncompressed TPR

FPR

QP=10 TPR

QP=20

FPR

TPR

FPR

Rigid copy-move Proposed

44.45

0.00

43.97

0.00

42.46

0.00

PM - RGB

42.88

0.00

32.10

0.00

42.09

0.15

Copy-move with rotation Proposed

36.64

0.00

37.47

0.00

25.26

0.00

PM - RGB

1.96

0.00

3.40

0.00

0.00

0.05

Copy-move with resizing Proposed

34.69

0.00

34.82

0.00

34.12

0.00

PM - RGB

38.84

0.00

39.34

0.00

37.06

0.05

Example masks for video #3: rotated CM

TP

TN

FN

FP

A real-world case zdf-neo video

YouTube video

Comparison of videos

zdf-neo video

YouTube video

Difference They differ in only 80 frames

Results of PM-based technique

zdf-neo video

YouTube video

Differences between matched frames

Differences between aligned frames

The verdict zdf-neo video

YouTube video

YouTube video (previous frames)

A real-world case zdf-neo video

YouTube video #1

Video YouTube #2

Future work Use 3d descriptors Exploit motion information (e.g. objects tracking) speed-up search − Multi-resolution version − Non-random fast initialization

Create a richer database of copy-move video forgeries