Hertzmann - Frontiers in Computer Vision

3 downloads 217 Views 2MB Size Report
“Bone” by JeffSmith © 2007. Our result. Page 15. “Bone” by JeffSmith © 2007. Our result. Page 16. “Bone” by JeffSmith © 2007. Our result. Page 17. The Science ...
Cross-fertilization of graphics and vision Aaron Hertzmann University of Toronto

Image/Video Manipulation

Image and video manipulation

C. Barnes, E. Shechtman, A. Finkelstein, D. B. Goldman, “PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing, ACM Trans. Graph (SIGGRAPH) 2009

Special effects

Stargate Studios (www.stargatefilms.com)

Special effects

Stargate Studios (www.stargatefilms.com)

Look Effects

coming next year: Computer Vision for Visual Effects Richard J. Radke Cambridge University Press, Fall 2012

Generalization of vision ideas

Geometry signal processing

Taubin, SIGGRAPH 1995

Jones, Durand, Desbrun, SIGGRAPH 2003

Geometry processing

E. Kalogerakis, A. Hertzmann, K. Singh, "Learning 3D Mesh Segmentation and Labeling", ACM Trans. Graphics / SIGGRAPH 2011

A. M. Bronstein, M. M. Bronstein, M. Ovsjanikov, L. J. Guibas, "Shape Google: geometric words and expressions for invariant shape retrieval", ACM Trans. Graphics (TOG), 2011.

Non-photorealistic rendering (demo)

Non-photorealistic rendering

Lambertian rendering

Contours and Suggestive Contours [DeCarlo et al. 2003]

“Bone” by Jeff Smith © 2007

Our result

“Bone” by Jeff Smith © 2007

Our result

“Bone” by Jeff Smith © 2007

Our result

The Science of Art Claim: Vision and graphics will be fundamental to the scientific study of art

See: A. Hertzmann, “Non-Photorealistic Rendering and The Science of Art,” NPAR 2010

Fei-Fei et al. 2007

Why vision should continue to look to graphics: Graphics builds models of precisely the “visual world”

Scene models

Chaudhuri et al. SIGGRAPH 2011

Yu et al. SIGGRAPH 2011

Google 3D Warehouse

M. Fisher, M. Savva, P. Hanrahan. “Characterizing Structural Relationships in Scenes Using Graph Kernels,” SIGGRAPH 2011

Human motion K. Grochow, S. L. Martin, A. Hertzmann, Z. Popović. “Style-Based Inverse Kinematics” ACM Trans. Graphics / SIGGRAPH 2004.

R. Urtasun, D. J. Fleet, A. Hertzmann, P. Fua. “Priors for People Tracking from Small Training Sets” ICCV 2005

Claim: Understanding people in motion requires physical models

Physics-Based Characters Features Contact

COM

AM

Posture

M. de Lasa, I. Mordatch, A. Hertzmann, “Feature-Based Locomotion Controllers,” SIGGRAPH 2010

M. de Lasa, I. Mordatch, A. Hertzmann, “Feature-Based Locomotion Controllers,” SIGGRAPH 2010

M. de Lasa, I. Mordatch, A. Hertzmann, “Feature-Based Locomotion Controllers,” SIGGRAPH 2010

M. de Lasa, I. Mordatch, A. Hertzmann, “Feature-Based Locomotion Controllers,” SIGGRAPH 2010

M. Brubaker, D. J. Fleet, A. Hertzmann, “Physics-Based Person Tracking Using the Anthropomorphic Walker” CVPR 2007, IJCV 2010

Summary Many successes of vision ideas in graphics Vision and graphics are key to the science of art Vision should look to graphics for models of the visual world