“Bone” by JeffSmith © 2007. Our result. Page 15. “Bone” by JeffSmith © 2007.
Our result. Page 16. “Bone” by JeffSmith © 2007. Our result. Page 17. The
Science ...
Cross-fertilization of graphics and vision Aaron Hertzmann University of Toronto
Image/Video Manipulation
Image and video manipulation
C. Barnes, E. Shechtman, A. Finkelstein, D. B. Goldman, “PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing, ACM Trans. Graph (SIGGRAPH) 2009
Special effects
Stargate Studios (www.stargatefilms.com)
Special effects
Stargate Studios (www.stargatefilms.com)
Look Effects
coming next year: Computer Vision for Visual Effects Richard J. Radke Cambridge University Press, Fall 2012
Generalization of vision ideas
Geometry signal processing
Taubin, SIGGRAPH 1995
Jones, Durand, Desbrun, SIGGRAPH 2003
Geometry processing
E. Kalogerakis, A. Hertzmann, K. Singh, "Learning 3D Mesh Segmentation and Labeling", ACM Trans. Graphics / SIGGRAPH 2011
A. M. Bronstein, M. M. Bronstein, M. Ovsjanikov, L. J. Guibas, "Shape Google: geometric words and expressions for invariant shape retrieval", ACM Trans. Graphics (TOG), 2011.
Non-photorealistic rendering (demo)
Non-photorealistic rendering
Lambertian rendering
Contours and Suggestive Contours [DeCarlo et al. 2003]
“Bone” by Jeff Smith © 2007
Our result
“Bone” by Jeff Smith © 2007
Our result
“Bone” by Jeff Smith © 2007
Our result
The Science of Art Claim: Vision and graphics will be fundamental to the scientific study of art
See: A. Hertzmann, “Non-Photorealistic Rendering and The Science of Art,” NPAR 2010
Fei-Fei et al. 2007
Why vision should continue to look to graphics: Graphics builds models of precisely the “visual world”
Scene models
Chaudhuri et al. SIGGRAPH 2011
Yu et al. SIGGRAPH 2011
Google 3D Warehouse
M. Fisher, M. Savva, P. Hanrahan. “Characterizing Structural Relationships in Scenes Using Graph Kernels,” SIGGRAPH 2011
Human motion K. Grochow, S. L. Martin, A. Hertzmann, Z. Popović. “Style-Based Inverse Kinematics” ACM Trans. Graphics / SIGGRAPH 2004.
R. Urtasun, D. J. Fleet, A. Hertzmann, P. Fua. “Priors for People Tracking from Small Training Sets” ICCV 2005
Claim: Understanding people in motion requires physical models
Physics-Based Characters Features Contact
COM
AM
Posture
M. de Lasa, I. Mordatch, A. Hertzmann, “Feature-Based Locomotion Controllers,” SIGGRAPH 2010
M. de Lasa, I. Mordatch, A. Hertzmann, “Feature-Based Locomotion Controllers,” SIGGRAPH 2010
M. de Lasa, I. Mordatch, A. Hertzmann, “Feature-Based Locomotion Controllers,” SIGGRAPH 2010
M. de Lasa, I. Mordatch, A. Hertzmann, “Feature-Based Locomotion Controllers,” SIGGRAPH 2010
M. Brubaker, D. J. Fleet, A. Hertzmann, “Physics-Based Person Tracking Using the Anthropomorphic Walker” CVPR 2007, IJCV 2010
Summary Many successes of vision ideas in graphics Vision and graphics are key to the science of art Vision should look to graphics for models of the visual world
