Computational Photography (CS 445) – Fall 2020
Instructor: Derek Hoiem
Coursera
Syllabus
Take-home
questions
Original Lecture
Recordings (FA19)
Piazza Discussion Board
Textbook: Computer
Vision: Algorithms and Applications by Rick Szeliski
Projects
Project 1:
Hybrid Images
|
Project
2: Image Quilting
|
Project
3: Gradient Domain Fusion
|
Project 4:
Image-based Lighting
|
Project 5:
Video Stitching
|
Final Project
|
Class Schedule
|
Week |
Date |
Topic |
Link |
Reading/Notes |
|
1 |
Aug 24 – Aug 30 |
Introduction |
||
|
|
|
Basics of
Working with Images |
|
|
|
1 |
Pixels and image filters |
S3.2 (linear filtering) S3.3 (non-linear filtering) |
||
|
2 |
Aug 31 – Sept 6 |
Thinking in frequency |
S3.4 (fourier transforms) S2.3.3 (compression) |
|
|
2 |
Templates and image pyramids |
S3.5.2 (image pyramids) S8.1.1 (pyramid alignment) Other reading: |
||
|
3 |
Sept 7 – Sept 13 |
Light and color |
S2.2 (light), S2.3.2 (color) or Forsyth and Ponce Ch 3 |
|
|
3 |
Histograms and color adjustment |
S3.1 (histograms and color adjustment) |
||
|
|
Sept 14
(Mon) |
Project 1
(Hybrid images) due |
|
|
|
|
The Digital
Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images |
|
|
|
|
4 |
Sept 14 – Sept 20 |
Growing: Texture synthesis and hole filling |
Texture
Synthesis – Efros Leung (1999) Image
Quilting – Efros Freeman (2001) |
|
|
4 |
Cutting: Intelligent Scissors and Graph Cuts |
|||
|
5 |
Sept 21 – Sept 27 |
Pasting: Compositing and blending |
Poisson
Image Editing – Perez et al. (2003) Burt
and Adelson, A multiresolution spline with
application to image mosaics, ACM ToG (1983) |
|
|
5 |
Image warping (translation, rotation, scale, etc.) |
S3.6 (warping) |
||
|
|
Sep 28 (Mon) |
Project 2 (Image quilting) due |
|
|
|
6 |
Sept 28 – Oct 4 |
Image morphing |
||
|
|
|
Modeling
the Physical World |
|
|
|
6 |
The Pinhole Camera |
S2.1.5 (3D to 2D projection) |
||
|
7 |
Oct 5 – Oct 11 |
Single-view Metrology + Camera Model continued |
||
|
7 |
Single-view 3D Reconstruction |
Tour into the picture (Horry et
al. 1997) |
||
|
8 |
Oct 12 –
Oct 18 |
Midterm
Exam (schedule with ProctorU) |
|
|
|
9 |
Oct 19 – Oct 25 |
Image-based Lighting: ray tracing, environment maps, light
probes |
Reading (do read this): |
|
|
9 |
Image-based Lighting cont.: HDR light probes, relighting |
Optional
Reading: Debevec & Malik, “Recovering High Dynamic Range
Radiance Maps from Photographs”, SIGGRAPH 1997 Debevec, Rendering Synthetic Objects in Real Scenes, 1998 |
||
|
|
Oct 26
(Mon) |
Project 3
(Gradient domain fusion) due |
|
|
|
10 |
Oct 26 – Nov 1 |
The image as a virtual stage |
Rendering
Synthetic Objects (Karsch et al. 2011) |
|
|
|
Correspondence
and Recognition |
|
|
|
|
10 |
Matching and alignment with interest points |
Grauman/Leibe Draft Chapter on
Local Features Optional: Lowe - SIFT paper |
||
|
|
Nov 2 (Mon) |
Final
Project Proposal due |
|
|
|
11 |
Nov 2 – Nov 8 |
Automatic Photo Stitching and RANSAC |
Brown Lowe 2007 ; S9 (stitching); slides |
|
|
11 |
Object recognition, retrieval, and augmented reality |
|||
|
|
Nov 9 (Mon) |
Project 4
(Image-based lighting) due |
|
|
|
12 |
Nov 9 – Nov 15 |
Opportunities of scale: texture synthesis, multi-view
reconstruction, im2gps, tiny images, etc. |
Reading: Hays & Efros,
Scene Completion Using Millions of Photographs |
|
|
|
|
More Topics
of Interest |
|
|
|
12 |
Generating and detecting fakes |
|||
|
13 |
Nov 16 – Nov 20 |
How the Kinect works |
||
|
13 |
Computational approaches to cameras |
Adelson
& Bergen, The Plenoptic Function and the Elements
of Early Vision (1991) Ng
et al., Light Field Photography with a
Hand-held Plenoptic Camera (2005) Levin
et al., Image and Depth with a Coded Aperture (2007) |
||
|
|
Nov 21 –
Nov 29 |
Fall Break |
|
|
|
|
Nov 30
(Mon) |
Project 5
(Video stitching) due |
|
|
|
14 |
Nov 30 – Dec 6 |
Understanding Faces |
|
|
|
14 |
Video Magnification |
|
||
|
|
|
|
|
|
|
15 |
Dec 7 – Dec 9 |
Last day – cutting edge and wrap up |
|
|
|
Dec 9 (Wed) |
Final
Project due |
|
|
|
|
16 |
Dec 11 – Dec 18 |
Final Exam (schedule
with ProctorU) |
|
Credits and Course Notes
I
am grateful for the many preceding efforts to design excellent courses and
course notes for this topic. I am
particularly indebted to Alyosha Efros
for borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for
course suggestions and materials, and to Steve Seitz, Lana Lazebnik,
Silvio Savarese, among others for their slides (as credited within). Anyone who would like to use my slides
is more than welcome; please do credit the original sources where appropriate.