Computational Photography (CS 445) –
Fall 2022
Instructor: Derek Hoiem
Lectures: Tues/Thurs 11-12:15 DCL
1320
Syllabus
Take-home questions
Lecture
Recordings (previous
recordings)
CampusWire
Discussion
Canvas Submission
Textbook: Computer
Vision: Algorithms and Applications by Rick Szeliski
Projects
Project 1:
Hybrid Images
|
Project
2: Image Quilting
|
Project
3: Gradient Domain Fusion
|
Project 4:
Image-based Lighting
|
Project 5:
Video Stitching
|
Final Project
|
Class Schedule
|
Week |
Date |
Topic |
Link |
Reading/Notes |
|
1 |
Aug 23 (Tues) |
Introduction |
||
|
|
|
Basics of Working with Images |
|
|
|
1 |
Aug 25 (Thurs) |
Pixels and image filters |
S3.2 (linear filtering) S3.3 (non-linear filtering) |
|
|
2 |
Aug 30 (Tues) |
Thinking in frequency |
S3.4 (fourier transforms) S2.3.3 (compression) |
|
|
2 |
Sep 1 (Thurs) |
Templates and image
pyramids |
S3.5.2 (image pyramids) S8.1.1 (pyramid alignment) Other reading: |
|
|
3 |
Sep 6 (Tues) |
Light and color |
S2.2 (light), S2.3.2
(color) or Forsyth and Ponce Ch 3 |
|
|
3 |
Sep 8 (Thurs) |
Histograms and color
adjustment |
S3.1 (histograms and color
adjustment) |
|
|
|
Sep 12 (Mon) |
Project 1 (Hybrid images) due |
|
|
|
|
The Digital Canvas: Coloring, Blending, Cutting,
Synthesizing, and Warping Images |
|
|
|
|
4 |
Sep 13 (Tues) |
Growing: Texture synthesis
and hole filling |
Texture
Synthesis – Efros Leung (1999) Image
Quilting – Efros Freeman (2001) |
|
|
4 |
Sep 15 (Thurs) |
Cutting: Intelligent Scissors
and Graph Cuts |
||
|
5 |
Sep 20 (Tues) |
Pasting: Compositing and
blending |
Poisson
Image Editing – Perez et al. (2003) Burt and Adelson, A multiresolution spline with
application to image mosaics, ACM ToG (1983) |
|
|
5 |
Sep 22 (Thurs) |
Image warping (translation,
rotation, scale, etc.) |
S3.6
(warping) |
|
|
|
Sep 26 (Mon) |
Project 2 (Image quilting) due |
|
|
|
6 |
Sep 27 (Tues) |
Image morphing |
||
|
|
|
Modeling the Physical World |
|
|
|
6 |
Sep 29 (Thurs) |
The Pinhole Camera |
S2.1.5
(3D to 2D projection) |
|
|
7 |
Oct 4 (Tues) |
Single-view Metrology +
Camera Model continued |
||
|
7 |
Oct 6 (Thurs) |
Single-view 3D
Reconstruction |
Tour into the
picture (Horry et al. 1997) |
|
|
8 |
Oct
11 (Tues) |
Midterm Review Derek away Oct 10 (office hour Oct 11 3-4) |
||
|
8 |
Oct 13 (Thurs) |
Midterm Exam (on PrairieLearn) Start exam between 9:30am and 10pm |
|
|
|
9 |
Oct 18 (Tues) |
Image-based Lighting: ray
tracing, environment maps, light probes |
Reading (do read this): |
|
|
9 |
Oct 20 (Thurs) |
Image-based Lighting cont.:
HDR light probes, relighting |
Optional Reading: Debevec & Malik, “Recovering High Dynamic Range
Radiance Maps from Photographs”, SIGGRAPH 1997 Debevec, Rendering Synthetic Objects in Real
Scenes, 1998 |
|
|
10 |
Oct 24 (Mon) |
Project 3 (Gradient domain fusion) due Derek away Oct 23-26 (no office hour) |
|
|
|
|
|
Correspondence and Recognition |
|
|
|
10 |
Oct 25 (Tues) |
Matching and alignment with
interest points Derek away, watch Interest
Points recording |
Grauman/Leibe
Draft Chapter on Local Features Optional: Lowe - SIFT paper |
|
|
10 |
Oct 27 (Thurs) |
Automatic Photo Stitching
and RANSAC |
Brown
Lowe 2007 ; S9 (stitching); slides |
|
|
11 |
Oct 31 (Mon) |
Final Project Proposal due |
|
|
|
11 |
Nov 1 (Tues) |
Retrieval, reconstruction,
and AR |
||
|
11 |
Nov 3 (Thurs) |
Opportunities of scale,
encoders-decoders |
Reading: Hays & Efros, Scene Completion
Using Millions of Photographs |
|
|
12 |
Nov 7 (Mon) |
Project 4 (Image-based lighting) due |
|
|
|
12 |
Nov 8 (Tues) |
Election Day (no classes) |
|
|
|
|
|
More Topics of Interest |
|
|
|
12 |
Nov 10 (Thurs) |
Generating and detecting
fakes |
||
|
13 |
Nov 15 (Tues) |
How the Kinect works |
|
|
|
13 |
Nov 17 (Thurs) |
Neural Radiance Fields
(NeRF) |
||
|
14 |
Nov 19-27 |
Fall Break |
|
|
|
|
Nov 28 (Mon) |
Project 5 (Video stitching) due |
|
|
|
15 |
Nov 29 (Tues) |
Computational approaches to
cameras |
Adelson & Bergen, The Plenoptic
Function and the Elements of Early Vision (1991) Ng et al., Light Field Photography with a
Hand-held Plenoptic Camera (2005) Levin et al., Image and Depth with a Coded Aperture (2007) |
|
|
15 |
Nov 31 (Thurs) |
Video
Magnification |
|
|
|
16 |
Dec 6 (Tues) |
Last day – cutting
edge and wrap up |
|
|
|
Dec 9 |
Final Exam (on PrairieLearn) |
|||
|
|
Dec
11 (Sun) |
Final Project due (cannot be late) |
|
|
Credits and Course Notes
I am grateful for the
many preceding efforts to design excellent courses and course notes for this
topic. I am particularly indebted
to Alyosha Efros for borrowing much of his course and project design, to Rick
Szeliski for making his book available, to James Hays for course suggestions
and materials, and to Steve Seitz, Lana Lazebnik, Silvio Savarese, among others
for their slides (as credited within).
Anyone who would like to use my slides is more than welcome; please do
credit the original sources where appropriate.