Computational Photography (CS 445) –
Fall 2023
Instructor: Derek Hoiem
Where: Coursera
Syllabus
Take-home questions
CampusWire
Discussion (6057)
Tutorial
on Linear Algebra
Tutorial
on NumPy &
Jupyter Notebook
Textbook: Computer
Vision: Algorithms and Applications by Rick Szeliski
Projects
Project 1:
Hybrid Images
|
Project
2: Image Quilting
|
Project
3: Gradient Domain Fusion
|
Project 4:
Image-based Lighting
|
Project 5:
Video Stitching
|
Final Project
|
Class Schedule
|
Week |
Date |
Topic |
Link |
Reading/Notes |
|
1 |
Aug 21 – Aug 27 |
Introduction |
||
|
|
|
Basics of Working with Images |
|
|
|
1 |
Pixels and image filters |
S3.2 (linear filtering) S3.3 (non-linear filtering) |
||
|
2 |
Aug 28 – Sept 3 |
Thinking in frequency |
S3.4 (fourier
transforms) S2.3.3 (compression) |
|
|
2 |
Templates and image
pyramids |
S3.5.2 (image pyramids) S8.1.1 (pyramid alignment) Other reading: |
||
|
3 |
Sept 4 – Sept 10 |
Light and color |
S2.2 (light), S2.3.2
(color) or Forsyth and Ponce Ch 3 |
|
|
3 |
Histograms and color
adjustment |
S3.1 (histograms and color
adjustment) |
||
|
|
Sept 11 (Mon) |
Project 1 (Hybrid images) due |
|
|
|
|
The Digital Canvas: Coloring, Blending, Cutting,
Synthesizing, and Warping Images |
|
|
|
|
4 |
Sept 11 – Sept 17 |
Growing: Texture synthesis
and hole filling |
Texture
Synthesis – Efros Leung (1999) Image
Quilting – Efros Freeman (2001) |
|
|
4 |
Cutting: Intelligent
Scissors and Graph Cuts |
|||
|
5 |
Sept 18 – Sept 24 |
Pasting: Compositing and
blending |
Poisson
Image Editing – Perez et al. (2003) Burt
and Adelson, A multiresolution spline with
application to image mosaics, ACM ToG (1983) |
|
|
5 |
Image warping (translation,
rotation, scale, etc.) |
S3.6
(warping) |
||
|
|
Sep 25 (Mon) |
Project 2 (Image quilting) due |
|
|
|
6 |
Sept 25 – Oct 1 |
Image morphing |
||
|
|
|
Modeling the Physical World |
|
|
|
6 |
The Pinhole Camera |
S2.1.5
(3D to 2D projection) |
||
|
7 |
Oct 2 – Oct 8 |
Single-view Metrology +
Camera Model continued |
||
|
7 |
Single-view 3D
Reconstruction |
Tour into the
picture (Horry et al. 1997) |
||
|
8 |
Oct 9 – Oct 15 |
Midterm Exam (on PrairieLearn) |
|
|
|
9 |
|
“Week 9” is
spring break, accidentally carried over into Coursera |
|
|
|
10 |
Oct 16 – Oct 22 |
Image-based Lighting: ray
tracing, environment maps, light probes |
Reading (do read this): |
|
|
10 |
Image-based Lighting cont.:
HDR light probes, relighting |
Optional
Reading: Debevec
& Malik, “Recovering High Dynamic Range
Radiance Maps from Photographs”, SIGGRAPH 1997 Debevec,
Rendering Synthetic Objects in Real Scenes, 1998 |
||
|
|
Oct 23 (Mon) |
Project 3 (Gradient domain fusion) due |
|
|
|
11 |
Oct 23 – Oct 29 |
The image as a virtual
stage |
Rendering
Synthetic Objects (Karsch et al. 2011) |
|
|
|
Correspondence and Recognition |
|
|
|
|
11 |
Matching and alignment with
interest points |
Grauman/Leibe
Draft Chapter on Local Features Optional: Lowe - SIFT paper |
||
|
|
Oct 30 (Mon) |
Final Project Proposal due |
|
|
|
12 |
Oct 30 – Nov 5 |
Automatic Photo Stitching
and RANSAC |
Brown
Lowe 2007 ; S9 (stitching); slides |
|
|
12 |
Object recognition,
retrieval, and augmented reality |
|||
|
|
Nov 6 (Mon) |
Project 4 (Image-based lighting) due |
|
|
|
13 |
Nov 6 – Nov 12 |
Opportunities of scale:
texture synthesis, multi-view reconstruction, im2gps, tiny images, etc. |
Reading: Hays & Efros, Scene Completion
Using Millions of Photographs |
|
|
|
|
More Topics of Interest |
|
|
|
13 |
Generating and detecting
fakes |
|||
|
14 |
Nov 13 – Nov 17 |
How the Kinect works |
||
|
14 |
Computational approaches to
cameras |
Adelson
& Bergen, The Plenoptic Function and the Elements
of Early Vision (1991) Ng
et al., Light Field Photography with a
Hand-held Plenoptic Camera (2005) Levin
et al., Image and Depth with a Coded Aperture (2007) |
||
|
|
Nov 18 – Nov 26 |
Fall Break |
|
|
|
|
Nov 27 (Mon) |
Project 5 (Video stitching) due |
|
|
|
15 |
Nov 27 – Dec 3 |
Understanding Faces |
|
|
|
15 |
Video
Magnification |
|
||
|
16 |
Dec 4 – Dec 6 |
Last day – cutting
edge and wrap up |
|
|
|
Dec 6 (Wed) |
Final Project due |
|
|
|
|
17 |
Dec
8 – Dec 15 |
Final Exam (on PrairieLearn) |
|
Credits and Course Notes
I am grateful for the
many preceding efforts to design excellent courses and course notes for this
topic. I am particularly indebted
to Alyosha Efros for borrowing much of his course and project design, to Rick
Szeliski for making his book available, to James Hays for course suggestions
and materials, and to Steve Seitz, Lana Lazebnik, Silvio Savarese, among others
for their slides (as credited within).
Anyone who would like to use my slides is more than welcome; please do
credit the original sources where appropriate.