Computational Photography (CS 445) – Fall 2021  

  

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: <i>The Key to the Fields</i> (Magritte 1936)

  Instructor:  Yuxiong Wang

 

 

 

  Where: Zoom

  When: Mon/Wed 11:00 AM - 12:15 PM (CT)

 

  Instructor Office Hours: Mon 2:30 PM - 3:30 PM (CT) Zoom

 

  TA: Eric Lee (kylee5), Yuanyi Zhong (yuanyiz2)

  Office Hours: (Eric) Tue 3:45 PM - 4:15 PM (CT) Zoom

  Office Hours: (Yuanyi) Wed 8:00 PM - 9:00 PM (CT) Zoom

  Office Hours: (Yuanyi) Thu 5:30 PM - 6:00 PM (CT) Zoom

  Office Hours: (Eric) Fri 9:00 AM - 10:00 AM (CT) Zoom

 

  Syllabus

  Take-Home Questions

  Piazza Discussion Board

  Lecture Recordings on Mediaspace

  Compass 2g (project submission and grades)

 

  Tutorial on Linear Algebra

  Tutorial on NumPy & Jupyter Notebook

 

 

 

 

  Textbook: Computer Vision: Algorithms and Applications by Rick Szeliski

                                                                                                           

  

   Projects

Project 1: Hybrid Images

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: leopard elephant

Project 2: Image Quilting

    Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: C:\Users\Hoiem\Documents\Classes\ComputationalPhotography - Fall 2012\projects\quilting\face_toast_cropped.png

   

Project 3: Gradient Domain Fusion

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: cloning_teaser

 

Project 4: Image-based Lighting

https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQJ6tcY1JugjgyMMpZMj9BgU6OaGyIv8jL4T8J-Qsgnn0UOrHT2tw

 

 

Project 5: Video Stitching

 

 

Final Project

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: http://t2.gstatic.com/images?q=tbn:ANd9GcSEXEDpOJHSX2MOS0mvolJmNhoGKiQZ6t4ky5dyEio1mph1-L1x

 

 

 

  

  Class Schedule  

Week

Date

Topic

Link

Reading/Notes

1

Aug 23

Introduction

ppt ; pdf

 

 

Basics of Working with Images

 

 

1

Aug 25

Pixels and image filters

ppt ; pdf

S3.2 (linear filtering)

S3.3 (non-linear filtering)

2

Aug 30

Thinking in frequency

ppt ; pdf

S3.4 (fourier transforms)

S2.3.3 (compression)

FFT Demo (original) ; Python demo by Jae

2

Sep 1

(Sep 6 break)

Templates and image pyramids

ppt ; pdf

S3.5.2 (image pyramids)

S8.1.1 (pyramid alignment)

Other reading:

   Burt Adelson 1983 – The Laplacian Pyramid

3

Sep 8

Light and color

ppt ; pdf

S2.2 (light), S2.3.2 (color)

or Forsyth and Ponce Ch 3

4

Sep 13

Histograms and color adjustment

ppt ; pdf

S3.1 (histograms and color adjustment)

Color balance demo

 

Sep 17 (Fri)

Project 1 (Hybrid images) due

 

 

 

The Digital Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images

 

 

4

Sep 15

Growing: Texture synthesis and hole filling

ppt ; pdf

Texture Synthesis – Efros Leung (1999)

Image Quilting – Efros Freeman (2001)

Image Analogies – Hertzmann et al. (2001)

Patch Match – Barnes et al. (2009)

5

Sep 20

Cutting: Intelligent Scissors and Graph Cuts

ppt ; pdf

Intelligent Scissors – Mortenson Barret (1995)

GrabCutRother et al. (2004)

5

Sep 22

Pasting: Compositing and blending

ppt ; pdf

Poisson Image Editing – Perez et al. (2003)

Burt and Adelson, A multiresolution spline with application to image mosaics, ACM ToG (1983) 

6

Sep 27

Image warping (translation, rotation, scale, etc.)

ppt ; pdf

S3.6 (warping)

Alignment demo

 

Sep 29 (Wed)

Project 2 (Image quilting) due

 

 

6

Sep 29

Image morphing

ppt ; pdf

 

 

Modeling the Physical World

 

 

7

Oct 4

The Pinhole Camera

ppt ; pdf

S2.1.5 (3D to 2D projection)

7

Oct 6

Single-view Metrology + Camera Model continued

ppt ; pdf

HoiemSavarese-SVG Chapter

8

Oct 11

Single-view 3D Reconstruction

ppt ; pdf

Tour into the picture (Horry et al. 1997)

8-9

Oct 12 – OCT 19

Midterm Exam (on PrarieLearn)

 

 

9

Oct 20

Image-based Lighting: ray tracing, environment maps, light probes

ppt ; pdf

Reading (do read this):

Debevec, Image-based Lighting Tutorial (2005)

10

Oct 25

Image-based Lighting cont.: HDR light probes, relighting

ppt ; pdf

Optional Reading:

Debevec & Malik, “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997

Debevec, Rendering Synthetic Objects in Real Scenes, 1998

 

Oct 27 (Wed)

Project 3 (Gradient domain fusion) due

 

 

10

Oct 27

The image as a virtual stage

ppt ; pdf

Rendering Synthetic Objects (Karsch et al. 2011)

 

Correspondence and Recognition

 

 

11

Nov 1

Matching and alignment with interest points

ppt ; pdf

Grauman/Leibe Draft Chapter on Local Features Optional: Lowe - SIFT paper

 

Nov 5 (Fri)

Final Project Proposal due

 

 

11

Nov 3

Automatic Photo Stitching and RANSAC

ppt ; pdf

Brown Lowe 2007 ; S9 (stitching);  slides

12

Nov 8

Object recognition, retrieval, and augmented reality

ppt ; pdf

Grauman/Leibe Draft Visual Vocabularies; S14.3

 

Nov 12 (Fri)

Project 4 (Image-based lighting) due

 

 

12

Nov 10

Opportunities of scale: texture synthesis, multi-view reconstruction, im2gps, tiny images, etc.

ppt ; pdf

Reading: Hays & Efros, Scene Completion Using Millions of Photographs

 

 

More Topics of Interest

 

 

13

Nov 15

Generating and detecting fakes  

ppt ; pdf

Pix2pix

Cycle GAN

Deep fakes in news

13

Nov 17

How the Kinect works

ppt ; pdf

 

Nov 22 – Nov 28  

Fall Break

 

 

14

Nov 29

Computational approaches to cameras

ppt ; pdf

Adelson & Bergen, The Plenoptic Function and the Elements of Early Vision (1991)

Ng et al., Light Field Photography with a Hand-held Plenoptic Camera (2005)

Levin et al., Image and Depth with a Coded Aperture (2007)

 

Dec 1 (Wed)

Project 5 (Video stitching) due

 

 

14

Dec 1

Understanding Faces

ppt ; pdf

 

15

Dec 6

Video Magnification

ppt ; pdf

 

 

 

 

 

 

15

Dec 8

Last day – cutting edge and wrap up

ppt ; pdf

 

Dec 10 (Fri)

Final Project due (cannot be late)

 

 

16

Dec 10 – Dec 17

Final Exam (on PrairieLearn)

 

 





 


 

 

 

Credits and Course Notes

I am grateful for the many preceding efforts to design excellent courses and course notes for this topic.  I am particularly indebted to Derek Hoiem for designing this course and borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for course suggestions and materials, and to Alyosha Efros, Steve Seitz, Lana Lazebnik, Silvio Savarese, among others for their slides (as credited within).  Anyone who would like to use my slides is more than welcome; please do credit the original sources where appropriate.

 

 

 


Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: GoStats.com