MP practicalities

Questions related to what you built in MP 5 and 6.

Computer vision

Image formation

• Image formation (pinhole camera, real lenses, human eye)
• Digitization (computer, human, including color)
• Edge detection, segmentation

Relating 2D to 3D: why might an object look different in two pictures?

Classification:

• Identifying/naming objects in a picture
• Localizing/registering objects within a picture
• Visual question answering, captioning, semantic role labelling for a picture

Reconstructing 3D geometry

• why is it useful?
• from multiple 2D views of a scene
• from a single picture

Other tasks

• Image generation
• Predicting the future

Classifier Overview

Know brief facts about the following people and systems/models:

• William Labov
• The BERT language model
• CFAR-10 dataset

General design

• Uses for classification: labelling objects, making decisions
• Multi-layer systems
• What can we tune?
  • parameters (e.g. weights)
  • hyper-parameters (e.g. tuning constants)
  • design, network topology
• Challenges with determining the correct answer
  • how specific/general should the class label be?
  • unfamiliar objects, unfamiliar words
  • context may affect best label to choose
  • deciding what's important in complex scenes, extended sentences
• Data for supervised training
  • "gold" answers
  • Noise in "correct" answers/annotation
  • Annotators with limited training
  • Data scraped off the web
  • Data available only for final output of system
• Workarounds of limited training data
  • Re-purposing layers trained for another purpose
  • Creating training pairs by removing information
  • Self-supervised, semi-supervised, unsupervised methods
• Batch vs. incremental training

K-nn and Decision Trees

Specific techniques

• k-nearest neighbors (how it works, what happens if you change k)
• L1 vs. L2 norm
• Decision trees, random forests
• Entropy: definition, how it relates to evaluating possible splits in a decision tree

Perceptrons

• "Linearly separable"
• Basics of how perceptrons work
• Replacing bias with an extra weight
• Overall training algorithm (e.g. epochs, random processing order)
• Rule for updating perceptron weights
• Limitations of perceptrons and ways to address them
• Multi-class perceptrons
• Comparison to Naive Bayes