MP practicalities

Questions related to what you built in MP 5 and 6.

Computer vision

Image formation

• Image formation (pinhole camera, real lenses, human eye)
• Digitization (computer, human, including color and aliasing)

Why might an object or type of object look different in two pictures?

• changes/variation in the object itself
• changes in the 3D scene
• changes in the 2D camera view

Classification:

• Identifying/naming objects in a picture
• Localizing/registering objects within a picture
• Visual question answering, captioning, semantic role labelling for a picture

Reconstructing 3D geometry

• why is it useful?
• from multiple 2D views of a scene
• from a single picture
• from changes in picture (e.g. obstacle avoidance)

Other tasks

• Image generation
• Predicting the future, explaining the past

Classifier Overview

Know brief facts about the following people and systems/models:

• William Labov
• The BERT language model
• CFAR-10 dataset
• Amazon's Mechanical Turk

General design

• Uses for classification: labelling objects, making decisions
• Recall basic classification concepts from early in the term, e.g. train/test/development data, overfitting
• Multi-layer systems
• What can we tune?
  • parameters (e.g. weights)
  • hyper-parameters (e.g. tuning constants)
  • design, network topology
• Batch vs. incremental training

What answer should we produce?

• Challenges with determining the correct answer
  • how specific/general should the class label be?
  • context may affect best label to choose
  • deciding what's important in complex scenes, extended sentences
• Ways to name unfamiliar object
  • By parts
  • As variants of familiar objects or by analogy with familiar objects
  • As modifications of familiar objects (e.g. broken)

Training data

• Supervised training and "gold standard" annotations
• Limitations of "correct" answers/annotation
  • Errors even with expert annotators
  • Use of amateur annotators (e.g. Mechanical Turk)
  • Annotations scraped off the web (e.g. captions for pictures)
  • Data available only for final output of system
• Workarounds of limited training data
  • Re-purposing layers trained for another purpose
  • Creating training pairs by removing information (e.g. color) or masking parts of the input image or text
  • Self-supervised, semi-supervised, unsupervised methods

K-nn and Decision Trees

Specific techniques

• k-nearest neighbors (how it works, what happens if you change k)
• L1 vs. L2 norm
• Decision trees, random forests
• Entropy: definition, how it relates to evaluating possible splits in a decision tree

Perceptrons

• "Linearly separable"
• Basics of how perceptrons work
• Replacing bias with an extra weight
• Overall training algorithm (e.g. epochs, random processing order)
• Rule for updating perceptron weights
• Limitations of perceptrons and ways to address them
• Multi-class perceptrons
• Comparison to Naive Bayes