Request for Approval

Description

The request for approval (RFA) is the very first step in successfully completing a senior design project. Before submitting your RFA, you must post your project idea to the Web Board using the "Idea" post type. Once your idea has been fleshed out through the Web Board, you can move on request for approval through PACE under the My Project page. Once submitted, your project will be cloned to the Web Board as "Project Request" post. You can edit the project on the My Project page, add your teammates and see comments from the instructors. The course staff may provide feedback on your idea (which will appear at the bottom of your project's page), or suggest changes in the scope of the project and ask you to re-submit an RFA. Based on your incorporation of feedback your project will be approved or rejected. If it is rejected, the My Project page will revert back to it's original format and your project will disappear.

Once the course staff has approved the project idea, you will receive instructions on how to submit your project through PACE, at which time you will be assigned a project number in the Projects list, a TA, and a locker in the lab. Once your project is approved, please go to the Projects page, log into the PACE system, and make sure all of the information is correct.

Video Lecture

Video, Slides

Requirements and Grading

The RFA is graded credit/no credit based on whether your project is approved before the deadline. Note that submitting an RFA before the deadline does not guarantee approval before the deadline. The RFA is submitted through PACE under the My Project page, and should be Markdown-formatted with the following information:

# Title

Team Members:
- Student 1 (netid)
- Student 2 (netid)
- Student 3 (netid)

# Problem

Describe the problem you want to solve and motivate the need.

# Solution

Describe your design at a high-level, how it solves the problem, and introduce the subsystems of your project.

# Solution Components

## Subsystem 1

Explain what the subsystem does.  Explicitly list what sensors/components you will use in this subsystem.  Include part numbers.

## Subsystem 2

## ...

# Criterion For Success

Describe high-level goals that your project needs to achieve to be effective.  These goals need to be clearly testable and not subjective.

Projects must be legal and ethical. They must have significant scope and complexity commensurate with the size of the team. This is, of course, a subjective assessment of the course staff. To gain some insight into this judgment, please browse projects from previous semesters. The project must involve the design of a significant hardware component at the circuit level. In exceptional cases, projects not meeting this criteria may be acceptable when augmented by a Special Circuit assignment (however this is typically a last resort).

Beyond these basic requirements, you have total discretion in proposing a project. This is a great opportunity for you to pursue your own interests. Since you choose your own projects, we expect a high level of enthusiasm from you and your team.

Submission and Deadlines

The RFA submission deadline may be found on the Course Calendar. Typically, approval of the RFA is due during the afternoon of the third Thursday of the semester.

Quick Tips and Helpful Hints

Posting: Choosing a project: Choosing partners: Some general project ideas that are fraught with pitfalls:

AI-Nutritious Culinary Assistant

Jackson Brown, Griffin Kelley, Tony Liu

AI-Nutritious Culinary Assistant

Featured Project

Team Members:

- Jackson Brown (jcb10)

- Griffin Kelley (gkelley4)

- Tony Liu (zikunl2)

# Problem

The processed food industry has become increasingly toxic due to chemical flavor additives (12%-32% higher cancer risk), yet cooking often intimidates new chefs when preparing. Therefore, students and working professionals may rely on convenient but unwholesome meals. Most available ‘smart cooking’ tools do not provide a real-time experience that guides users through the process from raw ingredients to the finished dish. This reduces the likelihood that the user will learn. It is also inefficient to design recipes that can be adjusted to the user's available ingredients. Users will waste food, and recipe creation is an expert skill that is difficult to customize. A healthy diet is important for increased productivity and long-term health, but is difficult to accomplish.

# Solution

We propose an AI-Nutritious Culinary Assistant that recognizes available ingredients and generates a personalized recipe with interactive, step-by-step guidance. Using the Meta Quest 3 as the user interface and sensor front-end, the system streams video and voice commands to an edge vision processor running an ingredient recognition pipeline. In addition to vision, the device integrates an environmental sensor module that measures ingredient weight (for portion verification) and ambient temperature (for context/safety telemetry). Finally, the appliance includes a circular rotating seasoning carousel driven by a stepper motor for proportional seasoning action and a servo-actuated gate for controlled dispensing, enabling closed-loop “dispense to target grams” assistance during cooking.

# Solution Components

Explain what the subsystem does. Explicitly list what sensors/components you will use in this subsystem. Include part numbers.

## Subsystem 1 - VR Headset Sensor Platform (Software)

This subsystem uses the Meta Quest 3 as the primary user interface and sensing front-end. The headset’s built-in RGB cameras capture the cooking scene for ingredient recognition, and its microphone captures the user’s wake word and spoken prompts. The headset also serves as the output display, presenting step-by-step recipe instructions as an AR/VR overlay. Captured camera frames and voice transcripts/commands are transmitted to the Vision Processor subsystem for inference and planning, while the returned recipe steps and alerts are rendered back in the headset.

### Parts:

Meta Quest 3 (VR headset with RGB cameras + microphone + display)

Wireless link (Wi-Fi) between Quest and Jetson (stream frames + commands, receive instructions)

## Subsystem 2 - Vision Processor (Software)

This subsystem performs the core perception and recipe-planning computation on the edge compute unit. It receives camera frames from the Meta Quest 3 and first detects/segments candidate ingredient regions using a real-time model (YOLO for bounding boxes, or FastSAM for masks). Each candidate region is then cropped (or masked) and passed through a CLIP-style vision encoder to generate an image embedding. In parallel, the system maintains a library of text embeddings for ingredient labels (e.g., “tomato”, “onion”, “spinach”). By comparing image embeddings to text embeddings (cosine similarity), the processor assigns the most likely ingredient label to each detected region, enabling more flexible recognition than closed-set detection alone. The final output is a structured ingredient list (label + confidence + location/mask), which is then provided to the LLM agent to generate step-by-step recipes and instructions that are sent back to the headset.

### Parts:

NVIDIA Jetson Nano (edge compute unit for model inference + agent logic)

Candidate region model: YOLOv8/YOLO11 (det/seg) (bounding boxes or segmentation) or FastSAM (mask proposals)

Vision-language classifier: CLIP / OpenCLIP / MobileCLIP (region embedding + text embedding matching)

LLM agent (recipe selection + instruction generation using detected ingredients)

Communication interface: Wi-Fi (Quest ↔ Jetson) for frames/instructions; optional UART/I2C/USB (ESP32 ↔ Jetson) for weight/safety telemetry

## Subsystem 3 — Environmental Sensor Subsystem (Weight + Room Temperature)

This subsystem measures ingredient mass for closed-loop dispensing and ambient temperature for environmental context (improving recipes by understanding room temperature) and essential fire hazard checks. The ESP32 reads both sensors, filters/calibrates the data (tare + scale factor), and forwards measurements to the main controller.

### Sensors/Components

Load cell(s): 5 kg single-point load cell (e.g., TAL220B 5kg) or 4× half-bridge load cells (for a round platform)

Load cell ADC/Amplifier: HX711 24-bit load cell amplifier breakout/module (HX711)

Ambient temperature sensor: SHT31-DIS (Sensirion SHT31-DIS-B) or BME280 (Bosch BME280) over I2C

MCU interface: ESP32 reads HX711 (GPIO clock/data) + temp sensor (I2C)

## Subsystem 4 — Battery Subsystem (Rechargeable Power for Portability)

This subsystem powers the device from a rechargeable battery, supports charging via USB, and generates stable rails for logic and actuation. It provides regulated 3.3 V for ESP32/sensors and 5–6 V for servo/actuators while preventing brownouts during motor current spikes.

### Sensors/Components

Battery pack: 1-cell LiPo, 3.7 V (e.g., 2000–5000 mAh, JST-PH)

Battery charger: TP4056 1S Li-ion charger module (with protection variant preferred) or MCP73831 (Microchip MCP73831T)

3.3 V regulator (logic rail): Buck converter module (e.g., MP1584EN) set to 3.3 V or LDO if current is small

5–6 V regulator (servo rail): Buck converter module (e.g., MP1584EN) set to 5.0–6.0 V

Power monitoring (optional but helpful): MAX17048 LiPo fuel gauge (MAX17048G+U) over I2C

Protection/robustness: power switch + fuse/polyfuse + bulk capacitors near servo rail

## Subsystem 5 — Rotating Carousel Ring + Dispenser Subsystem (One-Piece Circular Device)

This subsystem’s purposing on seasoning and provides the “lazy Susan” mechanism: a circular rotating ring that indexes ingredient pods to a fixed dispense station above the scale. A motor rotates the ring to the selected pod; a servo opens a gate to dispense into the center bowl. The ESP32 controls indexing, homing, and dispensing, using the scale feedback to stop at a target mass.

### Sensors/Components

Rotation motor: NEMA 17 stepper motor (e.g., 42BYGH-class)

Stepper driver: DRV8825 or A4988 stepper driver module

Homing/index sensor: A3144 Hall-effect sensor + small neodymium magnet (defines “slot 0”)

Dispense actuator: MG90S micro servo (metal gear) or SG90 (lighter duty)

Mechanical drive: GT2 timing belt + pulley set (e.g., GT2 6mm belt, 20T pulley) or friction wheel drive

Ring support: lazy-susan bearing (turntable bearing) or printed rail + small rollers/V-wheels

Dispense hardware: fixed chute + passive pod gate (flap/valve) engaged by the servo at the station

# Criterion For Success

We’d like to set the criterion for success with numerical benchmarks:

The sensor for weight measurement should have a resolution of 1 gram and a gram error range of 0 to 100 grams.

Fully chargeable battery offering approximately 20 minutes of runtime.

The rotating carousel dispenser should spin to the desired spot within 2 error.

Ingredient classification and localization accuracy >= 85% with 10 FPS real-time performance.

Eventually, the pipeline of feeding input from a VR headset (visual image stream) and sensors (hardware) to the vision processor and returning a contextual recipe back to the VR headset should be completed.

## References:

[1]: Hasenböhler, Anaïs et al. “Intake of food additive preservatives and incidence of cancer: results from the NutriNet-Santé prospective cohort.” BMJ (Clinical research ed.) vol. 392 e084917. 7 Jan. 2026, doi:10.1136/bmj-2025-084917

Project Videos