Project

# Title Team Members TA Documents Sponsor
6 SMART GLASSES FOR HANDS-FREE DATASHEET RETRIEVAL
Hridik Hingorani
Preity Varanasi
Shiv Bahl
Yushi Zhou design_document1.pdf
proposal1.pdf
Team Members:

Hridik Hingorani

Shiv Bahl

Preity Varanasi

Problem

When working with electronic hardware, engineers frequently need to stop what they are doing to identify a component, search for its datasheet, and locate specific information such as pinouts, voltage limits, timing specifications, or recommended operating conditions.
This interrupts the workflow and is particularly inconvenient when the user is soldering, probing a circuit, or otherwise using both hands.
Our goal is to reduce this interruption by creating a low-cost wearable system that can identify an electronic component being viewed by the user and automatically retrieve the corresponding datasheet.
Solution
We propose building a pair of smart glasses containing a camera and a custom embedded PCB.
When the user looks at a component and activates the system, the camera will capture an image and send it wirelessly to a laptop. The laptop will use computer vision and OCR to extract identifying information such as the component's part number or package markings.
Once the component has been identified, the laptop will locate the appropriate datasheet and process it so that important information can be retrieved quickly. The result will initially be displayed on the laptop.
The system is intended primarily for clearly labeled electronic components such as integrated circuits, modules, sensors, and other devices whose identifying markings can be captured by the camera.

High-Level System

Smart Glasses
Camera → Embedded PCB → Wi-Fi Communication
↓
Laptop
Image Processing/OCR → Component Identification → Datasheet Retrieval → Datasheet Processing → User Output

Hardware

We will design a custom PCB mounted on or integrated into the glasses.

The PCB will include:
ESP32-based microcontroller
Camera interface
Wireless communication
User input for triggering image capture
Power regulation
Battery/power management
Necessary supporting circuitry
The embedded system will be responsible for capturing images, managing the camera, handling user input, and wirelessly transmitting data to the laptop.
The glasses themselves will be inexpensive commercially available frames modified to hold our electronics.

Software

The laptop-side software will:
Receive the image from the glasses.
Process the image to improve readability of component markings.
Use OCR/computer vision to determine the component identifier.
Search for and retrieve the correct datasheet.
Parse the datasheet and make relevant specifications accessible to the user.
Display the component identity, datasheet, and requested information on the laptop.
The LLM will only answer questions using information retrieved from the identified component's datasheet rather than relying solely on its existing knowledge.

Requirements

For a successful final demonstration, the system should:
Capture a usable image from the wearable camera.
Wirelessly transfer the image from the glasses to the laptop.
Correctly identify a predefined set of clearly marked electronic components.
Retrieve the correct datasheet associated with the identified component.
Extract and display relevant information from that datasheet.
Operate using our custom PCB rather than a standalone commercial development board.
Be wearable and operate without a wired connection between the glasses and laptop.
We will create a test set of electronic components and quantitatively evaluate component-identification accuracy and end-to-end response time.

Stretch Goal

Our stretch goal is to make the system fully hands-free after component identification by adding voice interaction.
The user would be able to ask questions such as:
"What is the maximum supply voltage?"
"What does pin 4 do?"
"What value capacitor does the manufacturer recommend here?"
The system would transcribe the question, search the retrieved datasheet, and provide an answer grounded specifically in that datasheet.

Future Work

A future version could integrate a small near-eye or AR display into the glasses so that information could be presented directly in the user's field of view.
An AR display is not part of the scope of this semester's project.
Complexity
The project combines several independently testable hardware and software subsystems:
Custom wearable PCB design
Camera interfacing
Battery and power-management circuitry
Wireless embedded communication
Image processing
OCR/component identification
Automated datasheet retrieval
Datasheet parsing and information extraction
Integration between the embedded hardware and laptop software
A major technical challenge will be reliably extracting part markings from small electronic components under different viewing angles, distances, orientations, and lighting conditions.

Uniqueness

Existing smart glasses and visual assistants are generally designed for broad image recognition or general-purpose AI assistance.
Our project is specifically designed around electronics work. Instead of simply describing what the camera sees, the system will identify a specific electronic component, locate its technical documentation, and provide information grounded in the manufacturer's datasheet.
The project therefore combines a purpose-built wearable embedded platform with a specialized datasheet retrieval and processing pipeline.

Scope

The core project will focus on labeled electronic components and laptop-based output. We will not attempt to recognize every possible component or construct an AR display during this semester.
Restricting the identification problem to a controlled but varied set of components allows us to quantitatively evaluate the system while still addressing the major technical challenges of wearable image acquisition, wireless communication, component identification, datasheet retrieval, and system integration.

Waste Bin Monitoring System

Benjamin Gao, Matt Rylander, Allen Steinberg

Featured Project

# Team Members:

- Matthew Rylander (mjr7)

- Allen Steinberg (allends2)

- Benjamin Gao (bgao8)

# Problem

Restaurants produce large volumes of waste every day which can lead to many problems like overflowing waste bins, smelly trash cans, and customers questioning the cleanliness of a restaurant if it is not dealt with properly. Managers of restaurants value cleanliness as one of their top priorities. Not only is the cleanliness of restaurants required by law, but it is also intrinsically linked to their reputation. Customers can easily judge the worth of a restaurant by how clean they keep their surroundings. A repulsive odor from a trash can, pests such as flies, roaches, or rodents building up from a forgotten trash can, or even just the sight of a can overflowing with refuse can easily reduce the customer base of an establishment.

With this issue in mind, there are many restaurant owners and managers that will likely purchase a device that will help them monitor the cleanliness of aspects of their restaurants. With the hassle of getting an employee to leave their station, walk to a trash can out of sight or far away, possibly even through external weather conditions, and then return to their station after washing their hands, having a way to easily monitor the status of trash cans from the kitchen or another location would be convenient and save time for restaurant staff.

Fullness of each trash can isn’t the only motivating factor to change out the trash. Maybe the trash can is mostly empty, but is extremely smelly. People are usually unable to tell if a trash can is smelly just from sight alone, and would need to get close to it, open it up, and expose themselves to possible smells in order to determine if the trash needs to be changed.

# Solution

Our project will have two components: 1. distributed sensor tags on the trash can, and 2. A central hub for collecting data and displaying the state of each trash can.

The sensor tags will be mounted to the top of a waste bin to monitor fullness of the can with an ultrasonic sensor, the odor/toxins in the trash with an air quality/gas sensor, and also the temperature of the trash can as high temperatures can lead to more potent smells. The tags will specifically be mounted on the underside of the trash can lids so the ultrasonic sensor has a direct line of sight to the trash inside and the gas sensor is directly exposed to the fumes generated by the trash, which are expected to migrate upward past the sensor and out the lid of the can.

The central hub will have an LCD display that will show all of the metrics described in the sensor tags and alert workers if one of the waste bins needs attention with a flashing LED. The hub will also need to be connected to the restaurant’s WiFi.

This system will give workers one less thing to worry about in their busy shifts and give managers peace of mind knowing that workers will be warned before a waste bin overflows. It will also improve the customer experience as they will be much less likely to encounter overflowing or smelly trash cans.

# Solution Components

## Sensor Tag Subsystem x2

Each trash can will be fitted with a sensor tag containing an ultrasonic sensor transceiver pair, a hazardous gas sensor, a temperature sensor, an ESP32 module, and additional circuitry necessary for the functionality of these components. The sensors will be powered with 3.3V or 5V DC from a wall adapter. A small hole will need to be drilled into the side of each trash can to accommodate the wall adapter output cord. They may also need to be connected to the restaurant’s WiFi.

- 2x ESP32-S3-WROOM

https://www.digikey.com/en/products/detail/espressif-systems/ESP32-S3-WROOM-1-N16R2/16162644

- 2x Air Quality Sensor (ZMOD4410)

https://www.digikey.com/en/products/detail/renesas-electronics-corporation/ZMOD4410AI1R/8823799

- 2x Temperature/Humidity Sensor(DHT22)

https://www.amazon.com/HiLetgo-Digital-Temperature-Humidity-Replace/dp/B01DA3C452?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A30QSGOJR8LMXA#customerReviews

- 2x Ultrasonic Transmitter/Receiver

https://www.digikey.com/en/products/detail/cui-devices/CUSA-R75-18-2400-TH/13687422

https://www.digikey.com/en/products/detail/cui-devices/CUSA-T75-18-2400-TH/13687404

## Central Hub Subsystem

The entire system will be monitored from a central hub containing an LCD screen, an LED indicator light, and additional I/O modules as necessary. It will be based around an ESP32 module connected to the restaurant’s WiFi or ESPNOW P2P protocol that communicates with the sensor tags. The central hub will receive pings from the sensor tags at regular intervals, and if the central hub determines that one or more of the values (height of trash, air quality index, or temperature) are too high, it will notify the user. This information will be displayed on the hub’s LCD screen and the LED indicator light on the hub will flash to alert the restaurant staff of the situation.

- 1x ESP32-S3-WROOM

https://www.digikey.com/en/products/detail/espressif-systems/ESP32-S3-WROOM-1-N16R2/16162644

- 1x LCD Screen

https://www.amazon.com/Hosyond-Display-Compatible-Mega2560-Development/dp/B0BWJHK4M6/ref=sr_1_4?keywords=3.5%2Binch%2Blcd&qid=1705694403&sr=8-4&th=1

# Criteria For Success

This project will be successful if the following goals are met:

- The sensor tags can detect when a trash can is almost full (i.e. when trash is within a few inches of the lid) and activate the proper protocol in the central hub.

- The sensor tags can detect when an excess of noxious fumes are being produced in a trash can and activate the proper protocol in the central hub.

- The sensor tags can detect when the temperature in a trash can has exceeded a user-defined threshold and activate the proper protocol in the central hub.

- The central hub can receive wireless messages from all sensor tags reliably and correctly identify which trash cans are sending the messages.

Project Videos