Projects


# Title Team Members TA Professor Documents Sponsor
1
Sound Asleep
Adam Tsouchlos
Ambika Mohapatra
Shub Pereira
Weiman Yan Rakesh Kumar presentation1.pdf
proposal1.pdf
2
Autonomous Car for WiFi Mapping
Avi Winick
Ben Maydan
Josh Powers
Jason Jung Arne Fliflet proposal1.pdf
3
Follow-Me Cart: App controlled smart assistant
Alex Huang
Jiaming Gu
Shi Qiao
Shengkun Cui Arne Fliflet other1.pdf
proposal1.pdf
4
Champaign MTD Bus Tracker Map
Amber Wilt
Daniel Vlassov
Ziad AlDohaim
Wesley Pang Arne Fliflet other1.pdf
proposal1.pdf
5
Navigation Vest Suite For People With Eye Disability
Haoming Mei
Jiwoong Jung
Pump Vanichjakvong
Rishik Sathua Cunjiang Yu proposal1.pdf
6
E-Bike Crash Detection and Safety
Adam Arabik
Ayman Reza
Muhammad Daniyal Amir
Eric Tang Arne Fliflet proposal1.pdf
7
Omnidirectional Drone
Dhruv Satish
Ivan Ren
Mahir Koseli
Jason Zhang Arne Fliflet proposal1.pdf
8
Hybrid Actuation Arm Exoskeleton
Alan Lu
Rubin Du
Haocheng Bill Yang Cunjiang Yu proposal1.pdf
9
Ant Weight 3-D Printed BattleBot
John Tian
Mig Umnakkittikul
Yanhao Yang
Gayatri Chandran Rakesh Kumar proposal1.pdf
10
NeuroBand
Arrhan Bhatia
Vansh Vardhan Rana
Vishal Moorjani
Wenjing Song Cunjiang Yu proposal1.pdf
11
Glove Controlled Drone
Aneesh Nagalkar
Atsi Gupta
Zach Greening
Wenjing Song Cunjiang Yu proposal1.pdf
12
New Generation Addiction Control and Recovery Device System with Absolute Safety and Privacy - working with the Drug Addiction Research Team
Adrian Santosh
Leo Li
Richawn Bernard
Shengyan Liu Cunjiang Yu proposal1.pdf
13
Sun Tracking Umbrella
Dora Stavenger
Megan Cubiss
Sarah Wilson
Wesley Pang Arne Fliflet proposal1.pdf
14
Enhanced Golf Rangefinder
Peter Maestranzi
Emma DiBiase
Jacob Hindenburg
Eric Tang Arne Fliflet presentation1.pdf
proposal1.pdf
15
Auto adjusted lighting system for room
Howard Li
Jihyun Seo
Kevin Chen
Zhuoer Zhang Arne Fliflet proposal1.pdf
16
Antweight Battlebot - Blade Blade
Jack Tipping
Patrick Mugg
Sam Paone
Gayatri Chandran Rakesh Kumar other1.pdf
presentation1.pptx
proposal1.pdf
proposal2.pdf
17
LED Persistence of Vision Globe
Gavi Campbell
Melvin Alpizar Arrieta
Owen Bowers
Gayatri Chandran Arne Fliflet other1.pdf
18
RFID Poker Board
Darren Liao
KB Bolor-Erdene
Satyam Singh
Eric Tang Rakesh Kumar proposal1.pdf
19
Suction Sense - Pitch Project
Hugh Palin
Jeremy Lee
Suleymaan Ahmad
Lukas Dumasius Cunjiang Yu other2.pdf
presentation1.pdf
proposal1.pdf
20
Glove controlled mouse with haptic feedback
Khushi Kalra
Vallabh Nadgir
Vihaansh Majithia
Frey Zhao Rakesh Kumar proposal1.pdf
21
MULTI-SENSOR MOTION DETECTOR FOR RELIABLE LIGHTING CONTROL
Joseph Paxhia
Lukas Ping
Sid Boinpally
Shiyuan Duan Cunjiang Yu proposal1.pdf
22
Adherascent
Dhiraj Dayal Bijinepally
Hardhik Tarigonda
Jonathan Liu
Shiyuan Duan Cunjiang Yu proposal1.pdf
23
Drink Dispensing Robot
Andrew Jung
Ethan Cao
Megan Cheng
Frey Zhao Rakesh Kumar proposal1.png
proposal2.png
proposal3.pdf
24
Autonomous Cylindrical Root Camera
Aidan Veldman
Nathaniel McGough
Zach Perkins
Rishik Sathua Rakesh Kumar proposal1.pdf
25
Auto-Guitar Tuner
Daniel Cho
Ritvik Patnala
Timothy Park
Eric Tang Rakesh Kumar proposal1.pdf
26
Orion Med
wenhao Zhang
XiangYi Kong
Yuxin Zhang
Zhuoer Zhang Rakesh Kumar proposal1.pdf
27
Team Heart Restart
Brian Chiang
Ethan Moraleda
Will Mendez
Frey Zhao Arne Fliflet proposal1.pdf
28
Real-time EEG Drowsiness Detection Device
Nikhil Talwalkar
Senturran Elangovan
Zhuoer Zhang Arne Fliflet proposal1.pdf
29
Modular Wafer Track for Semiconductor Fabrication
Hayden Kunas
Jack Schnepel
Nathan Pitsenberger
Shengyan Liu Rakesh Kumar proposal1.pdf
30
Transverse String Organ
Ash Huang
Eddy Perez
Kellen Sakaitani
Shengyan Liu Rakesh Kumar proposal1.pdf
31
NueroGaurd
Aidan Moran
Alexander Krejca
Stephen Simberg
Shiyuan Duan Cunjiang Yu other1.pdf
proposal1.pdf
32
Insight: Cardiovascular Screening Device
Ethan Pereira
Jay Nathan
Rishab Iyer
Weiman Yan Cunjiang Yu proposal1.pdf
33
Budget Clip-On Posture Checker
Ashit Anandkumar
Destiny Jefferson
Edward Ruan
Wenjing Song Cunjiang Yu proposal1.pdf

Smart Glasses for the Blind

Siraj Khogeer, Abdul Maaieh, Ahmed Nahas

Smart Glasses for the Blind

Featured Project

# Team Members

- Ahmed Nahas (anahas2)

- Siraj Khogeer (khogeer2)

- Abdulrahman Maaieh (amaaieh2)

# Problem:

The underlying motive behind this project is the heart-wrenching fact that, with all the developments in science and technology, the visually impaired have been left with nothing but a simple white cane; a stick among today’s scientific novelties. Our overarching goal is to create a wearable assistive device for the visually impaired by giving them an alternative way of “seeing” through sound. The idea revolves around glasses/headset that allow the user to walk independently by detecting obstacles and notifying the user, creating a sense of vision through spatial awareness.

# Solution:

Our objective is to create smart glasses/headset that allow the visually impaired to ‘see’ through sound. The general idea is to map the user’s surroundings through depth maps and a normal camera, then map both to audio that allows the user to perceive their surroundings.

We’ll use two low-power I2C ToF imagers to build a depth map of the user’s surroundings, as well as an SPI camera for ML features such as object recognition. These cameras/imagers will be connected to our ESP32-S3 WROOM, which downsamples some of the input and offloads them to our phone app/webpage for heavier processing (for object recognition, as well as for the depth-map to sound algorithm, which will be quite complex and builds on research papers we’ve found).

---

# Subsystems:

## Subsystem 1: Microcontroller Unit

We will use an ESP as an MCU, mainly for its WIFI capabilities as well as its sufficient processing power, suitable for us to connect

- ESP32-S3 WROOM : https://www.digikey.com/en/products/detail/espressif-systems/ESP32-S3-WROOM-1-N8/15200089

## Subsystem 2: Tof Depth Imagers/Cameras Subsystem

This subsystem is the main sensor subsystem for getting the depth map data. This data will be transformed into audio signals to allow a visually impaired person to perceive obstacles around them.

There will be two Tof sensors to provide a wide FOV which will be connected to the ESP-32 MCU through two I2C connections. Each sensor provides a 8x8 pixel array at a 63 degree FOV.

- x2 SparkFun Qwiic Mini ToF Imager - VL53L5CX: https://www.sparkfun.com/products/19013

## Subsystem 3: SPI Camera Subsystem

This subsystem will allow us to capture a colored image of the user’s surroundings. A captured image will allow us to implement egocentric computer vision, processed on the app. We will implement one ML feature as a baseline for this project (one of: scene description, object recognition, etc). This will only be given as feedback to the user once prompted by a button on the PCB: when the user clicks the button on the glasses/headset, they will hear a description of their surroundings (hence, we don’t need real time object recognition, as opposed to a higher frame rate for the depth maps which do need lower latency. So as low as 1fps is what we need). This is exciting as having such an input will allow for other ML features/integrations that can be scaled drastically beyond this course.

- x1 Mega 3MP SPI Camera Module: https://www.arducam.com/product/presale-mega-3mp-color-rolling-shutter-camera-module-with-solid-camera-case-for-any-microcontroller/

## Subsystem 4: Stereo Audio Circuit

This subsystem is in charge of converting the digital audio from the ESP-32 and APP into stereo output to be used with earphones or speakers. This included digital to audio conversion and voltage clamping/regulation. Potentially add an adjustable audio option through a potentiometer.

- DAC Circuit

- 2*Op-Amp for Stereo Output, TLC27L1ACP:https://www.ti.com/product/TLC27L1A/part-details/TLC27L1ACP

- SJ1-3554NG (AUX)

- Connection to speakers/earphones https://www.digikey.com/en/products/detail/cui-devices/SJ1-3554NG/738709

- Bone conduction Transducer (optional, to be tested)

- Will allow for a bone conduction audio output, easily integrated around the ear in place of earphones, to be tested for effectiveness. Replaced with earphones otherwise. https://www.adafruit.com/product/1674

## Subsystem 5: App Subsystem

- React Native App/webpage, connects directly to ESP

- Does the heavy processing for the spatial awareness algorithm as well as object recognition or scene description algorithms (using libraries such as yolo, opencv, tflite)

- Sends audio output back to ESP to be outputted to stereo audio circuit

## Subsystem 6: Battery and Power Management

This subsystem is in charge of Power delivery, voltage regulation, and battery management to the rest of the circuit and devices. Takes in the unregulated battery voltage and steps up or down according to each components needs

- Main Power Supply

- Lithium Ion Battery Pack

- Voltage Regulators

- Linear, Buck, Boost regulators for the MCU, Sensors, and DAC

- Enclosure and Routing

- Plastic enclosure for the battery pack

---

# Criterion for Success

**Obstacle Detection:**

- Be able to identify the difference between an obstacle that is 1 meter away vs an obstacle that is 3 meters away.

- Be able to differentiate between obstacles on the right vs the left side of the user

- Be able to perceive an object moving from left to right or right to left in front of the user

**MCU:**

- Offload data from sensor subsystems onto application through a wifi connection.

- Control and receive data from sensors (ToF imagers and SPI camera) using SPI and I2C

- Receive audio from application and pass onto DAC for stereo out.

**App/Webpage:**

- Successfully connects to ESP through WIFI or BLE

- Processes data (ML and depth map algorithms)

- Process image using ML for object recognition

- Transforms depth map into spatial audio

- Sends audio back to ESP for audio output

**Audio:**

- Have working stereo output on the PCB for use in wired earphones or built in speakers

- Have bluetooth working on the app if a user wants to use wireless audio

- Potentially add hardware volume control

**Power:**

- Be able to operate the device using battery power. Safe voltage levels and regulation are needed.

- 5.5V Max

Project Videos