Meeting with Your TA

Description

By the Thursday of the third week, you must have a project approved, and should be ready to get working! At this time, you'll need to log into PACE and submit your schedule for the semester. Please be sure to make this as accurate as possible because once it's submitted, it can only be changed manually. Making a block of your schedule red means that you are unavailable during that time.

Once each person on your team has submitted their schedule, your TA will be able to easily check for available times to schedule a weekly meeting. Your TA should contact you, usually by the fourth week, via email, to set up a weekly meeting schedule at mutual convenience. During the first weekly meeting, your TA will assign your team a locker and a lab kit.

Weekly meetings with your TA are required and will be held throughout the entire semester until demonstrations are completed. Your TA is your project manager. The "homework" of the course consists of preparing for the weekly meetings. Your TA will evaluate your lab notebook each week, provide feedback, and recommend improvements. At each meeting you will be expected to present your progress since your last meeting, plans for the coming week, and any technical or administrative questions you need to discuss with your TA. You are expected to arrive on time and prepared to make good use of your time with your TA. Your TA may require that each team member to fill out the Progress Report Template and submit it to them prior to each weekly meeting.

Requirements and Grading

Attendance and participation in weekly meetings is required and will affect Teamwork and Lab Notebook scores. If you can't make it to a particular weekly meeting, it is your responsibility to inform your TA prior to the meeting time and set up an alternate time.

Submission and Deadlines

Your schedule must be submitted by the end of the third week of class and you will receive an email from your TA shortly after. Your first meeting with your TA should be during the fourth week of the semester.

Navigation Vest Suite For People With Eye Disability

Jiwoong Jung, Haoming Mei, Pump Vanichjakvong

Navigation Vest Suite For People With Eye Disability

Featured Project

# Navigation Vest Suite For People With Eye Disability

Team Members & Experiences:

- Jiwoong Jung (jiwoong3): Experienced in Machine Learning, and some Embedded programming. Worked on many research and internships that requires expertise in Machine Learning, Software Engineering, Web Dev., and App Dev. Had some experience with Embedded programming for Telemetry.

- Haoming Mei (hmei7): Experienced in Embedded programming and PCB design. Worked with projects like lights, accelerometer, power converter, High-Fet Board, and motor control for a RSO that involve understanding of electronics, PCB design, and programming with STM32 MCUs.

- Pump Vanichjakvong (nv22): Experienced with Cloud, Machine Learning, and Embedded programming. Done various internships and classes that focuses on AI, ML, and Cloud. Experience with Telemetry and GPS system from RSO that requires expertises in SPI, UART, GPIOs, and etc with STM32 MCUs.

# Problem

People with Eye Disability often face significant challenges navigating around in their daily lives. Currently, most available solutions ranges like white canes and guide dogs to AI-powered smart glasses, many of which are difficult to use and can cost as much as $3,000. Additionally, problems arises for people with disability, especially in crowded/urban areas, and that includes injuries from collision with obstacles, person, or from terrains. According to the U.S department of Transportation's 2021 Crash Report , 75% of pedestrian fatalities occurred at locations that were not intersections. Thus we aim to design a navigation vest suite to help people with eye disability to deal with these issues.

https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/813458.pdf

# Solution

We have devised a solution which helps ease visually impaired individuals in daily activities such as walking from two places, or navigating around a building with multiple obstacles. Our focus will for out-door navigation in urban areas, where obstacles, terrain, and pedestrians. But, if time permits we will also deal with traffics and crosswalks.

In order to achieve this, we will be utilizing 3 main components:

- Lidar sensors to help the wearer with depth perception tasks

- Vibration Motors to aid navigation (turning left/right)

- Magnetometer to enable more accurate GPS coordination

All the above components will contribute to the sensory fusion algorithm.

# Solution Components

## Subsystem 1

### Microcontroller System

We are planning to use a STM32 microcontroller as main processing unit for sensory data from lidar sensors (magnetometer and GPS if time permits) and object detection data from the **machine learning system**, and direction data from navigation app (our design on phone). We will use these information to generate vibration in the direction the wearer should navigate.

### Power Systems

The whole system will be battery-powered by a battery module, which contains 5V battery-cells. It will be connected to the **Microcontroller System**, which will also supply it to the **Machine Learning System**. We will also implement the necessary power protection, buck converter, regulator, and boost converters as necessary per sensors or components.

- Battery Module Pack

- Buck Converter (Step-Down)

- Boost Converter (Step-Up)

- Voltage Regulator

- Reverse Polarity Protection

- BMS

## Subsystem 2

### Navigation Locator Systems

Our navigation system will consist of an App which directly connects to the Google Maps API, paired with our existing sensors. We plan to utilize a magnetometer sensor, which will indicate the direction the user is facing (North, South, East, West, .etc). In order to pinpoint which direction the wearer needs to be heading, our built-in LiDAR sensors will enable us to create a SLAM (Simultaneous Localization and Mapping) to build a map of the environment. With these systems in place, we would be able to assist users in navigation. To deal with Terrain hazards, we will use the LiDAR to sensors to assist us in dealing with elevation changes the user needed to make.

- LiDAR

- Android App (Connected to Google API)

- Magnetometer

- Vibration Motors

Extra Features (if time permits):

- Audio Output (Text to Speech Generation from Raspberry PI 5 sends to microcontroller through AUX cable )

## Subsystem 3

### Machine Learning Systems

- We plan to employ a object detection model on a 16GB Raspberry PI 5 (already) along with a PI camera to detect objects, signs, and people on the road, which will be feed to the microcontroller

- Raspberry PI 5

- PI Camera

a) The image video model will be expected to be less than 5 billion parameters with convolutional layers, running on device in raspberry pi Obviously the processing power on the raspberry pi is expected to be limited, however we are planning to accept the challenge and find out ways to improve the model with limited hardware capabilities.

b) If the overall project for subtask a) becomes arduous or time consuming, we can utilize api calls or free open source models to process the image/video in real time if the user wants the feature enabled. The device is paired with the phone via the wifi chip on the raspberry pi to enable the API call. Some of the best candidates we can think of are the YOLO family models, MMDetection and MMTracking toolkit, or Detectron2 model that is developed by Facebook AI Research that supports real time camera feedbacks.

# Criterion For Success

### Navigational Motor/Haptic Feedback

1) The Haptic feedback (left/right vibration) should perfectly match with the navigation directions received from the app (turn left/right)

2) Being able to Detect Obstacles, Stairs, Curbs, and people.

3) Being able to detect intersections infront and the point of turn through the lidar sensory data.

4) Being able to obey the haptic feedback patterns that is designed. (tap front for walking forward, tap right to go right etc...)

### Object Detection

1) Using the Illinois Rules of the Road and the Federal Manual on Uniform Traffic Control Device Guidelines, we will be using total of 10-30 distinct pedestrian road signs to test the object detection capability. We will be using a formal ML testing methods like geometric transformations, photometric transformations, and background clutter. Accuracy will be measured by the general equation (Total Number of correctly classified Datasets)/(Total Number of Datasets)

2) The ML Model should be able to detect potential environmental hazards including but not limited to Obstacles, Stairs, Curbs, and people. We are planning onto gather multiple hazard senarios via online research, surveys, and in-person interviews. Based on the collected research, we will be building solid test cases to ensure that our device can reliably identify potential hazards. More importantly, we are planning design strict timing and accuracy measures metrics.

3) The ML model should be able to detect additional road structures such as curbs, crosswalks, and stairs to provide comprehensive environment awareness. We will be utilizing different crosswalks located on north quad and utilize the accuracy measurement techniques mentioned in 1)

### Power and Battery Life

1) The device should support at least 3 hours of battery life.

2) The device should obey the IEC 62368-1 safety standard. IEC 62368-1 safety standard lays out different classes such as ES1, ES2, and ES3 that considers electrical and flame

Project Videos