Project
| # | Title | Team Members | TA | Documents | Sponsor |
|---|---|---|---|---|---|
| 26 | PATH: Proximity Awareness Through Haptics |
Juheon Woo Kori Gillis-Harry Zeead Sowelam |
Eric Tang | ||
| **Proximity Awareness Through Haptics (PATH)** **Team Members:** - Juheon Woo (juheonw2) - Zeead Sowelam (sowelam2) - Kori Gillis-Harry (korimg2) **Problem** While new assistance devices are being developed, the function of and accessibility to those devices are still extremely limited, and most visually impaired individuals rely only on a white cane or mobile applications that are oftentimes inconvenient to put into action passively at all times. Individuals often face difficulty and delay in avoiding or being notified of nearby obstacles or people in a day-to-day environment. **Solution** In order to help resolve such difficulties faced by visually impaired individuals, we propose an assistive obstacle warning and vibration device that can be attached to a white cane to help the user preemptively avoid approaching obstacles. Depending on the approximate proximity of the obstacle ahead, the device will either vibrate stronger or weaker. The handle will have 2 vibration motors, one on the left side of the handle and another on the right side of the handle. Each vibration motor is attached to the actual handle of the cane through sticker-like pads, one pad on the left side and one pad on the right. The left vibration motor will go into action when the obstacle is approaching from the left and the right vibration motor will go into action when the obstacle is approaching from the right, and this will be felt by the user through the pads attached to the handle. When the obstacle is straight ahead, both motors will vibrate, with the strength of the vibration corresponding to the distance from the user to the obstacle. The detection device that will generate the vibration signal will consist of a multi-zone time-of-flight (ToF) sensor paired with a 6-axis IMU for dynamic tilt compensation. The sensor array will provide directional haptic feedback; vibrating a specific motor corresponding to the zone (left or right) where the obstacle is detected, with intensity scaling based on distance. Real-time orientation data from the IMU dynamically adjusts the sensor grid to compensate for cane angle changes, preserving forward-looking accuracy. This removes the need for the user to manually sweep the device to localize obstacles, prevents false detections due to normal cane motion, and improves on the spatial resolution of existing single-sensor commercial products. **How We Improve on Prior Work** A Spring 2026 team built a modular smart cane attachment (Project 15) that relayed phone navigation audio, provided basic distance-warning vibration cues, and utilized a standard linear power supply. While we retain the foundational goal of improving obstacle awareness for visually impaired users, our design significantly upgrades the system's functionality (especially in terms of local obstacle detection and notification), hardware, firmware, and circuit complexity in three key ways. First, obstacle detection and spatial sensor fusion. Project 15 relied on a single-point distance sensor that could only report a generic distance to an obstacle somewhere ahead, offering no spatial resolution in terms of local obstacle detection. Project 15 does incorporate 2 motors for left and right haptic signals, however, this is strictly to signal left and right turns when navigating a predefined path, and the obstacle awareness/detection itself remains simple and does not provide any complex spatial image to the user. In contrast, our design replaces this 1D single beam with an 8x8 multi-zone ToF matrix, providing 64 independently ranged depth zones. This enables true spatial mapping, allowing us to selectively actuate left or right grip motors so the haptic cue actively points to the hazard rather than just issuing a passive warning with not much spatial information. In addition, our design will also have a special vibration pattern for overhead objects, which is not supported in project 15. Furthermore, we integrate a 6-axis IMU running real-time orientation filtering to dynamically shift active matrix rows as the cane tilts. Project 15 uses its IMU passively for fall detection logic. Our project implements active sensor fusion, combining IMU pitch/roll orientation data with ToF spatial matrix readings via a complementary filter to dynamically re-index active matrix rows. This holds the detection window locked at an upper-body height regardless of grip angle and eliminates false ground trigger, which is an advanced sensor-fusion capability absent in both Project 15 and commercial products like the WeWalk cane. Second, actuator reliability and closed-loop haptic control. Beyond sensing, we add closed-loop fault detection on the output side: each vibration motor is driven through a discrete MOSFET switching stage with low-side current sensing, so the firmware can distinguish a working motor from a stalled or disconnected one in real time via ADC sampling. Additionally, we implement a short initial 100% duty-cycle overdrive pulse to overcome motor mechanical inertia and reduce startup latency. This is a custom circuit-level contribution with no equivalent in Project 15, where the vibration output relied on basic open-loop PWM driving with no diagnostic feedback. Third, power management and hardware circuit complexity. Project 15 relied on a basic LDO regulator and a simple resistor divider into an ADC pin to estimate battery percentage. Our design increases custom hardware complexity by replacing the LDO with a high-efficiency synchronous Buck-boost switching regulator for system power, integrating a dedicated I2C fuel gauge IC for precise state-of-charge tracking, and incorporating a standalone battery protection IC circuit with dual MOSFET switching. This delivers significantly better power efficiency, professional-grade battery monitoring, and robust circuit protection. In addition, we also added a low battery haptic notification system that notifies the user of low battery, which is not done in project 15. In short, our project focuses mainly on the complex obstacle detection aspect, rather than focusing on general navigation assistance along a predefined path like project 15. With this goal, we also introduce much sophisticated hardware and feature implementations like mentioned above. **Solution Components (System Architecture and Subsystems)** - Sensor & Orientation Array: STMicroelectronics VL53L5CX multi-zone Time-of-Flight sensor (8x8 zone output, up to 4m theoretical range but scaled to 2m reliable operating range) and a 6-axis IMU (like LSM6DS3TR-C), both interfaced via a shared I2C bus alongside dedicated hardware interrupt lines, mounted near the top of the handle facing forward/outward. - Processor: ESP32 module (ESP32-S3-MINI dual core module), surface-mounted directly on the custom PCB (not a dev board), housed in a small enclosure on the cane shaft below the handle. - Haptic Output & Current-Sense Stage: 2 ERM (eccentric rotating mass) coin vibration motors, one adhesive-mounted to the left of the handle and one to the right, each driven by a discrete N-channel MOSFET PWM switching circuit with flyback diode protection and low-side current sensing for motor fault and stall detection. Motors will be attached with some type of strong adhesive with wiring routed along the shaft to the PCB. - Power Management: Li-Po (or Li-ion) battery, MCP73831 charge management IC, DW01A battery protection circuit with dual MOSFET switch, USB-C charging input with ESD protection, high-efficiency synchronous Buck-boost switching regulator (like the TPS63020 or the TPS63070) for 3.3V logic supply, and an I2C battery fuel gauge IC (e.g., MAX17048); housed in the same shaft enclosure as the processor. **Core Subsystems Requiring Custom Circuit Design** - Power Management & Regulation Subsystem: Battery charging, protection, high-efficiency Buck-boost regulation, and I2C fuel gauge state-of-charge monitoring. - Closed-Loop Haptic Motor Drive Subsystem: Custom MOSFET PWM switching stage per pad with flyback protection, 100% duty-cycle startup pulse, and low-side current-sense amplification for real-time stall and fault detection. - Multi-Sensor & Spatial Mapping Subsystem: I2C bus conditioning, pull-up network, and hardware interrupt handling for the ToF sensor and 6-axis IMU, enabling dynamic matrix row shifting based on real-time pitch/roll cane tilt angles. **Mechanical Integration** - Retrofit onto an existing standard white cane (rigid shaft, hard or lightly cushioned handle) rather than designing a cane from scratch. - Vibration pads are mounted directly against the rigid handle shell (not buried in soft cushioning) to ensure reliable vibration transfer to the user's palm/fingers. - Sensor and electronics enclosure mounted at/near the top of the cane, angled to detect torso/head-height obstacles (complementing, not duplicating, the cane tip's ground-level detection). - Wiring from pads to PCB routed along the shaft with strain relief to survive repeated gripping and flexing. **Functional Logic (Firmware/Software)** - Read pitch and roll orientation data from the IMU to compute real-time cane tilt angle via a complementary filter. - Read multi-zone distance data from the ToF sensor over I2C. - Dynamically adjust active 8x8 ToF row indexing based on IMU tilt angle to keep the detection field focused at an upper-body level that is not traditionally covered by a white cane. - Classify detected obstacles into left/right zones based on sensor sub-regions. - Map distance-to-obstacle into PWM duty cycle per pad (closer = stronger vibration) with a short initial overdrive voltage pulse to minimize motor spin-up latency. - Have a special vibration pattern for overhead/hanging obstacles by utilizing upper rows of the 8x8 ToF sensor - Sample current-sense ADC channels during motor operation to detect open circuits or stalled motor faults. - Drive corresponding pad(s) via the MOSFET switching circuit in real time. - If an obstacle is at the center, then both left and right motors will vibrate. - The vibration strength from the motor will be proportional to the distance to the obstacle. **Criterion for Success** - Obstacle Ranging & Directional Accuracy: Detect obstacles up to 2 meters indoors with at least 90% correct left/right/center zone classification as well as correct overhead/hanging obstacle classification, and hold that classification accuracy across cane tilt angles up to +15 and -15 degrees. - Have accurate motor vibration strength based on the distance to the obstacle. An obstacle that is farther away should not have a vibration strength greater than an object close by. The distance accuracy tolerance should be +-6cm for obstacles at 1m and should be +-11cm for obstacles at 2m. - System Response Latency: Under 250 ms from TOF data-ready interrupt to motor PWM activation, accounting for the sensor's 15Hz refresh limit in 8x8 high-resolution mode, verified on an oscilloscope. At a typical walking speed of 1.4m/s that is under 40 cm of user travel. - Haptic Control & Fault Detection: Scale vibration intensity proportionally with obstacle proximity while utilizing current sensing to detect a stalled motor or a disconnected motor wire. - Battery Life: Provide at least 4 hours of continuous operation on a single battery charge under standard sensing and haptic load. **Existing Solutions** - There are a few commercial products that already put obstacle detection on a cane such as the WeWalk Smart Cane. It uses a single 40kHz ultrasonic sensor with a 55 degree cone. It covers roughly 80 to 170cm and is aimed at obstacles between waist and head height and the whole handle vibrates when it finds something. - We replace the single cone with an 8x8 time of flight array featuring 63 degrees diagonal (around 45 x 45 degrees) field of view. With 64 independently ranged zones we will be able to tell which side an obstacle is on and vibrate that side of the handle. So the cue points rather than just warning. We also will add a 6 axis IMU to track cane pitch and shift which rows of the grid are active, thus keeping the detection band at torso height as grip angle changes. This is something that none of the existing commercial products do. **Extension** - Expanded Directional Granularity: Incorporate additional haptic pads (e.g., a center pad or top/bottom channels) to provide finer multi-angular obstacle directional awareness directly through the grip. - Adjustable sensitivity/mode via a single button: Oftentimes, you don’t require as much of a range detection inside compared to when you are outside. There could be a button that controls “outside mode” (longer detection range) and “inside mode” (shorter detection range). - Intersection & Crossing Awareness Subsystem + GPS: The device connects to a companion phone and uses the phone's GPS as a location reference. When the user approaches a mapped intersection or crossing, the phone signals the cane and the cane outputs a distinct haptic pattern that is clearly different from an obstacle cue. This alerts the user to an upcoming crossing and, combined with the directional pads, helps them orient toward the crosswalk. - Fall Detection & Emergency Alert Subsystem: Uses the existing 6-axis IMU to detect fall-like acceleration and orientation events through a threshold and timing algorithm, with a canceling window to reject false alarms. On a confirmed fall or a manual SOS button press, the ESP32 triggers the paired phone to send an alert and last known location to a preset contact using the phone’s built-in automation so no custom app is required. |
|||||