Project

# Title Team Members TA Documents Sponsor
49 Tactile Screen For Visual Aid
Joshua Sluis
Tyler Wille
Walter Chang
Jianchong Chen design_document1.pdf
proposal1.pdf
Team Members:

Walter Chang (walterc2)
Tyler Wille (tmwille2)
Joshua Sluis (jsluis2)

Problem
People with visual disabilities have very limited options for aids. Braille and walking sticks are the most popular options, however, they are both extremely limited by proximity. Additionally, they often require built in infrastructure that may not exist, or may not be well maintained.

With the onset of AI, there are many new tools that leverage this new technology in order to assist in these tasks. However, we find that these technologies are severely lacking and extremely limiting. They put an overreliance on AI detection and prevent people from making their own judgements. Our solution gives this power back to the people who are using it.

Solution

Our system will aim to create a tactile physical screen connected to a camera that will allow users to "feel" images. The camera will be attached to a wearable device, ideally glasses, which allow for head movements to control it.

Our solution will use a camera through greyscale processing, and automatically detect edges in real time images. It will then draw these lines on this tactile screen, and the resulting image will be able to be felt through raised and lowered pins. Our screen will feature 32x32 pixels, although this may increase depending on time constraints. Our pins will be around 3mm apart, as this is as close as we can get.

There will be three main components to the project, the camera, the video processing, and the screen itself.

Solution Components
Subsystem 1: Camera

The camera will be the first step in our system. The camera we will be using will be a lightweight model that will have autofocus. We would like to be able to switch between two modes, navigation and reading. This will allow for more versatility during operation. Navigation will be focused at around 1-5 m while reading will be focused at around 30-40 cm. We will be using the OV5640 AF module to allow for this. It has autofocus, which allows for more ease of use, as well as being able to toggle this feature on and off, for switching between the two modes.
During reading mode, we would switch between processing the entire image to processing a smaller part of the whole, 64x32 pixels in the center of the screen. This would allow us to actually show much smaller letters in finer detail. The camera’s autofocus already has this feature.

Additionally, the reading system would have a higher refresh rate as compared to navigation, with much less processing needed. It would also include a reading light, to be able to allow the camera to capture letters more precisely.

Subsystem 2: Image Processing
Our system will feature lightweight image processing on ESP32-S3. This already has a camera interface, which will simplify programming. We will run two different algorithms, depending on the read vs nav modes.

Nav:
In Nav mode, we will process our image, and use box binning to smooth it out to a granularity of 160x160 to reduce random noise. Then, we will run a Sobel edge detection algorithm to find edges. Our tactile screen will use edges rather than areas, because this will be easier for fingers to find. Then, we will further reduce the granularity to be able to fit on our screen, down to 32x32

Read:
In Read mode, we will not need to use box binning. We will also use Otsu thresholding instead of edge detection, as text should be easier to distinguish. We will also set a baseline depending on the light level of the LED. Then, we will reduce the granularity further to 32x16, to be able to display on the tactile screen. This reduction of screen utilization will also further reduce power consumption

The camera and image processing subsystem will communicate with the screen subsystem via UART

Subsystem 3: Screen

The screen is split into two main parts, a moving carriage, and the screen itself. The carriage wil update 32 bits at once, rather than updating the entire screen. This is a much more feasible design, due to wiring and physical constraints.

Screen:

The screen will have 32x32 pixels. It will consist of 5 layers, one plastic layer, a metal layer, the magnet layer, another metal layer, and final plastic layer. The first plastic layer will be where the user touches the screen. It will be relatively thick, to make the pins stable. The metal layers will hold the magnet layer together. The metal allows the magnets to stick to either the top or the bottom, allowing for bistability, reducing the amount of power needed. They will need to be fairly thin as well in order to achieve a balance between being strong enough to hold the magnet, and weak enough that we can move it magnetically. Both metal layers will have holes, one side will allow for the pins to pass through, while the other side will allow the magnetic field through. Additionally, we will need to add relief cuts to reduce eddy currents.We will use permanent neodymium magnets that are around 1.5mm in diameter and 1mm in height. We will additionally attach pins to the magnets that will protrude from the plastic to interface with the user. The final plastic layer is on the bottom to prevent the the magnets from falling out.

Carriage:

Our carriage will consist of 32 electromagnets in two rows. Odd and even pixels will be staggered, which both reduces magnetic cross talk, as well as allowing more commercially available electromagnets, and reducing the peak current needed. We will use a NEMA 14 Bipolar Stepper combined with a GT2 Rubber Belt and a 20-Tooth GT2 Pulley to be able to move the carriage along the screen.

On the carriage itself, the 32 electromagnets will be Bourns RLB0608-222KL. They will be powered by H bridges, TI DRV8912-Q1, which will allow us to change the polarity of the magnetization. We will communicate to the H-bridges using the SPI protocol with the main controller.

Control

The carriage will be controlled by an ESP32, which will store the incoming frame as well as the previous. This will allow us to make changes only when there are updates, and leave the screen as is, reducing power consumption. This will also control the stepper motor.

Power

We will utilize commercially available battery packs, and use USB C power delivery. We will have 3 distinct power rails, on at 9 V, 5V and 3.3V. The 9 V will be used for the motor, the 5V will be used for the H bridge and the 3.3 V will be used for the camera and image processing, as well as the control of the motor. We will also require bulk capacitors on the carriage PCB in order to supply enough current.


Criteria For Success
- Navigation Mode can display large objects - doorways, cars.
- Read Mode can display letters
- Read Mode letters can be felt and read with eyes closed.
- Refresh rate is greater than 1Hz
- Relatively handheld device < 1 kg


HARMONEX

Rubin Du, Alan Lu, David Song

HARMONEX

Featured Project

**Team**

Alan Lu -- jialin8

Rubin Du -- rd25

**Problem**

Lifting and carrying heavy objects is a common but physically demanding task faced in both personal and industrial environments. Whether it is a person at home carrying groceries or a logistics worker handling cargo, repetitive lifting puts stress on the musculoskeletal system and can result in fatigue, reduced productivity, and even long-term injuries. Existing exoskeleton solutions often focus on industrial use, but they suffer from limited backdrivability, high weight, or overly complex designs that prevent practical everyday use. A lightweight, safe, and efficient solution is needed to reduce the physical burden of lifting while maintaining user freedom of movement.

**Solution**

Our team proposes the development of a wearable exoskeleton system designed to assist users in lifting objects of up to 10 kilograms with minimal effort. The system employs a hybrid actuation strategy that combines the strengths of both a BLDC motor and a servo motor: the BLDC provides the torque required for large-angle lifting motions, while the servo supplies stable holding torque to maintain the lifted position without excess energy drain. The BLDC goes through a 64:1 planetary gear set to amplify torque, and the servo motor goes through a moveable linkage system to create sufficient mechanical advantage to further reduce the load on the motor. A detachable drivetrain allows the user to disengage the actuation system, enabling free arm movement when lifting support is not required. The skeleton itself is lightweight, manufactured using carbon-fiber-reinforced nylon (PA-CF), ensuring durability and comfort. This modular design starts with elbow actuation and can be scaled to include shoulder actuation, broadening its application.

**Solution Components**

**Subsystem 1: Mechanical Skeleton and Drivetrain**

- Lightweight PA-CF composite structure, under 3 kg excluding the battery.

- Hybrid drivetrain using BLDC with planetary gear for motion and servo motor for holding.

- Drivetrain disengagement mechanism for free arm movement.

- Moveable armor integrated with a linkage system on the drivetrain that elaborately moves upper limb armor to avoid structural interference.

**Subsystem 2: Actuation and Power System**

- Actuated by BLDC + servo combination for efficiency and safety.

- Powered by a 6S LiPo battery (~200 Wh), providing several hours of continuous assistance.

- Custom PCB with DC-DC buck converters for peripheral loads and power distribution.

- Thermal management through ventilation and optional forced convection.

**Subsystem 3: Control and Signal Processing**

- Joint actuation regulated through PID controllers.

- User intent detected via EMG sensors integrated into the arm.

- Signal conditioning pipeline: Kalman filter → Chebyshev low-pass filter → controller input.

- Optional manual override via a simple forearm-mounted control panel.

- Microcontroller and peripheral signals integrated on a customized PCB/FPGA.

**Subsystem 4: Peripherals**

- Armor ambient light will be integrated into the shell of the skeleton for aesthetics.

- Ventilation port openings will be controlled by microservos to ensure good heat dissipation.

- A manual control panel will be placed on the lower limb skeleton to include manual operations and emergency switches.

- TPU-based soft pads inside the skeleton to provide a comfort experience for the user.

**Scalability and Modularity**

- The initial prototype targets elbow actuation.

- Design is scalable to include shoulder actuation grounded to chest armor.

- The modular approach ensures meaningful demonstration even if full-body integration is not achieved.

**Criterion for Success**

The final solution will be a wearable exoskeleton capable of assisting the user in lifting and holding objects up to 10 kg through a dual-actuation BLDC–servo system with a detachable drivetrain for free arm movement, powered by an onboard 6S battery, lightweight (under 3 kg excluding the battery), and controlled via EMG signals or a manual override panel to ensure safe, efficient, and natural operation.