Project
| # | Title | Team Members | TA | Documents | Sponsor |
|---|---|---|---|---|---|
| 49 | Tactile Screen For Visual Aid |
Joshua Sluis Tyler Wille Walter Chang |
Jianchong Chen | ||
| Team Members: Walter Chang (walterc2) Tyler Wille (tmwille2) Joshua Sluis (jsluis2) Problem People with visual disabilities have very limited options for aids. Braille and walking sticks are the most popular options, however, they are both extremely limited by proximity. Additionally, they often require built in infrastructure that may not exist, or may not be well maintained. With the onset of AI, there are many new tools that leverage this new technology in order to assist in these tasks. However, we find that these technologies are severely lacking and extremely limiting. They put an overreliance on AI detection and prevent people from making their own judgements. Our solution gives this power back to the people who are using it. Solution Our system will aim to create a tactile physical screen connected to a camera that will allow users to "feel" images. The camera will be attached to a wearable device, ideally glasses, which allow for head movements to control it. Our solution will use a camera through greyscale processing, and automatically detect edges in real time images. It will then draw these lines on this tactile screen, and the resulting image will be able to be felt through raised and lowered pins. Our screen will feature 32x32 pixels, although this may increase depending on time constraints. Our pins will be around 3mm apart, as this is as close as we can get. There will be three main components to the project, the camera, the video processing, and the screen itself. Solution Components Subsystem 1: Camera The camera will be the first step in our system. The camera we will be using will be a lightweight model that will have autofocus. We would like to be able to switch between two modes, navigation and reading. This will allow for more versatility during operation. Navigation will be focused at around 1-5 m while reading will be focused at around 30-40 cm. We will be using the OV5640 AF module to allow for this. It has autofocus, which allows for more ease of use, as well as being able to toggle this feature on and off, for switching between the two modes. During reading mode, we would switch between processing the entire image to processing a smaller part of the whole, 64x32 pixels in the center of the screen. This would allow us to actually show much smaller letters in finer detail. The camera’s autofocus already has this feature. Additionally, the reading system would have a higher refresh rate as compared to navigation, with much less processing needed. It would also include a reading light, to be able to allow the camera to capture letters more precisely. Subsystem 2: Image Processing Our system will feature lightweight image processing on ESP32-S3. This already has a camera interface, which will simplify programming. We will run two different algorithms, depending on the read vs nav modes. Nav: In Nav mode, we will process our image, and use box binning to smooth it out to a granularity of 160x160 to reduce random noise. Then, we will run a Sobel edge detection algorithm to find edges. Our tactile screen will use edges rather than areas, because this will be easier for fingers to find. Then, we will further reduce the granularity to be able to fit on our screen, down to 32x32 Read: In Read mode, we will not need to use box binning. We will also use Otsu thresholding instead of edge detection, as text should be easier to distinguish. We will also set a baseline depending on the light level of the LED. Then, we will reduce the granularity further to 32x16, to be able to display on the tactile screen. This reduction of screen utilization will also further reduce power consumption The camera and image processing subsystem will communicate with the screen subsystem via UART Subsystem 3: Screen The screen is split into two main parts, a moving carriage, and the screen itself. The carriage wil update 32 bits at once, rather than updating the entire screen. This is a much more feasible design, due to wiring and physical constraints. Screen: The screen will have 32x32 pixels. It will consist of 5 layers, one plastic layer, a metal layer, the magnet layer, another metal layer, and final plastic layer. The first plastic layer will be where the user touches the screen. It will be relatively thick, to make the pins stable. The metal layers will hold the magnet layer together. The metal allows the magnets to stick to either the top or the bottom, allowing for bistability, reducing the amount of power needed. They will need to be fairly thin as well in order to achieve a balance between being strong enough to hold the magnet, and weak enough that we can move it magnetically. Both metal layers will have holes, one side will allow for the pins to pass through, while the other side will allow the magnetic field through. Additionally, we will need to add relief cuts to reduce eddy currents.We will use permanent neodymium magnets that are around 1.5mm in diameter and 1mm in height. We will additionally attach pins to the magnets that will protrude from the plastic to interface with the user. The final plastic layer is on the bottom to prevent the the magnets from falling out. Carriage: Our carriage will consist of 32 electromagnets in two rows. Odd and even pixels will be staggered, which both reduces magnetic cross talk, as well as allowing more commercially available electromagnets, and reducing the peak current needed. We will use a NEMA 14 Bipolar Stepper combined with a GT2 Rubber Belt and a 20-Tooth GT2 Pulley to be able to move the carriage along the screen. On the carriage itself, the 32 electromagnets will be Bourns RLB0608-222KL. They will be powered by H bridges, TI DRV8912-Q1, which will allow us to change the polarity of the magnetization. We will communicate to the H-bridges using the SPI protocol with the main controller. Control The carriage will be controlled by an ESP32, which will store the incoming frame as well as the previous. This will allow us to make changes only when there are updates, and leave the screen as is, reducing power consumption. This will also control the stepper motor. Power We will utilize commercially available battery packs, and use USB C power delivery. We will have 3 distinct power rails, on at 9 V, 5V and 3.3V. The 9 V will be used for the motor, the 5V will be used for the H bridge and the 3.3 V will be used for the camera and image processing, as well as the control of the motor. We will also require bulk capacitors on the carriage PCB in order to supply enough current. Criteria For Success - Navigation Mode can display large objects - doorways, cars. - Read Mode can display letters - Read Mode letters can be felt and read with eyes closed. - Refresh rate is greater than 1Hz - Relatively handheld device < 1 kg |
|||||