Sponsors

Aili Wang

Sponsored Projects

  • Touch Controlled Programmable DC Power Supply Circuit (Spring 2024)
  • Mask Detection System (Spring 2021)
  • simplified device for fasteners counter (Spring 2021)

Binbin Li

Sponsored Projects

  • A climbing robot for building 3d printed concrete wall (Spring 2024)
  • MEMS-based Feedback Controller (Spring 2023)

Bruce Xinbo Yu

Sponsored Projects

  • Intelligent Home Security System (Spring 2025)
  • The Smart Fitness Coach (Spring 2025)

Chushan Li

Sponsored Projects

  • Autonomous Transport Car (Spring 2025)
  • High efficient resonance tank design and output voltage control for wireless power transfer system (Spring 2025)
  • Autonomous Transport Car (Spring 2024)
  • Wireless Fast Charging Autonomous Car (Spring 2024)
  • Digital Controlled LED Rotating Display System (Spring 2023)
  • Digital Design Tool for Magnetic and Thermal Components (Spring 2022)
  • Digital design tool for power converters (Spring 2022)
  • A MULTICORE ELECTRONIC CIRCUIT CONTROL SYSTEM (Spring 2021)
  • A novel hanging remote door opener (Spring 2021)

Cristoforo Dimartino

Sponsored Projects

  • ML-based Weather Forecast on Raspberry Pi (Spring 2023)

Fangwei Shao

Sponsored Projects

  • Clickers For ZJUI Undergraduate Mk Ⅱ (Spring 2024)

Gaoang Wang

Sponsored Projects

  • Handwriting Robot With User-Customized Font Style (Spring 2025)
  • Human-Robot Interaction for Object Grasping with Visual Reality and Robotic Arms (Spring 2025)
  • Long-horizon Task Completion with Robotic Arms by Human Instructions (Spring 2025)
  • Tennis Ball Pick-up Machine (Spring 2025)
  • Visual chatting and Real-time acting Robot (Spring 2024)
  • Dancing Scoring Robot (Spring 2023)
  • Raspberry Pet Pal (Spring 2023)
  • Robot for Gym Exercise Guidance (Spring 2023)
  • A Deep Learning Based Paradigm in 3D Human Pose Detection and Estimation in Multi-View Videos (Spring 2021)
  • A Wearable Device Outputting Scene Text For Blind People (Spring 2021)
  • Radar Object Detection Model for Autonomous Driving (Spring 2021)

Hanzhi Ma

Hongwei Wang

Sponsored Projects

  • Project Polaris: An Air and Ground Autonomous Collaboration System (Spring 2022)
  • An EMG-Controlled Robotic Glove for Rehabilitation (Spring 2021)
  • An Indoor Human Activity Detection System and Intelligent Buildings (Spring 2021)
  • Drone-SUGV Cooperative System (Spring 2021)

Howard Yang

Sponsored Projects

  • Sensing your heartbeat (and others) (Spring 2025)
  • Search and Identify (Spring 2024)
  • Recommend songs by your mood (Spring 2022)

Hua Chen

Sponsored Projects

  • DESIGN AND CONTROL OF A FETCHING QUADRUPED (Spring 2025)
  • Design, build and control of a jumping robot (Spring 2025)

Huan Hu

Sponsored Projects

  • A Bio-inspired AI-based Underwater Locating System (Spring 2025)
  • A Compact Material Modulus Measurement Instrument (Spring 2025)
  • Bird-Watching Telescope with Real-Time Bird Identification (Spring 2024)
  • Privacy-protected Elderly monitoring system: Wi-Fi and wearable sensors (Spring 2024)
  • Virtual Band (Spring 2024)
  • Miniaturized Breath Sensors (Spring 2022)
  • Health Monitoring System Based on Radar Fall Motion Detection (Spring 2021)

Jiahuan Cui

Sponsored Projects

  • AIRLOOM TYPE VERTICAL AXIS WIND TURBINE (Spring 2025)
  • Automated guided vehicle for cargo delivery in factories (Spring 2025)
  • Drone Power System Design and Build (Spring 2025)
  • Drone Delivery System for Takeaway Business (Spring 2024)
  • eVTOL Drone (Spring 2024)
  • Movable Robotic Arm Platform (Spring 2024)
  • Electromagnetic Launch System with Switchblade Drone (Spring 2023)
  • Fixed wing drone with auto-navigation (Spring 2023)
  • VTOL Drone with Only Two Propellers (Spring 2023)
  • Fixed-wing rescue UAV (Spring 2022)
  • An Engineering Solution to Auto Chess Set (Spring 2021)
  • Design, manufacture, and control system development for fixed wing drone (Spring 2021)

Kemal Celebi

Sponsored Projects

  • Continuous Roll-To-Roll LB Film Deposition Machine (Spring 2024)
  • Miniterized Langmuir Blodgett Trough (Spring 2023)
  • Interfacial nanocoater (Spring 2022)

Liangjing Yang

Sponsored Projects

  • Automatic Page_Turning Photocopier (Spring 2025)
  • ROBOTIC ARM INTEGRATED INTO WHEELCHAIR WITH MR INTERFACE (Spring 2025)
  • Terrain-adaptive bipedal service robot (Spring 2025)
  • Airport Baggage Robot (Spring 2024)
  • Supernumerary Robotic Limbs (Spring 2024)
  • An immersive human-driven robot detecting foreign matter in tubes (Spring 2023)
  • An Intelligent Assistant Using Sign Language (Spring 2023)
  • MassageMate: Smart Robot Masseur (Spring 2023)
  • Remote Driving System (Spring 2023)
  • Haptic Device (Spring 2022)
  • Intelligent Texas Hold 'Em Robot (Spring 2022)
  • X-reality interface for human-robot interaction (Spring 2022)
  • Camera-based Augmented Reality Endoscope Auxiliary System (Spring 2021)
  • Keebot, a humanoid robot performing 3D pose imitation (Spring 2021)

Lin Qiu

Sponsored Projects

  • 24V Smart Battery Charging System with Health Management (Spring 2025)
  • Cheat for lottery wheel based on servo motor control (Spring 2025)
  • Rudimentary Spherical Motor System for All-Terrain Vehicles (Spring 2025)
  • Smart Power Routing with MPPT-Based Wind Turbine (Spring 2024)
  • High-renewable microgrid for Railway Power Conditioner(RPC) (Spring 2023)
  • Microgrids (Spring 2023)

Mark Butala

Sponsored Projects

  • Seat U: Sensing System for Real-time Library Seat Occupation Detection (Spring 2021)
  • Wireless Charging Table Supporting Multiple Devices with Arbitrary Placement (Spring 2021)

Meng Zhang

Sponsored Projects

  • Automated IC Card Dispenser System for Residential College (Spring 2024)
  • Fingerprint Recognition Door Lock (Spring 2024)
  • VEHICULAR EDGE COMPUTING SYSTEM (Spring 2024)
  • Robot Vacuum (Spring 2023)
  • Semantic Communications for Unmanned Aerial Vehicles (Spring 2023)

Olesksiy Penkov

Sponsored Projects

  • A device for evaluation of frictional properties of surfaces (Spring 2025)
  • Four-axis vacuum stage for advanced nano-manufacturing (Spring 2025)
  • A Micro-Tribotester to Characterize the Wear Phenomenon (Spring 2021)

Pavel Loskot

Sponsored Projects

  • A 2D Model of Optical Satellite Communication System (Spring 2025)
  • A smart glove for HCI (Spring 2025)
  • 3D model of a satellite footprint over the Earth (Spring 2024)
  • Arduino-Powered Network Flow Visualization Toolbox (Spring 2024)
  • Display for ODEs (Spring 2024)
  • Submarine Model (Spring 2024)
  • 3D Scanner (Spring 2023)
  • Remote Robot Car Control System with RGBD Camera for 3D Reconstruction (Spring 2023)
  • Thermo-Camera Based Energy Consumption Monitoring System (Spring 2023)
  • Authentication System for SARS-CoV-2 Management (Spring 2022)
  • Identification and tracking of UAVs (Spring 2022)
  • A Multiple Sensors Human Movement Detection Framework for Interactive Games and Applications (Spring 2021)

Piao Chen

Sponsored Projects

  • Intelligent shared item cabinet (Spring 2025)

Rakesh Kumar

Sponsored Projects

  • A Transformer (Spring 2023)
  • An Autonomous Pool Cleaner (Spring 2023)
  • Augmenting AR/VR with Smell (Spring 2023)

Ruisheng Diao

Sponsored Projects

  • Carbon Emission Tracking System (Spring 2025)
  • Advanced Modeling and Display of ZJU International Campus Power System (Spring 2024)
  • Electric Load Forecasting (ELF) System (Spring 2023)

Shi Ye

Sponsored Projects

  • SoftBot: Jellyfish-Inspired Bionic Soft Robot with Visual Perception (Spring 2025)

Shurun Tan

Sponsored Projects

  • A remote environment recording system with online access portals (Spring 2025)
  • A Remote Microwave Environmental Monitoring System: Automation and Power Management (Spring 2025)
  • A micro-penetrometer for snow and soil structural analysis (Spring 2024)
  • Image acquisition, 3D reconstruction and a visual interactive digital heritage system (Spring 2024)
  • A Direct Digitally Modulated Wireless Communication System (Spring 2023)
  • A mm-Wave Breath Monitoring System for Smart Vehicle Applications (Spring 2023)
  • UAV-borne SAR, InSAR, and TomoSAR System (Spring 2022)
  • Augmented Reality and Virtual Reality for Electromagnetics Education (Spring 2021)
  • Auto Model Navigation with SNN Prototype on FPGA (Spring 2021)
  • Integrated Synthetic Aperture Radar Remote Sensing System on UAV (Spring 2021)

Simon Hu

Sponsored Projects

  • Campus Tour Guide by AI-Powered Autonomous System (Spring 2024)
  • Digital Twin Bridge Monitoring System (Spring 2024)
  • V2V Based Network Cooperative Control System (Spring 2023)
  • Low-cost sensor for seamless road quality monitoring (Spring 2022)
  • Smart and Connected Traffic Light With Autonomous Vehicle (Spring 2022)
  • A crowd-sourcing urban air quality monitoring system with bikes (Spring 2021)

Timothy Lee

Sponsored Projects

  • A DodgeBot System (Spring 2025)
  • Dodgeball bot (Spring 2025)
  • Dodgeball Bots (Spring 2025)
  • AR Sandbox (Spring 2024)
  • Particle Image Velocimetry (Spring 2024)
  • Robotic T-shirt Launcher Mark III (Spring 2024)
  • Smart Power Routing (Spring 2024)
  • Laser System to Shoot Down Mosquitos (Spring 2023)
  • Clickers for ZJUI Undergraduate (Spring 2023)
  • High Noon Sheriff Robot (Spring 2023)
  • Observation Balloon For Testing Centers (Spring 2023)
  • Robotic T-Shirt Launcher Mark II (Spring 2023)
  • Tea Blend Distributor (Spring 2023)
  • Autonomous Food Delivery Robot (Spring 2022)
  • Camera Shy (Spring 2022)
  • Directed fan (Spring 2022)
  • Fountain show (Spring 2022)
  • High-precision handwriting reader (Spring 2022)
  • Mimicry Stage Lighting Control System (Spring 2022)
  • Movable Pizza Oven (Spring 2022)
  • Personalized information display mirror (Spring 2022)
  • Robotic T-shirt launcher (Spring 2022)
  • Unified model for flash boiling of binary mixtures (Spring 2022)

Wee-Liat Ong

Sponsored Projects

  • Customizable Automatic Pottery Wheel Throwing Machine (Spring 2025)
  • Smart medicine box (Spring 2025)
  • intelligent robot arm (Spring 2024)
  • Teaching Heat to Student (Spring 2024)
  • A Desktop-Size Environment-Controlled Greenhouse for Multi-Variable Optimization of Crop Growth (Spring 2023)
  • Smartphone-based Fluorescence Microscope (Spring 2023)
  • Desktop-size Environment-controlled Greenhouse for Soilless Cultivation of Lettuce (Spring 2022)

Wenchao Chen

Xiao Yan

Xuguang Wang

Sponsored Projects

  • Smart Refrigerator Inventory Add-on Unit (Spring 2022)

Yang Xu

Sponsored Projects

  • Sensor-based Automatic Delivery Vehicle (Spring 2021)

Yu Lin

Sponsored Projects

  • Automated Chinese Traditional Chimes with Song (Spring 2024)
  • Intelligent fire protection ecosystem (Spring 2024)
  • Smart Laundry FoldBot (Spring 2024)

Yushi Cheng

Sponsored Projects

  • Smart Assistive Walking Stick for the Visually Impaired (Spring 2025)
  • Epicast: Augmented Board Game (Spring 2024)

Zuozhu Liu

Sponsored Projects

  • Dental Health Monitoring System (Spring 2023)
  • 3D Oral Scan Teeth Model Segmentation Using Deep Learning (Spring 2021)
  • Federated Learning with Medical Data (Spring 2021)
  • Unsupervised Representation Learning in 3D Medical Image (Spring 2021)

Said Mikki

Sponsor

Sponsored Projects

  • Analog Computer ODE Solver (Spring 2025)
  • Automated Pour-over Coffee Machine with Imitation Learning (Spring 2024)
  • A Wearable Device That Can Detect Mood (Spring 2024)
  • Actions to Mosquitoes (Spring 2024)
  • Automatic Intelligent Fishing Pod (Spring 2024)
  • FPGA-based object tracking, obstacle avoidance, and voice-activated trolley (Spring 2023)
  • Autonomous Behavior Supervisor (Spring 2022)
  • InWhat: An Intelligent System for Wardrobe (Spring 2022)

A Wearable Device Outputting Scene Text For Blind People

Hangtao Jin, Youchuan Liu, Xiaomeng Yang, Changyu Zhu

A Wearable Device Outputting Scene Text For Blind People

Featured Project

# Revised

We discussed it with our mentor Prof. Gaoang Wang, and got a solution to solve the problem

## TEAM MEMBERS (NETID)

Xiaomeng Yang (xy20), Youchuan Liu (yl38), Changyu Zhu (changyu4), Hangtao Jin (hangtao2)

## INSTRUCTOR

Prof. Gaoang Wang

## LINK

This idea was pitched on Web Board by Xiaomeng Yang.

https://courses.grainger.illinois.edu/ece445zjui/pace/view-topic.asp?id=64684

## PROBLEM DESCRIPTION

Nowadays, there are about 12 million visually disabled people in China. However, it is hard for us to see blind people in the street. One reason is that when the blind people are going to the location they are not familiar with, it is difficult for blind people to figure out where they are. When blind people travel, they are usually equipped with navigation equipment, but the accuracy of navigation equipment is not enough, and it is difficult for blind people to find the accurate position of the destination when they arrive near the destination. Therefore, we'd like to make a device that can figure out the scene text information around the destination for blind people to reach the direct place.

## SOLUTION OVERVIEW

We'd like to make a device with a micro camera and an earphone. By clicking a button, the camera will take a picture and send it to a remote server to process through a communication subsystem. After that, text messages will be extracted and recognized from the pictures using neural network, and be transferred to voice messages by Google text-to-speech API. The speech messages will then be sent back through the earphones to the users. The device can be attached to glasses that blind people wear.

The blind use the navigation equipment, which can tell them the location and direction of their destination, but the blind still need the detail direction of the destination. And our wearable device can help solve this problem. The camera is fixed to the head, just like our eyes. So when the blind person turns his head, the camera can capture the text of the scene in different directions. Our scenario is to identify the name of the store on the side of the street. These store signs are generally not tall, about two stories high. Blind people can look up and down to let the camera capture the whole store. Therefore, no matter where the store name is, it can be recognized.

For example, if a blind person aims to go to a book store, the navigation app will tell him that he arrives the store and it is on his right when he are near the destination. However, there are several stores on his right. Then the blind person can face to the right and take a photo of that direction, and figure out whether the store is there. If not, he can turn his head a little bit and take another photo of the new direction.

![figure1](https://courses.grainger.illinois.edu/ece445zjui/pace/getfile/18612)

![figure2](https://courses.grainger.illinois.edu/ece445zjui/pace/getfile/18614)

## SOLUTION COMPONENTS

### Interactive Subsystem

The interactive subsystem interacts with the blind and the environment.

- 3-D printed frame that can be attached to the glasses through a snap-fit structure, which could holds all the accessories in place

- Micro camera that can take pictures

- Earphone that can output the speech

### Communication Subsystem

The communication subsystem is used to connect the interactive subsystem with the software processing subsystem.

- Raspberry Pi(RPI) can get the images taken by the camera and send them to the remote server through WiFi module. After processing in the remote server, RPI can receive the speech information(.mp3 file).

### Software Processing Subsystem

The software processing subsystem processes the images and output speech, which including two subparts, text recognition part and text-to-speech part.

- A OCR recognition neural network which is able to extract and recognize the Chinese text from the environmental images transported by the communication system.

- Google text-to-speech API is used to transfer the text we get to speech.

## CRITERION FOR SUCCESS

- Use neural network to recognize the Chinese scene text successfully.

- Use Google text-to-speech API to transfer the recognized text to speech.

- The device can transport the environment pictures or video to server and receive the speech information correctly.

- Blind people could use the speech information locate their position.