Course Overview :: ECE 445 - Senior Design Laboratory

Course Overview

Welcome to ECE 445 / ME 470 Senior Design ZJUI Spring 2024!

Welcome to the class! If you've looked at the course Calendar, you've probably already noticed that this class is quite different from most other classes in the department. The class only meets as a whole for the first four weeks of the semester. During these lectures, you will meet the Course Staff, learn about specific requirements, resources, and project choices for the course, and have a chance to meet other students. These are some of the most important weeks for the class since the decisions you make during this time will determine what you'll get out of this class and, in many ways, how much you'll enjoy it.

In this course, you will form teams and propose projects that solve an engineering problem in a unique way. The projects generally involve a device that you will design, build, and demonstrate. We are excited to see what projects you create this semester! In the midst of an ever-changing learning environment, we want to encourage you to think, create, design, and build exemplary projects. We want to ensure that your experience in 445 demonstrates your potential as an engineer graduating from the University of Illinois.

This course is taught hybrid for ME and ECE students, and some projects are mentored by ZJUI faculty. Here are a few items that you will need to consider as we enter this semester.

Expectations and Requirements

We have high expectations for students participating in ECE/ME Senior Design. You are soon to be an alumnus of one of the top engineering schools in the world. Our alumni hold themselves to high technical and professional standards of conduct. In general, projects are expected to be safe, ethical, and have a level of design complexity commensurate with the rigour of the ECE/ME Illinois curriculum. Requirements for specific assignments due throughout the semester can be found by looking through the Grading Scheme for the course. Please read through this documentation well before each assignment is due. Specific due dates can be found on the course Calendar.

Below are a few words of wisdom to keep in mind throughout the semester to increase your enjoyment and success in the course:

 

An Intelligent Assistant Using Sign Language

Qianzhong Chen, Howie Liu, Haina Lou, Yike Zhou

Featured Project

# TEAM MEMBERS

Qianzhong Chen (qc19)

Hanwen Liu (hanwenl4)

Haina Lou (hainal2)

Yike Zhou (yikez3)

# TITLE OF THE PROJECT

An Intelligent Assistant Using Sign Language

# PROBLEM & SOLUTION OVERVIEW

Recently, smart home accessories are more and more common in people's home. A center, which is usually a speaker with voice user interface, is needed to control private smart home accessories. But a interactive speaker may not be the most ideal for people who are hard to speak or hear. Therefore, we aim to develop a intelligent assistant using sign language, which can understand sign languages, interact with people, and act as a real assistant.

# SOLUTION COMPONENTS

## Subsystem1: 12-Degree-of-Freedom Bionic Hand System

- Two moveable joints every finger driven by 5-V servo motors

- The main parts of the hand manufactured with 3D printing

- The bionic hand is fixed on a 2-DOF electrical platform

- All of the servo motors controlled by PWM signals transmitted by STM32 micro controller

## Subsystem2: The Control System

- The controlling system consists of embedded system modules including the microcontroller, high performance edge computing platform which will be used to run dynamic gesture recognition model and more than 20 motors which can control the delicate movement of our bionic hand. It also requires a high-precision camera to capture the hand gesture of users.

## Subsystem3: Dynamic Gesture Recognition System

- A external camera capturing the shape, appearance, and motion of objective hands

- A pre-trained model to help other subsystems to figure out the meaning behind the sign language. To be more specific, at the step of objects detection, we intended to adopt YOLO algorithm as well as Mediapipe, a machine learning framework developed by Google to recognize different sign language efficiently. Considering the characteristic of dynamic gesture, we also hope to adopt 3D-CNN and RNN to build our models to better fit in the spatio-temporal features.

# CRITERION OF SUCCESS

- The bionic hand can move free and fluently as designed, all of the 12 DOFs fulfilled. The movement of single joint of the finger does not interrupt or be interrupted by other movements. The durability and reliability of the bionic hand is achieved.

- The controlling system needs to be reliable and outputs stable PWM signals to motors. The edge computing platform we choose should have high performance when running the dynamic gesture recognition model.

- Our machine could recognize different sign language immediately and react with corresponding gestures without obvious delay.

# DISTRIBUTION OF WORK

- Qianzhong Chen(ME): Mechanical design and manufacture the bionic hand; tune the linking between motors and mechanical parts; work with Haina to program on STM32 to generate PWM signals and drive motors.

- Hanwen Liu(CompE): Record gesture clips to collect enough data; test camera modules; draft reports; make schedules.

- Haina Lou(EE): Implement the embedded controlling System; program the microcontroller, AI embedded edge computing module and implement serial communication.

- Yike Zhou(EE): Accomplish object detection subsystem; Build and train the machine learning models.