Project

# Title Team Members TA Documents Sponsor
34 A smart glove for HCI
 Hongwei Dong
Jinhao Zhang
Shanbin Sun
Zhan Shi
 design_document1.pdf
final_paper1.pdf
final_paper2.pdf
proposal2.pdf
 Pavel Loskot
# TEAM MEMBERS
Hongwei Dong (hd2), Shanbin Sun (shanbin3), Jinhao Zhang (jinhaoz2), Zhan Shi (zhans6)

# PROBLEM & SOLUTION OVERVIEW
In today's society, people are increasingly interacting with smart devices such as laptops and smartphones. This trend underscores the need for innovative methods to improve the efficiency of interaction with these devices. Among the emerging solutions, smart gloves hold great promise as a means to address this need. The smart glove is able to collect the positional information of the user's fingers. It then processes the information to recognize the user's gestures and maps the recognized gestures to predefined shortcuts, thereby facilitating efficient interaction between the user and the computer.

# PROJECT TITLE
A smart glove for HCI

# SOLUTION COMPONENTS
## Subsystem1: IMU based gesture sensing system
- MPU6050, a six DOF IMU, is placed on each fingertip to collect the position and angle information.
- I2C bus to communicate with the ESP32.

## Subsystem2: gesture recognition system
- Raw data pre-processing to obtain high accuracy gesture
- Pre-trained gesture recognition model using ESP-DL inference library
- User-defined gesture shortcut map to support any type of input

## Subsystem 3: Communication System
- Serialization and deserialization on both the device and host side to package the information to be transmitted in binary/JSON format.
- CP2102/CH340 USB module to support USB serial communication such as UART when the glove is charging or high-bandwidth transmission is required
- Bluetooth module to support Bluetooth (LE optional) communication for gesture and command transmission

## Subsystem 4: Power Management System
- High energy density lithium polymer battery (e.g. 1000-3000mAh) to power the IMUs, esp32, and peripheral components, ensuring a long wireless user experience.
- Step-up or step-down voltage regulator circuitry to meet the stable power requirements of each subsystem.

# Criterion For Success
- The MPU6050 IMU system should reliably collect raw gesture data, with stable I2C data transfer between the MPU6050 and the ESP32 without significant latency or data loss.
- The deep learning gesture recognition model used on the ESP32 should be able to map gestures to appropriate keystrokes, enabling mouse operations and various custom functions.
- The communication system should ensure seamless data transfer between the computer and mobile devices without significant latency.
- The battery management system should ensure that all components receive the correct voltage, with the charge management module and power monitoring functions operating correctly.

# DISTRIBUTION OF WORK
- Jinhao Zhang [EE]: Responsible for the design and implementation of the power subsystem and other circuit systems, including the design, test, and optimization of the battery charging module and power monitoring module.
- Hongwei Dong [ECE]: Responsible for raw gesture data pre-processing program. Training and deployment of the gesture recognition model. Serialization/deserialization of data structure for device and host.
- Shanbin Sun [ECE]: Responsible for collect the gesture data used for training the model. and I2C bus development. Develop the user interface to define the gesture shortcut map. Develop the IMU and device driver.
- Zhan Shi [EE]: Responsible for USB and Bluetooth communication between host and device. Develop the I2C bus for IMU to ESP32 communication. PCB design and verification. Unit testing of each subsystems.

Master Bus Processor

Featured Project

General Description

We will design a Master Bus Processor (MBP) for music production in home studios. The MBP will use a hybrid analog/digital approach to provide both the desirable non-linearities of analog processing and the flexibility of digital control. Our design will be less costly than other audio bus processors so that it is more accessible to our target market of home studio owners. The MBP will be unique in its low cost as well as in its incorporation of a digital hardware control system. This allows for more flexibility and more intuitive controls when compared to other products on the market.

Design Proposal

Our design would contain a core functionality with scalability in added functionality. It would be designed to fit in a 2U rack mount enclosure with distinct boards for digital and analog circuits to allow for easier unit testings and account for digital/analog interference.

The audio processing signal chain would be composed of analog processing 'blocks’--like steps in the signal chain.

The basic analog blocks we would integrate are:

Compressor/limiter modes

EQ with shelf/bell modes

Saturation with symmetrical/asymmetrical modes

Each block’s multiple modes would be controlled by a digital circuit to allow for intuitive mode selection.

The digital circuit will be responsible for:

Mode selection

Analog block sequence

DSP feedback and monitoring of each analog block (REACH GOAL)

The digital circuit will entail a series of buttons to allow the user to easily select which analog block to control and another button to allow the user to scroll between different modes and presets. Another button will allow the user to control sequence of the analog blocks. An LCD display will be used to give the user feedback of the current state of the system when scrolling and selecting particular modes.

Reach Goals

added DSP functionality such as monitoring of the analog functions

Replace Arduino boards for DSP with custom digital control boards using ATmega328 microcontrollers (same as arduino board)

Rack mounted enclosure/marketable design

System Verification

We will qualify the success of the project by how closely its processing performance matches the design intent. Since audio 'quality’ can be highly subjective, we will rely on objective metrics such as Gain Reduction (GR [dB]), Total Harmonic Distortion (THD [%]), and Noise [V] to qualify the analog processing blocks. The digital controls will be qualified by their ability to actuate the correct analog blocks consistently without causing disruptions to the signal chain or interference. Additionally, the hardware user interface will be qualified by ease of use and intuitiveness.