Project

# Title Team Members TA Documents Sponsor
27 Smart Foot-Controlled Mouse with Sensor Fusion and UI-Aware Assistance
 Chaoxiang Yang
Hao Liu
Jiongye Liu
Zhihao Cheng
 Wee-Liat Ong
Team Members
Zhihao Cheng(zhihao10)
Hao Liu(hao25)
Chaoxiang Yang(cy60)
Jiongye Liu(jl244)

#Problem

In the present and in the foreseeable future, the interaction of electronic devices still heavily relies on devices such as keyboards and mice. However, these devices have overlooked certain groups of people who are unable to use them, such as users with upper limb disabilities. For instance, it is often difficult for them to use ordinary mice that require precise hand control. Due to the widespread use of computers in our daily lives, we hope that people with disabilities can break free from the shackles of certain device limitations. To address the issue of inconvenient interaction for disabled individuals, there are already some assistive devices available on the market. Eye-tracking systems work well, but they are often expensive. Simple foot switches are easy to operate, but they can only handle simple commands. Therefore, we propose manufacturing wearable foot-controlled mice, which will provide users with a more complete and practical way of using computers. This approach may also provide an alternative interaction method for human-computer interfaces.

#Solution Overview

Our project is to design a smart wearable device similar to a pair of slippers, namely a smart foot-controlled mouse. Firstly, multiple pressure sensors are used to detect foot gestures. These data will be converted into mouse movements and mouse operations (such as clicking, double-clicking, and dragging) through algorithms. If the operation is allowed, it can also support custom key selection. The system also includes auxiliary functions that are compatible with the user interface to assist in achieving precise movement operations. For example, the system can detect small icons or interactive links, and intelligently reduce the mouse movement speed to achieve precise clicks, making it easier to perform delicate tasks.

#Solution Components

##Sensing Hardware
- FSR pressure sensors for detecting foot pressure distribution
- The sensor signals are used to distinguish different foot gestures

##Embedded Control Hardware
-Sensor data collection and signal processing
-The controller generates signals into mouse commands

##Communication and Power Hardware
- The communication hardware sends control signals to the computer.
- The system supports either wired or wireless communication with the computer.
- A rechargeable battery or wired power supply can be used for power.

##Device Structure
- An ergonomic structure designed for comfortable and stable foot interaction.
- A lightweight structural frame used to hold sensors and electronic components.
- Durable materials are used to improve stability and long-term reliability.

##Software
-Foot gesture recognition and signal processing
-Cursor movement control and mouse command generation
-Customization for sensitivity adjustment
-Provide UI-aware assistance for precise interaction

#Criterion for Success

-The device should move the cursor smoothly and perform basic mouse commands like click, double-click and drag.
-The mis-trigger rate should be relatively low in standard testing conditions.
-The system should reliably distinguish different foot gestures after a short calibration process.
-The wearable structure should stay stable and reasonably comfortable during extended use.
-The device should work with common computers using either wired or wireless communications.
-The UI-aware assistance should help users select small on-screen targets more easily than basic foot-only control.

#Distribution of Work

Jiongye Liu is responsible for the Power Hardware and Device Structure, including power support, ergonomic design, and hardware integration. Zhihao Cheng, Hao Liu, and Chaoxiang Yang are responsible for the Sensing Hardware, Communication Hardware, Embedded System, and Software, including signal collection and processing, gesture recognition, cursor control, mouse command generation, sensitivity adjustment, and UI-aware assistance.

Augmented Reality and Virtual Reality for Electromagnetics Education

Zhanyu Feng, Zhewen Fu, Han Hua, Daosen Sun

Featured Project

# PROBLEM

Many students found electromagnetics a difficult subject to master partly because electromagnetic waves are difficult to visualize directly using our own eyes. Thus, it becomes a mathematical abstract that heavily relies upon mathematical formulations.

# SOLUTION OVERVIEW

We focus on using AR/VR technology for large-scale, complex, and interactive visualization for the electromagnetic waves. To speed up the calculation, we are going to compute the field responses and render the fields out in real-time probably accelerated by GPU computing, cluster computation, and other more advanced numerical algorithms. Besides, we propose to perform public, immersive, and interactive education to users. We plan to use the existing VR equipment, VR square at laboratory building D220 to present users with a wide range of field of view, high-resolution, and high-quality 3D stereoscopic images, making the virtual environment perfectly comparable to the real world. Users can work together and interact with each other while maneuvering the virtual objects. This project also set up the basis for us to develop digital-twins technology for electromagnetics that effectively links the real world with digital space.

# COMPONENTS

1.Numerical computation component: The part that responsible for computing the field lines via Maxwell equations. We will try to load the work on the GPU to get better performance.

2.Graphic rendering component: The part will receive data from the numerical computation component and use renderers to visualize the data.

3.User interface component: This part can process users’ actions and allow the users to interact with objects in the virtual world.

4.Audio component: This part will generate audio based on the electromagnetic fields on charged objects.

5.Haptic component: This part will interact with the controller to send vibration feedback to users based on the field strength.

# CRITERIA OF SUCCESS

Set up four distinct experiments to illustrate the concept of four Maxwell equations. Students can work together and use controllers to set up different types of charged objects and operate the orientation/position of them. Students can see both static and real-time electromagnetic fields around charged objects via VR devices. Achieve high frame rates in the virtual world and fasten the process of computation and using advanced algorithms to get smooth electromagnetic fields.

# WHAT MAKES OUR PROJECT UNIQUE

We will build four distinct scenarios based on four Maxwell Equations rather than the one Gaussian’s Law made by UIUC team. In these scenarios, we will render both electric and magnetic field lines around charged objects, as well as the forces between them.

The experiments allow users to interact with objects simultaneously. In other words, users can cooperate with each other while conducting experiments. While the lab scene made by UIUC team only allows one user to do the experiment alone, we offer the chance to make the experiment public and allow multiple users to engage in the experiments.

We will use different hardware to do the computation. Rather than based on CPU, we will parallelize the calculation and using GPU to improve the performance and simulate large-scale visualization for the fields to meet the multi-users needs.

Compared to the project in the UIUC, we will not only try to visualize the fields, but also expand the dimension that we can perceive the phenomena i.e., adding haptic feedback in the game and also using audio feedback to give users 4D experience.