Project

# Title Team Members TA Documents Sponsor
12 ROBOTIC ARM INTEGRATED INTO WHEELCHAIR WITH MR INTERFACE
 Xingru Lu
Yilin Wang
Yinuo Yang
Yunyi Lin
 design_document1.pdf
proposal1.pdf
 Liangjing Yang
#TEAM MEMBER
Yunyi Lin, yunyil3
Yinuo Yang, yinuoy4
Xingru Lu, xingrul2
Yilin Wang, yilin14

# PROBLEM

Wheelchair users often face significant challenges when interacting with objects beyond their immediate reach, particularly behind them. Without external assistance, tasks such as pressing buttons or navigating through environments with complicated surroundings can become difficult. These difficulties are compounded when operating independently, highlighting the need for supplementary support to simplify routine activities. Additionally, wheelchair users may struggle with limited situational awareness, as their field of view is primarily forward-facing. As a result, there is a pressing need for innovative solutions that enhance both accessibility and autonomy, enabling wheelchair users to interact more conveniently with their surroundings.

# SOLUTION OVERVIEW

Our solution integrates a rear-facing camera that streams real-time visuals to a Mixed Reality (MR) interface, allowing wheelchair users to gain visual awareness of their surroundings, including blind spots behind them. Additionally, a robotic arm mounted at the back of the wheelchair can be controlled through MR, enabling users to perform assistive actions such as pressing buttons and interacting with objects beyond their physical reach. This system enhances both situational awareness and independent mobility, providing a more intuitive and convenient way for users to navigate and interact with their environment.

# SOLUTION COMPONENT

## OPEN MANIPULATOR-P ROBOTIC ARM

The Open Manipulator-P robotic arm will be responsible for helping disabilities to extend their reachable area and assist them with tasks in their blind spots, such as pressing elevator buttons behind them.

## APPLE VISION PRO

Apple Vision Pro will be responsible for detecting user’s hand movements and giving feedback to the user. It provides a camera matrix consisting of eight depth cameras and RGB cameras. These cameras will be helpful in spatial computing and object detection.

## MIXED REALITY INTERFACE

The mixed reality interface will provide a live view from behind the wheelchair, allowing people with disabilities to see from their blind spots. The interface will also offer feedback when user tries to control the robotic arm, such as draggable buttons. These feedbacks ill enhance the interaction between user and robotic arm.

# CRITERIA FOR SUCCESS

- Precision: The robotic arm should reliably press buttons with a diameter of at least 35mm, which is a common size of elevator buttons. The force applied must be sufficient to activate buttons without excessive pressure that could cause damage or failure.
- Clear Vision Pro View: Users should be able to see both the front and rear environments through Vision Pro, while also adjusting the robotic arm’s perspective to gain a broader field of view.
- Safety and Stability: The system must ensure that wheelchair stability is not compromised during operation. Movements of the robotic arm should not cause the wheelchair to become unbalanced.
- Low Latency: The system should ensure smooth and intuitive control. The latency should be low enough that it does not disrupt normal usage or cause noticeable delays in operation.

ML-based Weather Forecast on Raspberry Pi

Xuanyu Chen, Zheyu Fu, Zhenting Qi, Chenzhi Yuan

Featured Project

#Team Members

Zheyu Fu (zheyufu2@illinois.edu 3190110355)

Xuanyu Chen (xuanyuc2@illinois.edu 3190112156)

Chenzhi Yuan (chenzhi2@illinois.edu 3190110852)

Zhenting Qi (qi11@illinois.edu 3190112155)

#Problem

Weather forecasting is crucial in our daily lives. It allows us to make proper plans and get prepared for extreme conditions in advance. However, meteorologists always get it wrong half of the time and still keep their job :) To overcome the limitations of traditional weather forecasting, machine learning models have become increasingly important in weather forecasting. Building our own weather forecast ML system is a perfect idea for us to analyze vast amounts of area data and generate more accurate and timely weather predictions on the go in our surrounding areas.

#Solution Overview

A weather forecast system can be created by using a few different hardware components and software tools. Our solution mainly consists of two parts. For weather measurement and data collection, temperature, humidity, and barometric pressure sensors are considered the main components. A machine learning-based algorithm is to be applied for data analysis and weather predictions.

#Solution Components

##Hardware Subsystem

Due to the complexity of weather conditions, our system incorporates the following weather indicators and their corresponding collectors:

-a barometric pressure sensor, a temperature sensor, and a humidity sensor

-a digital thermal probe for heat distribution

-an anemometer for wind speed, wind vane for wind direction, and rain gauge for precipitation

The aforementioned equipment would be integrated into a single device, and weatherproof enclosures are needed to protect it. Plus, a Raspberry Pi, either with built-in wireless connectivity or a WiFi dongle, is required for conducting computations.

##Software Subsystem

A practically usable weather forecast system is supposed to make reliable predictions for real-world multi-variable weather conditions. We apply Machine Learning techniques to suffice such generalization to unseen data. To this end, a high-quality dataset for training and evaluating the Machine Learning model is required, and a specially designed Machine Learning model would be developed on such a dataset. Once a well-trained system is obtained, we deploy the such model on portable devices with easy-to-use APIs.

#Criterion for Success

1. The weather measurement prototype with sensors should be able to accurately collect the temperature, humidity, and barometric pressure. etc.

2. A machine learning algorithm should be successfully trained to make predictions on the weather conditions: rainy, sunny, thunderstorm, etc.

3. Our system can forecast the weather in Haining, in real-time, and/or longer-period forecast.

4. The forecasted weather information could be demonstrated elegantly through some UI interface. A display screen would be a baseline, and an application on phones would be extra credit if time permitted.

5. Extra: Make our own weather dataset for Haining. If good, make it open-source.

#Work Distribution

**EE Student Zheyu Fu**:

-Design the sensor module circuit

-Development of visualization interface

**ECE Students Xuanyu Chen & Zhenting Qi**:

-Weather data collection and analysis

-Build and test Machine Learning model on Raspberry Pi

**ME Student Chenzhi Yuan**:

-Physical structure hardware design

-Proper distribution of the sensors to collect accurate data on temperature, humidity, barometric pressure, etc.