Project
| # | Title | Team Members | TA | Documents | Sponsor |
|---|---|---|---|---|---|
| 12 | ROBOTIC ARM INTEGRATED INTO WHEELCHAIR WITH MR INTERFACE |
Xingru Lu Yilin Wang Yinuo Yang Yunyi Lin |
design_document1.pdf final_paper1.pdf final_paper2.pdf final_paper3.pdf proposal1.pdf |
Liangjing Yang | |
| #TEAM MEMBER Yunyi Lin, yunyil3 Yinuo Yang, yinuoy4 Xingru Lu, xingrul2 Yilin Wang, yilin14 # PROBLEM Wheelchair users often face significant challenges when interacting with objects beyond their immediate reach, particularly behind them. Without external assistance, tasks such as pressing buttons or navigating through environments with complicated surroundings can become difficult. These difficulties are compounded when operating independently, highlighting the need for supplementary support to simplify routine activities. Additionally, wheelchair users may struggle with limited situational awareness, as their field of view is primarily forward-facing. As a result, there is a pressing need for innovative solutions that enhance both accessibility and autonomy, enabling wheelchair users to interact more conveniently with their surroundings. # SOLUTION OVERVIEW Our solution integrates a rear-facing camera that streams real-time visuals to a Mixed Reality (MR) interface, allowing wheelchair users to gain visual awareness of their surroundings, including blind spots behind them. Additionally, a robotic arm mounted at the back of the wheelchair can be controlled through MR, enabling users to perform assistive actions such as pressing buttons and interacting with objects beyond their physical reach. This system enhances both situational awareness and independent mobility, providing a more intuitive and convenient way for users to navigate and interact with their environment. # SOLUTION COMPONENT ## OPEN MANIPULATOR-P ROBOTIC ARM The Open Manipulator-P robotic arm will be responsible for helping disabilities to extend their reachable area and assist them with tasks in their blind spots, such as pressing elevator buttons behind them. ## APPLE VISION PRO Apple Vision Pro will be responsible for detecting user’s hand movements and giving feedback to the user. It provides a camera matrix consisting of eight depth cameras and RGB cameras. These cameras will be helpful in spatial computing and object detection. ## MIXED REALITY INTERFACE The mixed reality interface will provide a live view from behind the wheelchair, allowing people with disabilities to see from their blind spots. The interface will also offer feedback when user tries to control the robotic arm, such as draggable buttons. These feedbacks ill enhance the interaction between user and robotic arm. # CRITERIA FOR SUCCESS - Precision: The robotic arm should reliably press buttons with a diameter of at least 35mm, which is a common size of elevator buttons. The force applied must be sufficient to activate buttons without excessive pressure that could cause damage or failure. - Clear Vision Pro View: Users should be able to see both the front and rear environments through Vision Pro, while also adjusting the robotic arm’s perspective to gain a broader field of view. - Safety and Stability: The system must ensure that wheelchair stability is not compromised during operation. Movements of the robotic arm should not cause the wheelchair to become unbalanced. - Low Latency: The system should ensure smooth and intuitive control. The latency should be low enough that it does not disrupt normal usage or cause noticeable delays in operation. |
|||||