Project

# Title Team Members TA Documents Sponsor
11 JengaBot - A Robotic System for Playing Jenga with Human
 Hengtie Zhu
Jiacheng Ye
Peiran Wei
Wangyihan Guo
 Pavel Loskot
# JengaBot - A Robotic System for Playing Jenga with Human

Team members (listed A-Z):

- Wangyihan Guo
- Peiran Wei
- Jiacheng Ye
- Hengtie Zhu

## Problem

While technologies such as 3D printing and embodied intelligence have progressed significantly, precise object manipulation in constrained spaces continues to be a complex problem. Limited space restrict massive robotic approach yet allow 3D-priting frame to participate in the workflow, thus we dropped out this problem to discover the possibility of combing 3D-priting frame with robotic approaches.

## Solution Overview

While confined spaces often restrict the deployment of bulky robotic systems, they present a unique opportunity to utilize customized 3D-printed frameworks. To address this spatial limitation, we dedicated our project to exploring the integration of compact 3D-printed structures with precision robotic mechanisms.

To achieve this integration, we engineered a three-axis motion control system designed to maneuver an operation unit precisely within the restricted workspace. We demonstrated the capabilities of this system by programming the machine to play Jenga interactively against a human opponent, a task requiring delicate block removal without collapsing the tower.

The success of this interactive system relies on continuous environmental monitoring. The real-time status of the Jenga tower is captured by four cameras strategically mounted at the corners of the 3D-printed framework, ensuring comprehensive visual coverage of the workspace from multiple angles.

The visual data captured by these cameras is continuously fed into a Raspberry Pi for processing. Utilizing predefined algorithmic strategies, the Raspberry Pi analyzes the physical state of the tower and directly drives the operation unit to execute the optimal next move, completing the automated feedback loop.

## Solution Components

- Three-axis motion control system: used to move the operation unit precisely in the constrained space.

- Operation unit: a specialized operational unit capable of executing complex physical interactions with the Jenga tower, including grasping, pushing, and translating the bricks.

- Camera: used to collect live visual data of the Jenga tower to Raspberry Pi.

- Raspberry Pi: a central controller that constructs a real-time digital model of the Jenga tower from visual data, directing the end-effector to execute predefined algorithmic strategies.

## Criterion for Success

1. The system could play the game with human opponents with reasonable react.
2. The system could handle scenarios out of rules, e.g. human manually make tower collapse or try to make a second move in one round.

## Distribution of the Work

Wangyihan Guo & Peiran Wei: Responsible for the controlling unit and cameras, including strategy written, the interface between software and hardware, etc.

Jiacheng Ye & Hengtie Zhu: Responsible for the design and construction of the three-axis moving system along with the operation.

Augmenting AR/VR with Smell

Baoyi He, Yingying Liu, Kaiyuan Tan, Xiao Wang

Featured Project

# TEAM MEMBERS

- **Kaiyuan Tan** (kt19)

- **Baoyi He** (baoyihe2)

- **Xiao Wang** (xiaow4)

- **Yingying Liu** (yl73)

# TITLE OF THE PROJECT

Augmenting AR/VR with Smell

# PROBLEM

Augmented Reality (AR) and Virtual Reality (VR) technologies are rapidly growing and becoming more prevalent in our daily lives. However, these technologies have not yet fully addressed the sense of smell, which is a critical aspect of human experience. The absence of scent in AR/VR experiences limits the immersive potential of these technologies, preventing users from experiencing a full sensory experience.

# SOLUTION OVERVIEW

The solution is to augment AR/VR experiences with smell, enabling users to experience a full sensory experience. This will be achieved by incorporating hardware and software components that can simulate various scents in real-time, in response to events in the AR/VR environment. The solution will consist of a scent-emitting device and software that can track and simulate scents based on the user's location and orientation in the AR/VR environment.

# SOLUTION COMPONENTS

The solution will consist of the following components:

- **Scent-emitting device**: This device will be designed to emit various scents in real-time. It will be portable and lightweight, making it easy for users to carry around during AR/VR experiences.

- **Scent simulation software**: This software will be designed to track the user's location and orientation in the AR/VR environment and simulate scents accordingly. The software will use various algorithms to determine the intensity and duration of scent emissions.

- **AR/VR hardware**: The solution will require AR/VR hardware to create the immersive environment. This hardware will include AR/VR headsets, controllers, and other peripherals necessary to interact with the AR/VR environment.

# CRITERION OF SUCCESS

The success of the project will be determined by the following criteria:

- **Immersive Experience**: The solution must provide an immersive AR/VR experience that incorporates smell as a key sensory input.

- **User Acceptance**: The solution must be accepted by users, who should be able to appreciate and enjoy the experience.

- **Technical Feasibility**: The solution must be technically feasible and reliable, with a low latency and high accuracy in scent simulation.

- **Scalability**: The solution should be scalable and adaptable to different AR/VR environments and hardware configurations.

- **Safety**: The solution must be safe for users and the environment, with proper ventilation and control mechanisms to prevent any harm or discomfort caused by excessive or inappropriate scent emissions.

# DISTRIBUTION OF WORK

- Model various scenerios based on AR/VR hardware. *(Tan)*

- Design algorithms which output the intensity and duration of scents based on the constructed scenerios. *(He & Liu)*

- Merge the scene with scents smoothly. *(He & Wang & Liu)*

- Design a protable scent-emitting device. *(Wang)*

- Test using real scents, invite people to experience and adjust based on feedback. *(All)*