Project

# Title Team Members TA Documents Sponsor
13 Epicast: Augmented Board Game
 Di Wu
Jianli Jin
Yueshen Li
Zihao Zhu
 design_document1.pdf
design_document2.pdf
proposal1.pdf
 Yushi Cheng
# RFA


## Team Member

- Yueshen Li, yueshen7
- Jianli Jin, jianlij2
- Di Wu, diw10
- Zihao Zhu, zihao9


## Title

Epicast: Augmented Board Game

## Problem

Dungeons & Dragons (D&D) is a game that thrives on the breadth of imagination and the depth of interaction. It empowers players to construct elaborate worlds and characters that stem from the vast expanse of their creativity. This unfettered freedom not only offers a canvas for creation and interaction but also makes each gameplay experience profoundly personal and distinct. However, the richness of these imagined scenes and environments is often bottlenecked by the need for verbal expression and lacks an intuitive, sensory display. This limitation hampers the visualization of the game's full potential, presenting a hurdle for some players and constricting the game's broader allure. In addition, the Dungeon Master (DM) which is one of the players serving as the game's narrative architect, is often burdened with extensive preparatory work, juggling game mechanics with storytelling, which can be arduous and time-consuming.


## Solution Overview

"Project Epicast" is a comprehensive system designed to enhance the Dungeons & Dragons gaming experience. Central to this system is a GPT-powered AI that serves as an automated Dungeon Master, guiding gameplay with intelligent narrative creation and player interaction.

The visual aspect is handled by an overhead projector, capable of displaying intricate game scenes, animations, and simulating actions such as dice rolls directly onto the gaming surface. Its height is adjustable for optimal image quality. Gesture recognition is enabled through a sophisticated camera, allowing for intuitive control and the ability to capture memorable moments. Audio immersion is provided by integrated speakers and microphones for voice commands, narrative flow, and dynamic sound effects. Completing the sensory experience are ambient lights that adjust to the game's mood, providing synchronized lighting effects.

Lastly, while our focus is on enhancing the D&D experience, it's important to recognize that our board game experience-augmenting module has widespread applications. The demand for immersive, enriched board game interactions extends beyond D&D. Our target example of D&D serves as an ideal prototype for a broader market of board games seeking similar enhancements for a more engaging player experience.

## Solution Components

### [Real-time Data Processing System]

Real-time data processing system is used to capture, process and generate data in Real time. It consists of a data transmission module, a sophisticated camera, a projector, an integrated speaker and microphone, and an ambient light set.

- Data transmission module is responsible for transmitting the data obtained from sensors to the processing unit of the system. And it should be responsible for transmitting the next instructions of sensors from processing unit and some data that sensors need to express. This can be done through a wired or wireless connection.

- Camera is the core component of the real-time data processing system, used to capture image or video data. Those data will contain vital information like people’s hand gestures. The camera can be an ordinary USB camera, or it can be a high-resolution, high-speed industrial-grade camera.

- Projector is another core component of the real-time data processing system. Projection is a technique for projecting an image or video onto an object or plane. In our setting, the projector needs to project images on a wall or slab to inform players of the game status. And the processing unit should send those related data to projector in real time. Same as Camera, the projector can be an ordinary USB camera, or it can be a high-resolution, high-speed industrial-grade camera.

- Microphone will gather the voice of people and transform them into digital audio, and speaker will transform the digital data sent by processing unit to real voice. An intergrated speaker and microphone will deal with voice input and output of our whole model.

- Ambient light set is used to provide ambient light source, provide sufficient lighting conditions that can adjust to the game's mood. So that the light set will provide synchronized lighting effects to enhance the player’s experience. Also, the light set may help to improve the visibility and recognition ability of the image according to the camera feedback and improve the recognition accuracy and accuracy of the real-time recognition system.



### [GPT-Core DM System]

GPT-Core DM system acts as an assistant to Dungeon Masters (DM), providing support and assistance during gameplay. Through modeling and data training, GPT-Core as Dungeon Masters assistant should perform the following some basic functions and complete the game:

- **Generate adventure missions and plot**: The DM can provide some key information to GPT-Core, such as mission type, location, character, etc., and GPT-Core can then generate a complete adventure mission with a reasonable game plot including enemies, puzzles, rewards, etc.

- **Generate player’s character and NPC (Non-Player Character)**: The DM can use GPT-Core to ask players’ requirement of their willing characters, and then generate out their corresponding characters with balanced properties such as their backstory, personality traits, goals, etc. Also, GPT-Core can generate and provide detailed information of many NPCs easily to enhance game quality.

- **Generate other detailed information**: DM can use GPT-Core to generate any detailed information, for an environment such as the layout of the room, decoration, smell, etc. And GPT-Core can generate vivid descriptions of the environment, allowing the player to better engage with the game world. Also, for interaction of player’s character and NPCs, GPT-Core may help to provide detailed descriptions of NPCs’ reactions according to player’s operations. That will make user immerse into game better.



### [Sensor Assistance System]

Most of the time, we need to adjust projector orientation to let it project the screen to place we want or the camara’s orientation to make players’ hand gesture be captured. Design in mechanical engineering involves the physical structure and installation of the projector and the camera. Here are some common design considerations:

- **Mounting bracket**: To securely mount the projector and camera in the desired position, a suitable bracket or mounting bracket needs to be designed. These brackets should be able to adapt to different mounting environments and provide adjustable features to fine-tune projection angles and positions.

- **Adjustment mechanism**: To facilitate adjustment and alignment of the projector and camera, adjustable mechanisms such as rotation and tilt mechanisms can be designed to adjust under different projection angles and positions.

- **Cooling system**: Projector and camera, especially projector, will generate heat during operation, so an effective cooling system may be necessary to be designed to ensure the stable operation of the projector and prevent overheating.

- **Dust and protection**: To protect the projector and camera from dust, moisture and other external factors, it is necessary to design appropriate dust and protection measures, such as filters, seals, etc.

- Other possible small mechanisms can be provided further to assist the rest sensors...


## Criterion for Success

- Hand gesture and audio detections based on AI model are applied to depict players’ action.
- Split game sense projector region and gesture detect region which are supposed to be a multi-module integrated hardware system ought to improve game experience.
- Create a Dungeon Master AI using GPT as a core.
- Create a better experience with a free-moving projector and several ambient lights.

## Distribution of Work

- Yueshen Li will work on identification module and real-time signal transition system

- Jianli Jin will work on GPT-DM model building and data training

- Di Wu will work on hardware appliance construction and data logging system

- Zihao Zhu will work on realistic design and extension module combination.

Fountain show

Dingyi Feng, Tianli Ling, Zhelun Lu, Shibo Zhang

Featured Project

## Team

- Dingyi Feng(dingyif2)

- Tianli Ling(tling3)

- Zhelun Lu(zhelunl2)

- Shibo Zhang(shiboz2)

## Problem:

A fountain show on campus can make students feel more relaxed after class. However, some fountain shows only have monotonous, stiff, and single actions. Besides, they cannot automatically generate action and light effects. Compared with large fountain shows, small and medium-sized fountain shows have the advantage of time and space. In most cases, a large fountain show only has preloaded music which cannot be decided by audiences. Large fountain shows also require people to design the action and light effects for each music, which takes lots of time and effort. Compared with the large fountain show, our small fountain show will be more energy-efficient and environmentally friendly. During the COVID-19 pandemic, large fountain shows might result in large crowds gathering, but a small fountain show can reduce the risk of infection.

## Solution Overview:

Our fountain show would be built at a pool on the sourthwest of the main lake on campus. By manually programing, the fountain show could realize changing lighting effects and movements. Besides, our fountain show could also identify the music which was imported into our system, and automatically generate the lighting effects and movements with the music. If time permits, we will strengthen the human-computer interaction of our product. To be specific, people could scan the QR code or use our online system to choose the music they want, so that they can enjoy the fountain show at any time.

## Solution Components:

### Control Subsystem:

- Music colleccting and analyzing subsystem: Computer that can import music signals and analyze them.

- Converting subsystem: After music signal is analyzed, we need computer to convert useful signals into digital signals. Digital signals will be used to control LEDs and other mechanical subsystems.

### Mechanical Subsystem:

- Pump Subsystem: Water pump that can pump water from the lake. Valves will be used to control water’s flow rate of each nozzle.

- Lighting Subsystem: LEDs are needed to light our fountains. They are controlled by microprocessor on PCB. Their brightness and color can be changed with music.

- Motor Subsystem: Two motors are needed for each fountain nozzle to control the movements. The motors are controlled by microprocessor on PCB.

### Power subsystem:

- The pump is drived by DC power (12V 20~30A). PCB and computer will be drived by USB (5V 1A). Full module power supply with 12V and 5V output is needed.

## Criterion for Success:

- If it can successfully identify a piece of music and convert it to electrical signals that we need in controlling LED’s lighting and nozzle’s moving.

- If the whole system can work stably for a long time and whether it is safe to use without electric leakage or other problems.

- If music playing, fountain movements and LED lights are synchronized.

- If the fountain system is neat and whether the fountain performance is ornamental enough.