Final Demo

Description

The Final Demonstration (Final Demo) is the single, most important assignment in the course. It is the strongest measure of the success of your project. The evaluation focuses on the criteria of project completion, reliability, and professionalism. You will demo your full project to a group consisting of your Professor, your TA, and a few peer reviewers. Other guests (e.g. alumni, other course staff, visiting scholars, donors) may sometimes also be present.

Requirements and Grading

Students must be able to demonstrate the full functionality of their project to the instructors. If full functionality is not available, then students must be able to show the parts of the project that do function via the procedure listed in their Requirements and Verification Table. Credit will not be given for features which cannot be demonstrated, even if those features worked before and suddenly fail at the time of the final demo. Still, for any portion of the project which does not function as specified, students should have hypotheses and supporting evidence for what the problem may be.

The project team should be ready to justify design decisions and technical aspects of any part of the project (not just your own parts). Quantitative results are expected wherever applicable.

Grading is covered by the Demo Rubric, and is out of 150 points. Some of the key points are as follows:

  1. Completion: The project has been entirely completed.
  2. Thoroughness: Care and attention to detail are evident in construction and layout.
  3. Performance: Performance is completely verified, and operation is reliable.
  4. Understanding: Everyone on the project team must be able to demonstrate understanding of his/her technical work and show that all members have contributed significantly.

Submission and Deadlines

Sign-up for a demo time is handled through the PACE system. Again, remember to sign up for a peer review session as well.

Keebot, a humanoid robot performing 3D pose imitation

Zhi Cen, Hao Hu, Xinyi Lai, Kerui Zhu

Featured Project

# Problem Description

Life is movement, but exercising alone is boring. When people are alone, it is hard to motivate themselves to exercise and it is easy to give up. Faced with the unprecedented COVID-19 pandemics, even more people have to do sports alone at home. Inspired by "Keep", a popular fitness app with many video demonstrations, we want to build a humanoid robot "Keebot" which can imitate the movements of the user in real time. Compared to a virtual coach in the video, our Keebot can provide physical company by doing the same exercises as the user, thus making exercising alone at home more interesting.

# Solution Overview

Our solution to the create such a movement imitating robot is to combine both computer vision and robotic design. The user's movement is captured by a fixed and stabilized depth camera. The 3D joint position will be calculated from the camera image with the help of some neural networks and depth information from the camera. The 3D joint position data will be translated into the motor angular rotation information and sent to the robot using Bluetooth. The robot realizes the imitation by controlling the servo motors as commanded. Since the 3D position data and mechanical control are not ideal, we leave out the consideration of keeping robot's balance and the robot's trunk will be fixed to a holder.

# Solution Components

## 3-D Pose Info Translator: from depth camera to 3-D pose info

+ RealSense Depth Camera which can get RGB and depth frames

+ A series of pre-processors such as denoising, normalizing and segmentation to reduce the impact of noise and environment

+ Pre-trained 2-D Human Pose Estimation model to convert the RGB frames to 2-D pose info

+ Combine the 2-D pose info with the depth frames to get the 3-D pose info

## Control system: from model to motors

+ An STM32-based PCB with a Bluetooth module and servo motor drivers

+ A mapping from the 3-D poses and movements to the joint parameters, based on Inverse Kinematics

+ A close-loop control system with PID or State Space Method

+ Generate control signals for the servo motors in each joints

## Mechanical structure: the body of the humanoid robot

+ CAD drawings of the robot’s physical structure, with 14 joints (14 DOF).

+ Simulations with the Robotics System Toolbox in MATLAB to test the stability and feasibility of the movements

+ Assembling the robot with 3D print parts, fasteners and motors

# Criterion of Success

+ 3-D pose info and movements are extracted from the video by RealSense Depth Camera

+ The virtual robot can imitate human's movements in MATLAB simulation

+ The physical robot can imitate human's movements with its limbs while its trunk being fixed