Project

# Title Team Members TA Documents Sponsor
7 Tennis Ball Pick-up Machine
 Haopeng Jiang
Hengjia Yu
Shurui Liu
Yilin Xue
 design_document1.pdf
final_paper1.pdf
proposal1.pdf
 Gaoang Wang
# Tennis Ball Pick-up Machine
# members
Shurui Liu (shuruil2)

Yilin Xue (yilinx3)

Haopeng Jiang (haopeng4)

Hengjia Yu (hengjia2)

# Problem
In tennis training or competition, players and coaches often face a tedious and time-consuming task - picking up scattered tennis balls on the court. The traditional manual ball-picking method consumes physical energy and takes up valuable training time, especially in high-intensity training or large-scale competitions. The existing solutions either rely on manpower or lack intelligence and automation, which cannot meet the needs of modern tennis training. To address this issue, we plan to develop an automatic pickup tennis cart based on image recognition technology. This small car can accurately identify tennis balls scattered on the court, plan the optimal path, and quickly collect tennis balls through an efficient picking mechanism, thereby significantly reducing labor costs and improving training efficiency. Our innovation lies in combining image recognition technology with automated path planning to design an intelligent car that can independently complete tennis-picking tasks. Our design provides a convenient user experience for players and coaches and promotes the development of tennis in a more efficient and intelligent direction. Through this project, we hope to alleviate the physical burden on players and coaches, allowing them to focus more on technical improvement and tactical exercises while providing an innovative solution for the intelligent development of tennis.

# solution overview
Our solution aims to design a simple and efficient automatic pickup tennis cart that can identify tennis balls scattered on the court, plan the optimal path, and pick them up through mechanical devices into the collection container on the cart. The car is equipped with a movable picking mechanism that can navigate autonomously while achieving precise positioning and operation through cameras and visual detection algorithms. The team consists of two members majoring in Electrical Engineering (EE), one in Computer Engineering (CE), and one in Mechanical Engineering (ME). Members of the ECE program are responsible for developing image recognition algorithms based on computer vision to achieve the recognition and localization of tennis balls; EE professionals focus on designing electronic control systems and communication modules to ensure precise navigation and stable operation of the vehicle; Members of the ME profession are responsible for the mechanical structure design and power system optimization of the car, ensuring the efficiency and durability of the picking mechanism. This plan fully utilizes the team's professional background and existing resources to ensure that the project is implemented within technical feasibility and cost control, providing a simple and practical solution for tennis training.

# solution components
## Visual system

The visual system is responsible for real-time detection and localization of tennis balls scattered on the court, which is based on computer vision technology, capturing images of the court through cameras and using image processing algorithms to identify the position of the tennis ball. The algorithm will prioritize identifying the tennis ball closest to the car, calculate its relative direction, and provide data support for path planning. In addition, the visual system also has a dynamic tracking function, which can continuously update the position information of the tennis ball during the movement of the car, ensuring the efficiency and accuracy of the picking process.

## Control system

The control system is responsible for coordinating the operation of the visual system and mechanical structure. It receives the tennis ball position information provided by the visual system and plans the optimal path to control the direction and speed of the car's movement. The control system also integrates boundary detection and obstacle avoidance functions, which monitor the surrounding environment in real-time through sensors to avoid collisions between the car and the boundary of the field or other obstacles.

## Mechanical structure

The mechanical structure is the physical execution part of the car, mainly including the following key components:

Electric collection drum: installed at the front of the car and driven by an electric motor. When the car passes by the tennis ball, the drum will roll up the ball and send it into the collection frame, achieving efficient picking.

Lifting system: To increase the capacity of the collection box, the lifting system can automatically adjust the height according to the number of tennis balls in the collection box, ensuring that more tennis balls can be accommodated.

Mecanum wheels and control: The car uses Mecanum wheels as the mobile chassis, which can achieve omnidirectional movement, including forward and backward, left and right, and rotational motion. The control system precisely adjusts the speed and direction of each wheel, allowing the car to move flexibly in the narrow space of the court and quickly adjust its orientation to align with the target tennis ball.

# Criteria of Success
The success criteria of this project are mainly reflected in the following aspects:

Function implementation: The car can accurately identify tennis balls scattered on the court and plan the optimal path for picking them up, ensuring an efficient and error-free picking process.

Performance indicators: The recognition accuracy of the visual system should reach over 90%, the control system should achieve real-time path planning and obstacle avoidance, and the success rate of mechanical structure picking should exceed 95%.

User experience: The car is easy to operate, and users only need to start the device to complete the ball-picking task without complex settings or intervention.

Reliability: The car can adapt to changes in lighting, ensuring long-term reliability.

Scalability: The design has a certain degree of modularity and scalability, facilitating future functional upgrades or adapting to the needs of similar scenarios.

By meeting the above standards, our automatic pickup tennis cart will provide an efficient, intelligent, and practical solution for tennis training, significantly improving training efficiency and user experience.

Autonomous Behavior Supervisor

Shengjian Chen, Xiaolu Liu, Zhuping Liu, Huili Tao

Featured Project

## Team members

- Xiaolu Liu (xiaolul2)

- Zhuping Liu(zhuping2)

- Shengjian Chen(sc54)

- Huili Tao(huilit2)

## Problem:

In many real-life scenarios, we need AI systems not only to detect people, but also to monitor their behavior. However, today's AI systems are only able to detect faces but are still lacking the analysis of movements, and the results obtained are not comprehensive enough. For example, in many high-risk laboratories, we need to ensure not only that the person entering the laboratory is identified, but also that he or she is acting in accordance with the regulations to avoid danger. In addition to this, the system can also help to better supervise students in their online study exams. We can combine the student's expressions and eyes, as well as his movements to better maintain the fairness of the test.

## Solution Overview:

Our solution for the problem mentioned above is an Autonomous Behavior Supervisor. This system mainly consists of a camera and an alarm device. Using real-time photos taken by the camera, the system can perform face verification on people. When the person is successfully verified, the camera starts to monitor the person's behavior and his interaction with the surroundings. Then the system determines whether there is a dangerous action or an unreasonable behavior. As soon as the system determines that there are something uncommon, the alarm will ring. Conversely, if the person fails verification (ie, does not have permission), the words "You do not have permission" will be displayed on the computer screen.

## Solution Components:

### Identification Subsystem:

- Locate the position of people's face

- Identify whether the face of people is recorded in our system

The camera will capture people's facial information as image input to the system. There exists several libraries in Python like OpenCV, which have lots of useful tools. The identification progress has 3 steps: firstly, we establish the documents of facial information and store the encoded faceprint. Secondly, we camera to capture the current face image, and generate the face pattern coding of the current face image file. Finally, we compare the current facial coding with the information in the storage. This is done by setting of a threshold. When the familiarity exceeds the threshold, we regard this person as recorded. Otherwise, this person will be banned from the system unless he records his facial information to our system.

### Supervising Subsystem

- Capture people's behavior

- Recognize the interaction between human and object

- Identify what people are doing

This part is the capture and analysis of people's behavior, which is the interaction between people and objects. For the algorithm, we decided initially to utilize that based on VSG-Net or other developed HOI models. To make it suitable for our system or make some improvement, we need analysis and adjustment of the models. For the algorithm, it is a multi-branch network: Visual Branch: extracting visual features from people, objects, and the surrounding environment. Spatial Attention Branch: Modeling the spatial relationship between human-object pairs. Graph Convolutional Branch: The scene was treated as a graph, with people and objects as nodes, and modeling the structural interactions. This is a computational work that needs the training on dataset and applies to the real system. It is true that the accuracy may not be 100% but we will try our best to improve the performance.

### Alarming Subsystem

- Staying normal when common behaviors are detected

- Alarming when dangerous or non-compliant behaviors are detected

It is an alarm apparatus connected to the final of our system, which is used to report dangerous actions or behaviors that are not permitted. If some actions are detected in supervising system like "harm people", "illegal experimental operation", and "cheating in exams", the alarming system will sound a warning to let people notice that. To achieve this, a "dangerous action library" should be prepared in advance which contains dangerous behaviors, when the analysis of actions in supervising system match some contents in the action library, the system will alarm to report.

## Criteria of Success:

- Must have a human face recognition system and determine whether the person is in the backend database

- The system will detect the human with the surrounding objects on the screen and analyze the possible interaction between these items.

- Based on the interaction, the system could detect the potentially dangerous action and give out warnings.

## DIVISION OF LABOR AND RESPONSIBILITIES

All members should contribute to the design and process of the project, we meet regularly to discuss and push forward the process of the design. Each member is responsible for a certain part but it doesn't mean that this is the only work for him/her.

- Shengjian Chen: Responsible for the facial recognition part of the project.

- Huili Tao: HOI algorithm modification and apply that to our project

- Zhuping Liu: Hardware design and the connectivity of the project

- XIaolu Liu: Detail optimizing and test of the function.

Project Videos