Project

# Title Team Members TA Documents Sponsor
4 Actions to Mosquitoes
 Lumeng Xu
Peiqi Cai
Xiangmei Chen
Yang Dai
 design_document2.pdf
final_paper2.pdf
proposal1.pdf
 Said Mikki
# Team Members
Xiangmei Chen [xc47]
Peiqi Cai [peiqic3]
Yang Dai [yangdai2]
Lumeng Xu [lumengx2]

# Title
Actions to Mosquitoes

# Problem
Many of us get bitten by mosquitoes without notice. We come up with a device that can distinguish by sound whether a mosquito exist in a given area and take actions to keep it away. Solutions existing in the market include mosquito spray, insect-repelling lamp, and mosquito-repellent incense. However, they work continuously, and people may get uncomfortable with its smell. It would be less disturbing and resource saving if the device only reacts when a mosquito approaches.

# Solution Overview
In order to have in-time response of mosquitoes, we first need a device to detect sounds of mosquitoes. After the sound is collected, we need to process the signal to tell if a mosquito presents. If the presence is true, an actuator will take actions to keep the mosquitoes away.

# Solution Components
[Sound Detecting Subsystem] A sound detecting device, could be high accuracy microphone that can capture the sound of mosquitoes since they produce a characteristic buzzing sound when they fly, which varies depending on the species and gender. The frequency of the sound that the system capturing can be set to the range of frequencies of the mosquitoes to further improve accuracy.
[Signal Processing Subsystem] A signal processor that can analyze the sound and identify the presence and type of mosquitoes. The signal processor could use a machine learning or other algorithms, or a frequency filter to distinguish the mosquito sound from other noises.
[Mechanical Subsystem] An actuator that can take actions to keep the mosquitoes away. Depending on the desired effect, the actuator could emit a high-frequency sound that repels mosquitoes, a chemical spray that kills or deters them, or a device that could emit gas or light of specific wavelength that attract them and knock them down.

# Criterion for Success
Detection Accuracy: The device should be able to accurately detect the distinctive sound of mosquito wings flapping with a high degree of precision to minimize false positives (e.g., from other insects or ambient noise) and false negatives (failure to detect mosquitoes).
Responsiveness: Upon detecting a mosquito, the device should promptly activate the mechanical components to deter or eliminate the mosquito within a predefined time frame, ensuring efficient protection.
Coverage Area: The device must effectively monitor and protect a defined area, such as a standard-sized room, from mosquitoes, with clear specifications on its effective range.
User Interface: If applicable, any software interface for the device should be user-friendly and allow users to easily adjust settings, such as detection sensitivity or deterrent mechanisms.
Energy Efficiency: The device should operate efficiently, using a reasonable amount of power, and if battery-operated, should have a battery life that is practical for typical use cases (e.g., overnight use in a residential setting).
Safety: The device and its deterrent methods (such as acoustic waves or mosquito sprays) should be safe for use in the intended environment, not posing health risks to humans or pets.


# Distribution of Work
Peiqi Cai [EE]:
Responsible for the design and implementation of the microphone array and any other necessary sensors that are part of the hardware which collects the mosquito sounds. This will include circuit design, component selection, and integration of the sensors with the rest of the system.
Lumeng Xu [ECE]:
Develop the signal processing software that analyzes the audio data from the hardware to distinguish mosquito sounds. This includes writing the algorithm, possibly utilizing machine learning, and ensuring it can run efficiently in real-time.
Yang Dai [ECE]:
In charge of the overall system integration, ensuring that the hardware and software components communicate effectively. This student will also be responsible for the user interface, if applicable, and making sure that the software is user-friendly and robust.
Xiangmei Chen [ME]:
Design and test the mechanical components that take action to repel or eliminate mosquitoes. This could involve the design of the enclosure that houses the electronics, any moving parts for the actuation mechanism, and the dispersion system for the repellent if a spray is used. She will also ensure that the physical design adheres to safety and ergonomic standards.

Autonomous Behavior Supervisor

Shengjian Chen, Xiaolu Liu, Zhuping Liu, Huili Tao

Featured Project

## Team members

- Xiaolu Liu (xiaolul2)

- Zhuping Liu(zhuping2)

- Shengjian Chen(sc54)

- Huili Tao(huilit2)

## Problem:

In many real-life scenarios, we need AI systems not only to detect people, but also to monitor their behavior. However, today's AI systems are only able to detect faces but are still lacking the analysis of movements, and the results obtained are not comprehensive enough. For example, in many high-risk laboratories, we need to ensure not only that the person entering the laboratory is identified, but also that he or she is acting in accordance with the regulations to avoid danger. In addition to this, the system can also help to better supervise students in their online study exams. We can combine the student's expressions and eyes, as well as his movements to better maintain the fairness of the test.

## Solution Overview:

Our solution for the problem mentioned above is an Autonomous Behavior Supervisor. This system mainly consists of a camera and an alarm device. Using real-time photos taken by the camera, the system can perform face verification on people. When the person is successfully verified, the camera starts to monitor the person's behavior and his interaction with the surroundings. Then the system determines whether there is a dangerous action or an unreasonable behavior. As soon as the system determines that there are something uncommon, the alarm will ring. Conversely, if the person fails verification (ie, does not have permission), the words "You do not have permission" will be displayed on the computer screen.

## Solution Components:

### Identification Subsystem:

- Locate the position of people's face

- Identify whether the face of people is recorded in our system

The camera will capture people's facial information as image input to the system. There exists several libraries in Python like OpenCV, which have lots of useful tools. The identification progress has 3 steps: firstly, we establish the documents of facial information and store the encoded faceprint. Secondly, we camera to capture the current face image, and generate the face pattern coding of the current face image file. Finally, we compare the current facial coding with the information in the storage. This is done by setting of a threshold. When the familiarity exceeds the threshold, we regard this person as recorded. Otherwise, this person will be banned from the system unless he records his facial information to our system.

### Supervising Subsystem

- Capture people's behavior

- Recognize the interaction between human and object

- Identify what people are doing

This part is the capture and analysis of people's behavior, which is the interaction between people and objects. For the algorithm, we decided initially to utilize that based on VSG-Net or other developed HOI models. To make it suitable for our system or make some improvement, we need analysis and adjustment of the models. For the algorithm, it is a multi-branch network: Visual Branch: extracting visual features from people, objects, and the surrounding environment. Spatial Attention Branch: Modeling the spatial relationship between human-object pairs. Graph Convolutional Branch: The scene was treated as a graph, with people and objects as nodes, and modeling the structural interactions. This is a computational work that needs the training on dataset and applies to the real system. It is true that the accuracy may not be 100% but we will try our best to improve the performance.

### Alarming Subsystem

- Staying normal when common behaviors are detected

- Alarming when dangerous or non-compliant behaviors are detected

It is an alarm apparatus connected to the final of our system, which is used to report dangerous actions or behaviors that are not permitted. If some actions are detected in supervising system like "harm people", "illegal experimental operation", and "cheating in exams", the alarming system will sound a warning to let people notice that. To achieve this, a "dangerous action library" should be prepared in advance which contains dangerous behaviors, when the analysis of actions in supervising system match some contents in the action library, the system will alarm to report.

## Criteria of Success:

- Must have a human face recognition system and determine whether the person is in the backend database

- The system will detect the human with the surrounding objects on the screen and analyze the possible interaction between these items.

- Based on the interaction, the system could detect the potentially dangerous action and give out warnings.

## DIVISION OF LABOR AND RESPONSIBILITIES

All members should contribute to the design and process of the project, we meet regularly to discuss and push forward the process of the design. Each member is responsible for a certain part but it doesn't mean that this is the only work for him/her.

- Shengjian Chen: Responsible for the facial recognition part of the project.

- Huili Tao: HOI algorithm modification and apply that to our project

- Zhuping Liu: Hardware design and the connectivity of the project

- XIaolu Liu: Detail optimizing and test of the function.

Project Videos