Project
| # | Title | Team Members | TA | Documents | Sponsor |
|---|---|---|---|---|---|
| 23 | FPGA-based object tracking, obstacle avoidance, and voice-activated trolley |
Haomin Wang Jiarun Hu Yang Zhou Yihang He |
Tielong Cai | design_document3.pdf final_paper2.pdf proposal1.pdf |
Said Mikki |
| # Members: - Yang Zhou [yangz15] - Haomin Wang [haominw3] - Yihang He [yihangh2] - Jiarun Hu [jiarunh2] # Problem: Nowadays the development of electric vehicles today has become a trend. At the same time, more and more new energy vehicle startups like to equip their cars with intelligent systems. However, existing SOCs are always based on non-real-time operating systems and need to meet the real-time property and safety of the in-vehicle system. Common systems which are based on CPU + GPU tend to have high energy consumption, which will ha a negative impact on the endurance of the vehicle. Therefore, designing a system with low energy consumption and high real-time performance is necessary. # Solution Overview In order to achieve low energy consumption and high real-time performance, our solution is to design a specific system to control our trolley based on FPGA, which combines four subsystems. The first subsystem processes real-time data from the other subsystems to control the trolley. The second subsystem is designed to detect the target object and send a tracking signal to the movement control subsystem. The third subsystem is to detect obstacles in the path of the trolley and send an avoidance signal to the first one. The last subsystem is to recognize natural language instructions from the operator and sends the corresponding signal to the movement control subsystem. By taking these four aspects into account, we will create our object tracking, obstacle avoidance, and voice-activated trolley. # Solution components: 1. **Trolley movement control subsystem:** The movement control subsystem will process real-time data from the other subsystems and produce the signal to control the movement of the trolley. Control signals will be passed through the FPGA port to the PCB board, which is connected to electric motors. The PCB board can generate current to control the speed of electric motors depending on the control signal so that our trolley can move as designed. 2. **Object tracking subsystem:** The object tracking subsystem will use a camera to catch the image in front of the trolley. FPGA will receive the image and process it to identify the location of the color block and generate suitable control signals based on the location of the color block so that the trolley can move toward the color block. 3. **Obstacle avoidance subsystem:** We will use ultrasonic sensors to detect obstacles in the path of the trolley. The FPGA will be used to process the signals from the sensors and control the movement of the trolley. The microcontroller should be programmed with algorithms for obstacle detection and avoidance. 4. **Voice-activated subsystem:** Our design target is that the trolley can recognize specific natural language instructions and act accordingly. Thus, we will design a voice-activated system and combine it with the control system of the trolley. In order to reduce the latency as well as achieve high recognition accuracy, we will build a CNN network on FPGA instead of LSTM or DSP procedure to do this task. And this voice-activated system will give the corresponding signal to the control part. # CRITERION FOR SUCCESS: 1. The trolley should be able to move at a reasonable speed so that it can avoid obstacles and respond to voice commands in a timely manner. The movement control subsystem will also be able to process conflicting instructions and produce the correct signal to control the movement. The subsystem needs to be secure and reliable. 2. The trolley should be able to use a camera to detect a color block and move toward the color block. This can be measured by testing if the trolley can follow the movement of the color block closely. 3. The trolley should be able to detect obstacles accurately and reliably using its sensors and cameras. This can be measured by testing the trolley's ability to detect and avoid obstacles of different sizes and shapes. 4. The trolley should be able to recognize and respond to specific voice commands accurately and reliably. This can be measured by testing the trolley's ability to understand a range of voice commands and respond accordingly. # DISTRIBUTION OF WORK: ## Yang Zhou, Electrical Engineering: Design and implement the trolley movement subsystem. Implement and test the way control subsystems interact with other subsystems. ## Haomin Wang, Computer Engineering: Design and implement the object tracking subsystem. Test the trolley's ability to detect and follow the color block. ## Yihang He, Computer Engineering: Design and implement the obstacle avoidance subsystem. Test the trolley's ability to detect and avoid obstacles of different sizes and shapes. ## Jiarun Hu, Electrical Engineering: Design and implement the Voice-activated subsystem. Test the trolley's ability to recognize natural language instruction and control the movement of the trolley. |
|||||