Project

# Title Team Members TA Documents Sponsor
19 Autonomous Vehicle with Sign Recognition and Obstacle Clearing through Wi-Fi
 Pai Zhang
Pengyu Zhu
Rui Zhang
Wendi Wang
 design_document1.pdf
proposal1.pdf
# Team Members
Zhang Pai (paiz4)
Zhang Rui (rui14)
Wang Wendi (wendiw2)
Zhu Pengyu (pengyuz4)

# Problem
In recent years, autonomous vehicles have become popular in industrial research and daily life, with advanced functions like reading signal lights and giving ways to pedestrians. However, these advanced autonomous vehicles are seldom introduced to campus as shuttle buses or trash carts due to high costs. It would bring much convenience to students' campus life if some low-cost and efficient autonomous vehicles help students commute and provide some safety guarantees in obstacle detection and self-speed control by reading speed limit signs. We aim to develop the core of such low-cost autonomous vehicles to potentially take the responsibilities of commuting and obstacle-clearing, enabling wireless transmission to provide efficiency via campus Wi-Fi.

# Solution Overview
We aim to develop an autonomous vehicle supporting obstacle clearing and speed adjustment. Obstacle clearing requires a camera to detect in-the-way objects and a robot arm to collect those objects. Speed adjustment also calls for a camera capturing speed limit signs, crosswalks, pedestrians, etc. Object detection and image analysis are supported by computer vision techniques and algorithms like YOLO, R-CNN, and Swin Transformer. There are two potential places to perform such object detection, uploaded to the cloud server via Wi-Fi signals or based on the vehicle’s local computing device. We also target setting up a “memory” for the autonomous vehicle, enabling it to “memorize” and reuse some speed limit signs and object images when Wi-Fi signals are disabled. Further investigation includes enhancing the quality of object detection and image analysis via data augmentation to prepare such autonomous vehicles for a more complex working environment.

# Solution Components

## Image Capture and Simple Analysis Subsystem
A Raspberry Pi, a low-cost single-board computer, supporting image capturing and simple image analysis when Wi-Fi signals are not supported.

## Autonomous Vehicle Subsystem
An autonomous vehicle subsystem supported with Arduino control, which can communicate with the remote cloud server for more complex and powerful image analysis through Wi-Fi transmission.

## Thorough Image Analysis Subsystem
A remote cloud server with GPUs supporting powerful image analysis via common Computer Vision algorithms like YOLO, R-CNN, and Swin Transformer.

## Output Subsystem
A robot arm controlled by Arduino performing object clearing and collecting.
A self-adjusted speed vehicle responded to the analysis of speed limit signs.

# Criterion for Success
The Raspberry Pi successfully captures images of obstacles and speed limit signs.
The control system (mainly Arduino) receives images captured by the Raspberry Pi and sends them to the remote cloud server in time through Wi-Fi.
The remote cloud server successfully detects an obstacle and analyzes the correct speed limit, sending back the results of image analyses to the autonomous vehicle through Wi-Fi.
The robot arm successfully pulls up obstacles and collects them in a certain area of the vehicle.
The vehicle successfully lowers its speed after receiving information from the remote cloud server.
(Optional) If no Wi-Fi signal is detected, the vehicle can use its local computing device to analyze the speed limit signs and control its functioning.

# Distribution of Work
Zhang Pai: Data processing and repetitive work.
Zhang Rui: Documentation and reporting, hardware test.
Wang Wendi: Hardware selection and setup, software algorithm design.
Zhu Pengyu: Software test, robot arm test.

simplified device for fasteners counter

Zhiwei Shen, Shuyang Wang, Yijian Yang, Jinsong Yuan

Featured Project

# PROBLEM DESCRIPTION

Lots of Industrial manufacturers need to realize real-time, efficient and accurate automatic counting of the assembly line products in the stages of production and transportation. On a standardized assembly line with stable operations, equal intervals and boxed objects the control system with infrared detection and microchip as the control core is effective and simple to implement. However, due to cost considerations, downstream manufacturers often prefer faster and less standardized assembly line operations during product inspection. Those unpackaged objects may have complex and changeable structures, and different kinds may have very similar structures. Moreover, the intervals and directions of these products on the assembly line are all random, which greatly increases the difficulty of monitoring, as well as achieving subsequent controlling purposes such as mechanical classification or equal-quantity loading.

After we discussed with people from a manufacturer, we realized their needs in this regard, so we decided to design an effective and low-cost device that realizes real-time monitoring and controlling towards specific industrial products with complex and random structures. From our investigations, we found that some factories use image recognition technology to achieve this goal, which turned out to be insufficient and costly because of their improper design. The manager of company complained about the stability, flexibility and fee of the traditional ways. After listening to the manager, we decide to implement our own ways to count line products, and our target is to increase the stability, flexibility and lower the cost.

By doing some research online, we confirmed that the most common monitoring system is still the infrared detection and microcontroller/PLC, which is effective for most assembly lines with products in boxes. And some newly developed approaches are based on cameras and computer vision, which we think are very potential but costly. Also, we found some other engineers still used simple infrared detection to achieve non-boxed objects monitoring. However, they met similar accuracy issues, like when two objects are too close to each other. Not to mention the objects that we are going to detect have much more complicated structures. In a word, we didn’t find any other monitoring system without using computer vision that can achieve our accuracy goal. So, our first major task is to come up with a better algorithm. We may also try pressure sensors, which is rarely used in assembly line object counting. In fact, we are going to investigate the feasibility of our idea by doing some experiments at their factory this week.

The scope of this specific problem might involve designing an embedded system with sensors and microcontroller unit to achieve the industrial control purpose, as well as programming and data analysis. Moreover, it may involve some knowledge about IoT because we also hope to use network module to transfer data and improve the automation level.

# solution overview

We plan to use infared sensor to dector the fasteners on the pipeline. We have two different kind of infared sensor in schedule. The first type could detect whether there exists objects within one meter, and the other one, which uses laser at the same time, can measure the distance between the surface of fasteners and the detector. The first one is cheaper but the second one could provide more imformation. We would choose in terms of real condition. There are also some alternative plans: we plan to use pressure sensor to count the total mass coming in and then calculate the number; acoustic rangefinder is another way to detect the distant in place of the second kind of infared sensor, and we will choose this plan if the original plan doesn't work so well.

Then, we plan to use PRI or PLC to process imformation. RPI is more powerful and enable us to write more complex code and develop some complicated functions such as classification of fasteners and nerual network which can analyze cutting pieces of fasteners, but PLC would be more stable in industry environemnt. The choice is mainly determined by real industry environment and the comments from manufacturers. We tend to use PLC to handle imformation from detectors and command the pipeline.

As for pipeline, workers put fasteners on the track. During the transportation, our device would count the number and in the end of pipeline, fasteners would be packed. After collecting enough fasteners, our machine would stop the pipeline.

# Solution Components

- Mono-chip（Raspberry Pi）

Price: around 300￥

Function: Receiving the data collected by the detector, processing it to get the number of fasteners that have passed, and transmitting the data to the remote-control center through the wireless interface.

We are going to use the neural network for modeling and use this model to count.

- Pressure-sensitive sensor

Price: 10￥-200￥

Function: Measuring the real-time weight on the sensor to assist in determining the number of products passed.

- Infrared sensor

Price: Already have

Function: Determining whether there is product passing.

- Laser rangefinder

Price: 60￥-200￥

Function: Measuring the distance between the product to the boundary of the conveyor belt.

- Acoustic rangefinder

Price: 200￥-300￥

Function: Measuring the distance between the product to the boundary of the conveyor belt.

- Remote-control Center

Price: Already have

Function: Receiving the data transmitted by the mono-chip, presenting the past products so far, and commanding every component according to that.

# CRITERION FOR SUCCESS

- High accuracy is required. The counter should have a error rate at 1%+-0.1%.

- The classifier is supposed to perform well, then the device can be migrated to a similar pipeline. The device is a kind of baler. When the input products are not of the same kind, if there is no classification function, packaging errors are likely to occur.

- The process of counting and classifying should take less time.

- The devicey should be stable enougth to be used in manifacture.

- Additional Function: Operator can control the machine and see results easily and remotely.

# sponsor

This project is well connected to industry. The company that sponsors us is 杭州六联机械科技有限公司(Hangzhou Liulian Machinery Technology Co., Ltd.) and the manager with whom we talked is 杨向峰(Xiangfeng Yang).