ECE 110/120 Honors Lab Section : Autonomous Car

Statement of Purpose

We want to create a self-driving car because we are both interested in the role of machine learning and computer vision in the development of autonomous systems. We hope to use computer vision and machine learning techniques to create a car that can detect objects and drive within a set of lanes. 

Background Research

Computer vision uses machine learning to extract information from a camera or a webcam and use the information, in the case of an autonomous RC car, to move in the correct direction. In the case of lane detection, we have researched a lane detection pipeline, which includes altering the original image to make it easier to detect the set of lanes, getting rid of noise in the image using a filtering algorithm, and applying an edge detection algorithm to find the right set of lanes the car must stay within. Although we both have experience programming in Python, we don't have experience with OpenCV which means that we will have to use tutorials to learn how to implement the aforementioned pipeline. We will also have to learn how to use neural networks to detect objects like stop signs, which we have looked into as well. Using the webcam feed, we will have to manipulate the servo motors of our car to make sure that the car will follow the set of lanes and stop when it should stop. 

Block Diagram / Flow Chart

System Overview

We have two goals for our system, lane detection and object detection. 

In lane detection, the USB camera feeds live images to the TX2. Each image will then be masked to only contain our region of interest, which will likely be a triangle containing the lines and the horizon. As we do not care about the colors of the image in lane detection, we can convert the image into greyscale. This will remove the color information of the image and assign a single intensity value for each pixel. We will then use Canny Edge Detection to detect all the edges of the grayscale image. This will allow us to generate lines from the edges using Hough Transformation. After we have detected the lines in the image, we will create a linear representation for both the left and right lane. We will then generate instructions to stay within the lanes and send those instructions to the Arduino Uno. The Arduino will order the motors of the car to steer the wheels based on each video frame.

In object detection, we will use the same images as in lane detection. We will then used the functions in OpenCV to detect obstacles within the lanes. With this information, we can generate instructions to either stop or drive around the obstacle. This will be sent to the Arduino, which will then be carried out by the motors. Similarly, we will also use OpenCV to detect stop signs and tell the motors to stop accordingly. 

Parts

Provide a list of parts that you may need for your project. You should include details such as the quantity, model number, purpose, vendor, and price (excluding taxes and shipping) for each part. This list may change as you work on your project. 

Possible Challenges

One of the possible challenges we may encounter is the amount of materials we have to research and learn in order to implement computer vision and machine learning in our project as neither of us have had substantial experience in machine computer vision. However, this isn’t a huge problems as we both have previous programming experience and there are a lot of resources online dedicated to this field. Another challenge we can foresee is the limited amount of testing we can have with the TX2. As we cannot take our processor outside of the lab, we will have to come to the lab more often to test near the end of the project.

References

[1]M. Hardwick, "Simple Lane Detection with OpenCV", Medium, 2017. [Online]. Available: https://medium.com/@mrhwick/simple-lane-detection-with-opencv-bfeb6ae54ec0. [Accessed: 30- Jan- 2022]

[2]"A self-driving RC car! And a complete guide to build your own.", DEV, 2020. [Online]. Available: https://dev.to/ivanorsolic/a-self-driving-rc-car-and-a-complete-guide-to-build-your-own-23el. [Accessed: 2- Feb- 2022]

[3]A. Zugaldia, "Build your own (small) Autonomous Robocar", Medium, 2018. [Online]. Available: https://blog.mapbox.com/build-your-own-small-autonomous-robocar-41ae74927f55. [Accessed: 2- Feb- 2022]

[4]D. Tian, "DeepPiCar — Part 1: How to Build a Deep Learning, Self Driving Robotic Car on a Shoestring Budget", Medium, 2019. [Online]. Available: https://towardsdatascience.com/deeppicar-part-1-102e03c83f2c. [Accessed: 5- Feb- 2022]

[5]L. Venturi and K. Korda, Hands-On Vision and Behavior for Self-Driving Cars. Birmingham: Packt Publishing, Limited, 2020.

[6]Susmitha and K. Sharma, "Automated Driving Car Using Image Processing", International Journal of Recent Trends in Engineering and Research, vol. 3, no. 4, pp. 160-164, 2017.

[7]S. Waslander, "Introduction to Self-Driving Cars", Coursera. [Online]. Available: https://www.coursera.org/learn/intro-self-driving-cars. [Accessed: 5- Feb- 2022]

[8]A. Ng, "Machine Learning", Coursera. [Online]. Available: https://www.coursera.org/learn/machine-learning. [Accessed: 3- Feb- 2022]

[9]A. Rosebrock, "Real-time object detection with deep learning and OpenCV - PyImageSearch", PyImageSearch, 2017. [Online]. Available: https://pyimagesearch.com/2017/09/18/real-time-object-detection-with-deep-learning-and-opencv/. [Accessed: 26- Feb- 2022]