ECE 110/120 Honors Lab Section : COVID Checkpoint

Name	NetID	Section
Varun Kowdle	vkowdle2	ECE 110
Adrian Cheng	acheng27	ECE 110
Jiwoong Jung	jiwoong3	ECE 110
Aryan Mathur	aryanm6	ECE 110 and ECE 120

Statement of Purpose

Our goal is to develop a COVID-19 checkpoint that has the ability to scan the QR code on the Safer Illinois app, a person's face to determine if they are wearing a mask, and a person's body temperature to automate the system of COVID checking students entering and exiting buildings across campus. To accomplish the crux of this task we plan to utilize an infrared sensor, a programmable microcontroller, camera (between 5 megapixel ~ 12 megapixel), facial detection software, QR Code reader, and LCD or OLED display.

Background Research

We currently believe that the current method of checking students into buildings such as the ECEB and ARC is really inconvenient. For reference, students have to show faculty and staff their Safer Illinois App as well as swipe their I-Card to make sure that they are covid free. Though this system makes it so that everyone in the building is assured that the students are covid safe, this process is very tedious and can be automated using various software and engineering principles. Currently, there are no similar projects or products that include the features that we plan to include.

Block Diagram / Flow Chart

System Overview

The overall idea of our system is to gather temperature data from an IR camera, then using facial recognition on a normal camera to make sure the user is wearing a mask, and finally, a QR code scan to link it to their NetID's. This would all then be sent and stored within an internal database in order to allow for entry tracking in the event of a false negative. Finally, the validation of entry can be outputted on a small LCD screen.

Parts

Provide a list of parts that you may need for your project. You should include details such as the quantity, model number, purpose, vendor, and price (excluding taxes and shipping) for each part. This list may change as you work on your project.

Infrared sensor
Programmable Microcontroller
Camera and QR Code reader
Facial detection software
LCD or OLED display

	Items		Price
1	Infrared Sensor	https://store.arduino.cc/usa/grove-digital-infrared-temperature-sensor	20.05
2	Programmable Microcontroller	https://store.arduino.cc/usa/arduino-uno-rev3	23
3	Camera and QR Code Reader	https://www.target.com/p/tracfone-prepaid-blu-view-16gb-black/-/A-79395422?ref=tgt_adv_XS000000&AFID=google_pla_df_free_local&CPNG=Electronics&adgroup=80-2	20
4	Facial Detection Software	N/A	N/A
5	LCD and OLED Display	https://store.arduino.cc/usa/grove-oled-display-1-12	17.50

Facial Recognition Software

Facial recognition software is used to detect the user's face contour and scan to check the presence of mask. If the user is wearing mask, the system will return "Thank You". On the contrast, if the software does not detect the mask, it will return "Please Wear Mask".

Before building the actual software, building a beta testing unit is crucial, so that we have the backbone to work around the source code. We will choose a C/C++ coding style since our Arduino mainboard utilizes the C/C++ dialect. Python can also be a good candidate that can be implemented, since python can be utilized to execute machine learning to increase the accuracy of facial scanning and identification. This can be also useful in future implementations, with enhanced security to make sure the card holder matches with the actual identity of the student.

Python websites allows to download 'face_recognition' module, which makes the coding process more feasible as I can start the command with 'import face_recognition' or 'import face_recognition.api as face_recognition' Here are some general source code to follow utilizing the module:

The following code will be amended and updated throughout the time course.

#Facial Detection Software Source Code through Python
#Python is a good selection for Machine Learning
#import face_recognition

#imported_image = face_recognition.load_image_file('/img/groups/team1.jpg')
#face_locate = face_recognition.face_locations(imported_image)

#Analyze coordinate arrays for each face

#print(face_location)

import cv2
# Get user supplied values
imagePath = sys.argv[1]
cascPath = sys.argv[2]
# Create the haar cascade
faceCascade = cv2.CascadeClassifier(cascPath)
# Read the image
image = cv2.imread(imagePath)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Detect faces in the image
faces = faceCascade.detectMultiScale(
gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags = cv2.cv.CV_HAAR_SCALE_IMAGE
)
print "Found {0} faces!".format(len(faces))

# Draw a rectangle around the faces
for (x, y, w, h) in faces:
cv2.rectangle(image, (x, y), (x+w, y+h), (0, 255, 0), 2)

cv2.imshow("Faces found", image)
cv2.waitKey(0)

Facial Detection Software for Arduino using C++

#include "esp_camera.h"
#include "fd_forward.h"

#define PWDN_GPIO_NUM     32
#define RESET_GPIO_NUM    -1
#define XCLK_GPIO_NUM      0
#define SIOD_GPIO_NUM     26
#define SIOC_GPIO_NUM     27
#define Y9_GPIO_NUM       35
#define Y8_GPIO_NUM       34
#define Y7_GPIO_NUM       39
#define Y6_GPIO_NUM       36
#define Y5_GPIO_NUM       21
#define Y4_GPIO_NUM       19
#define Y3_GPIO_NUM       18
#define Y2_GPIO_NUM        5
#define VSYNC_GPIO_NUM    25
#define HREF_GPIO_NUM     23
#define PCLK_GPIO_NUM     22

bool initCamera() {

  camera_config_t config;

  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG;
  config.frame_size = FRAMESIZE_QVGA;
  config.jpeg_quality = 10;
  config.fb_count = 1;

  esp_err_t result = esp_camera_init(&config);

  if (result != ESP_OK) {
    return false;
  }

  return true;
}

mtmn_config_t mtmn_config = {0};
int detections = 0;

void setup() {
  Serial.begin(115200);

  if (!initCamera()) {

    Serial.printf("Failed to initialize camera...");
    return;
  }

  mtmn_config = mtmn_init_config();
}

void loop() {

  camera_fb_t * frame;
  frame = esp_camera_fb_get();

  dl_matrix3du_t *image_matrix = dl_matrix3du_alloc(1, frame->width, frame->height, 3);
  fmt2rgb888(frame->buf, frame->len, frame->format, image_matrix->item);

  esp_camera_fb_return(frame);

  box_array_t *boxes = face_detect(image_matrix, &mtmn_config);

  if (boxes != NULL) {
    detections = detections+1;
    Serial.printf("Faces detected %d times \n", detections);

    dl_lib_free(boxes->score);
    dl_lib_free(boxes->box);
    dl_lib_free(boxes->landmark);
    dl_lib_free(boxes);
  }

  dl_matrix3du_free(image_matrix);

}

Possible Challenges

Some challenges that we might face include:

1) The checkpoint must be able to recognize whether the QR code is a screenshot or actively being shown on screen. The checkpoint should only be able to read the QR code actively being shown on the app.

2) Moving targets will be a challenge as the mask detection and temperature scanners might not be able to pick up the moving human.

3) Temperature sensor might give inaccurate results since the weather effects human's outer temperature.

4) System needs to be simplified in order to effortlessly implement across campus.

References

List all references you used in your proposal. This is important, you do not want to be blamed for plagiarism. IEEE citation format is highly recommended. You can use citethisforme.com's IEEE citation generator to painlessly generate your references in this style.