Project

# Title Team Members TA Documents Sponsor
45 AI-based Meeting Transcription Device
Chang Liu
Gao Gao
Ziyang Huang
Jiankun Yang design_document1.docx
final_paper1.pdf
other1.pdf
other2.txt
photo2.jpg
photo3.jpg
photo1.jpg
presentation1.pptx
proposal1.pdf
## Team Members:
- **Ziyang Huang** (ziyangh3)
- **Gao Gao** (xgao54)
- **Chang Liu** (changl21)

## Problem

During the pandemic, we found Zoom’s live transcription very useful, as it helped the audience catch up quickly with the lecturer. In many professional and academic settings, real-time transcription of spoken communication is essential for note-taking. Additionally, individuals with hearing impairments face challenges in following spoken conversations, especially in environments where captions are unavailable.

Existing solutions, such as Zoom’s live transcription or mobile speech-to-text apps, require an internet connection and are often tied to specific platforms. To address this, we propose a standalone, portable transcription device that can capture, transcribe, and display spoken text in real time. The device will be helpful since it provides a distraction-free way to record and review conversations without relying on a smartphone or laptop.

## Solution

Our **Smart Meeting Transcription Device** will be a portable, battery-powered device that records with a microphone, converts speech into real-time text, and displays it on an LCD screen. The system consists of the following key components:

1. **A microphone module** to capture audio input.
2. **A speech processing unit** (Jetson Nano/Raspberry Pi/Arduino) running the Vosk speech-to-text model to transcribe the captured speech.
3. **An STM32 microcontroller**, which serves as the central controller for managing user interactions, processing text display, and storing transcriptions.
4. **An LCD screen** to display transcriptions in real-time.
5. **External memory** (SD card or NOR flash) for saving transcribed conversations.
6. **A power system** (battery with efficient power management) to enable portability.

---

## Solution Components

### **Subsystem 1: Speech Processing Unit**
- **Function:** Captures audio and converts speech into text using an embedded speech-to-text model.
- **Microphone Module:** Adafruit Electret Microphone Amplifier (MAX9814)
- **Processing Board:** Jetson Nano / Raspberry Pi 4B
- **Speech Recognition Model:** Vosk Speech-to-Text Model
- **Memory Expansion (if required):** SD card (SanDisk Ultra 32GB)

### **Subsystem 2: STM32 Central Controller**
- **Function:** Manages the user interface, processes the transcribed text, and sends data to the LCD screen.
- **Microcontroller:** STM32F4 Series MCU
- **Interface Components:** Buttons for navigation and text saving
- **Memory Module:** SPI-based NOR Flash (W25Q128JV)

### **Subsystem 3: Display Module**
- **Function:** Displays real-time transcriptions and allows users to scroll through previous text.
- **LCD Screen:** 2.8-inch TFT Display (ILI9341)
- **Controller Interface:** SPI Communication with STM32

### **Subsystem 4: Power Management System**
- **Function:** Provides reliable and portable power for all components.
- **Battery:** 3.7V Li-ion Battery (Adafruit 2500mAh)
- **Power Regulation:** TP4056 Li-ion Charger + 5V Boost Converter
- **Power Optimization:** Sleep mode for STM32 to enhance battery life

---

## **Criterion for Success**
1. The device must accurately transcribe speech to text with reasonable latency.
2. The LCD screen must display real-time transcriptions clearly.
3. The STM32 must successfully manage system operations and communicate with peripheral components.
4. The system should support local storage for saving transcriptions.
5. The battery life should last at least **2-3 hours** under normal usage conditions.

Microcontroller-based Occupancy Monitoring (MOM)

Vish Gopal Sekar, John Li, Franklin Moy

Microcontroller-based Occupancy Monitoring (MOM)

Featured Project

# Microcontroller-based Occupancy Monitoring (MOM)

Team Members:

- Franklin Moy (fmoy3)

- Vish Gopal Sekar (vg12)

- John Li (johnwl2)

# Problem

With the campus returning to normalcy from the pandemic, most, if not all, students have returned to campus for the school year. This means that more and more students will be going to the libraries to study, which in turn means that the limited space at the libraries will be filled up with the many students who are now back on campus. Even in the semesters during the pandemic, many students have entered libraries such as Grainger to find study space, only to leave 5 minutes later because all of the seats are taken. This is definitely a loss not only to someone's study time, but maybe also their motivation to study at that point in time.

# Solution

We plan on utilizing a fleet of microcontrollers that will scan for nearby Wi-Fi and Bluetooth network signals in different areas of a building. Since students nowadays will be using phones and/or laptops that emit Wi-Fi and Bluetooth signals, scanning for Wi-Fi and Bluetooth signals is a good way to estimate the fullness of a building. Our microcontrollers, which will be deployed in numerous dedicated areas of a building (called sectors), will be able to detect these connections. The microcontrollers will then conduct some light processing to compile the fullness data for its sector. We will then feed this data into an IoT core in the cloud which will process and interpret the data and send it to a web app that will display this information in a user-friendly format.

# Solution Components

## Microcontrollers with Radio Antenna Suite

Each microcontroller will scan for Wi-Fi and Bluetooth packets in its vicinity, then it will compile this data for a set timeframe and send its findings to the IoT Core in the Cloud subsystem. Each microcontroller will be programmed with custom software that will interface with its different radio antennas, compile the data of detected signals, and send this data to the IoT Core in the Cloud subsystem.

The microcontroller that would suit the job would be the ESP32. It can be programmed to run a suite of real-time operating systems, which are perfect for IoT applications such as this one. This enables straightforward software development and easy connectivity with our IoT Core in the Cloud. The ESP32 also comes equipped with a 2.4 GHz Wi-Fi transceiver, which will be used to connect to the IoT Core, and a Bluetooth Low Energy transceiver, which will be part of the radio antenna suite.

Most UIUC Wi-Fi access points are dual-band, meaning that they communicate using both the 2.4 GHz and 5 GHz frequencies. Because of this, we will need to connect a separate dual-band antenna to the ESP32. The simplest solution is to get a USB dual-band Wi-Fi transceiver, such as the TP-Link Nano AC600, and plug it into a USB Type-A breakout board that we will connect to each ESP32's GPIO pins. Our custom software will interface with the USB Wi-Fi transceiver to scan for Wi-Fi activity, while it will use the ESP32's own Bluetooth Low Energy transceiver to scan for Bluetooth activity.

## Battery Backup

It is possible that the power supply to a microcontroller could fail, either due to a faulty power supply or by human interference, such as pulling the plug. To mitigate the effects that this would have on the system, we plan on including a battery backup subsystem to each microcontroller. The battery backup subsystem will be able to not only power the microcontroller when it is unplugged, but it will also be able to charge the battery when it is plugged in.

Most ESP32 development boards, like the Adafruit HUZZAH32, have this subsystem built in. Should we decide to build this subsystem ourselves, we would use the following parts. Most, if not all, ESP32 microcontrollers use 3.3 volts as its operating voltage, so utilizing a 3.7 volt battery (in either an 18650 or LiPo form factor) with a voltage regulator would supply the necessary voltage for the microcontroller to operate. A battery charging circuit consisting of a charge management controller would also be needed to maintain battery safety and health.

## IoT Core in the Cloud

The IoT Core in the Cloud will handle the main processing of the data sent by the microcontrollers. Each microcontroller is connected to the IoT Core, which will likely be hosted on AWS, through the ESP32's included 2.4GHz Wi-Fi transceiver. We will also host on AWS the web app that interfaces with the IoT Core to display the fullness of the different sectors. This web app will initially be very simple and display only the estimated fullness. The web app will likely be built using a Python web framework such as Flask or Django.

# Criterion For Success

- Identify Wi-Fi and Bluetooth packets from a device and distinguish them from packets sent by different devices.

- Be able to estimate the occupancy of a sector within a reasonable margin of error (15%), as well as being able to compute its fullness relative to that sector's size.

- Display sector capacity information on the web app that is accurate within 5 minutes of a user accessing the page.

- Battery backup system keeps the microcontroller powered for at least 3 hours when the wall outlet is unplugged.

Project Videos