Project

# Title Team Members TA Documents Sponsor
33 HelpMeRecall
 Michael Jiang
Sravya Davuluri
William Li
 Hossein Ataee design_document2.pdf
final_paper2.pdf
proposal3.pdf
video
# HelpMeRecall

Team Members:
- Sravya Davuluri (sravyad2)
- William Li (wli202)
- Michael Jiang (mbjiang2)

# Problem

Many individuals have difficulty remembering recent activities and completing routine tasks like eating or taking medication.

# Solution

A standalone assistive device that supports activity recall using sensor-gated voice interaction. It allows users to verbally log activities they have completed, and later query if a specific activity has been performed. It uses an onboard microphone and on-device audio processing on a microcontroller to perform keyword detection.

This device is always on and will be verifiable with an LED, but the voice input is only accepted if the device is worn (capacitive touch sensor) and specific words from a limited vocabulary is said to avoid accidental logging. To address the possibility of reduced correct detection of supported keywords, we will have various keywords targeted for an activity. So in the case of taking medicine, it might be medicine, medication, pill, drug, and prescription. This also simplifies the problem and prevents confidence rate issues. To validate a completed action, the action is logged only if an accelerometer detects physical movement around the time in order to reduce false logging. If a voice log is accepted, haptic feedback is provided by the device. Activities are also timestamped and stored in local memory. If the device notes that a specific activity has been completed, it affirms it including the timestamp using an integrated speaker.

The logs reset at midnight automatically since the activities repeat on the daily. There is also an option of a hard reset button to clear logs. There will also be a button to delete the latest log in case of a logging mistake by the user.

# Solution Components

## Subsystem 1: Microcontroller Unit and Controls

Acts as the central unit for logic. Manages the sensor inputs, and executes a finite state machine. The FSM states are start, idle, listening, logging, and replying.

Components: ESP32-S3-WROOM-1

## Subsystem 2: Audio input processing unit

Captures the voice input from the user and performs keyword detection on a limited vocabulary, where each action can be mapped to multiple set keywords to improve detection.

Components: Digital MEMS microphone (INMP441), ESP32-S3-WROOM-1

## Subsystem 3: Sensor gating and activity validation

Uses a capacitive touch sensor and an accelerometer to detect motion, which ensures that voice input is only received and accepted if the device is worn and recent movement is detected by the accelerometer instead of continuous voice recognition. A "cooldown" period is enforced where the microphone will be disabled for 10 seconds if there's motion but no logging during the listening period multiple times in a row to help conserve some battery.

Components: Capacitive touch sensor (AT42QT1010), Accelerometer (MPU-6050)

## Subsystem 4: Feedback and Output

Uses a speaker for audio feedback as a response to the user’s query. This subsystem also provides haptic feedback as an indication of an accepted user voice log. To indicate if the device is on, the LED is green. If the device is listening, the LED is yellow. If the device is low on power, the LED will be red.

Components: Speaker (8 ohm speaker), amplifier (MAX98357A), coin vibration motor, transistor (2N3904), RGB LED

## Subsystem 5: Time logging and local storage

Stores the activity voice logs along with timestamps. Allows automatic reset at midnight to support daily repetitive tasks. Timekeeping is done using ESP32’s internal RTC.

Components: ESP32-S3-WROOM-1

## Subsystem 6: Power

Supplies power to the device.

Components: Battery (Li-Po battery)

# Criterion For Success
- Correctly detects supported keywords with an accuracy of at least 80% in a quiet environment
- Device will only log upon verifying physical activity and hearing a keyword from the user within a 5 second window
- Upon successful logging, the speaker will output audibly and haptic feedback can be felt by the user with a 2 second vibration
- While querying logs, speaker will output and LED will be solid
- Logs will be automatically cleared at midnight and can be manually reset with the reset button
- Latest log will be deleted upon pushing a separate button
- LED stays solid while device is powered
- False log rate < 1 per hour in normal conversation when worn.

Interactive Proximity Donor Wall Illumination

Sungmin Jang, Anita Jung, Zheng Liu

Interactive Proximity Donor Wall Illumination

Featured Project

Team Members:

Anita Jung (anitaj2)

Sungmin Jang (sjang27)

Zheng Liu (zliu93)

Link to the idea: https://courses.engr.illinois.edu/ece445/pace/view-topic.asp?id=27710

Problem:

The Donor Wall on the southwest side of first floor in ECEB is to celebrate and appreciate everyone who helped and donated for ECEB.

However, because of poor lighting and color contrast between the copper and the wall behind, donor names are not noticed as much as they should, especially after sunset.

Solution Overview:

Here is the image of the Donor Wall:

http://buildingcampaign.ece.illinois.edu/files/2014/10/touched-up-Donor-wall-by-kurt-bielema.jpg

We are going to design and implement a dynamic and interactive illuminating system for the Donor Wall by installing LEDs on the background. LEDs can be placed behind the names to softly illuminate each name. LEDs can also fill in the transparent gaps in the “circuit board” to allow for interaction and dynamic animation.

And our project’s system would contain 2 basic modes:

Default mode: When there is nobody near the Donor Wall, the names are softly illuminated from the back of each name block.

Moving mode: When sensors detect any stimulation such as a person walking nearby, the LEDs are controlled to animate “current” or “pulses” flowing through the “circuit board” into name boards.

Depending on the progress of our project, we have some additional modes:

Pressing mode: When someone is physically pressing on a name block, detected by pressure sensors, the LEDs are controlled to

animate scattering of outgoing light, just as if a wave or light is emitted from that name block.

Solution Components:

Sensor Subsystem:

IR sensors (PIR modules or IR LEDs with phototransistor) or ultrasonic sensors to detect presence and proximity of people in front of the Donor Wall.

Pressure sensors to detect if someone is pressing on a block.

Lighting Subsystem:

A lot of LEDs is needed to be installed on the PCBs to be our lighting subsystem. These are hidden as much as possible so that people focus on the names instead of the LEDs.

Controlling Subsystem:

The main part of the system is the controlling unit. We plan to use a microprocessor to process the signal from those sensors and send signal to LEDs. And because the system has different modes, switching between them correctly is also important for the project.

Power Subsystem:

AC (Wall outlet; 120V, 60Hz) to DC (acceptable DC voltage and current applicable for our circuit design) power adapter or possible AC-DC converter circuit

Criterion for success:

Whole system should work correctly in each mode and switch between different modes correctly. The names should be highlighted in a comfortable and aesthetically pleasing way. Our project is acceptable for senior design because it contains both hardware and software parts dealing with signal processing, power, control, and circuit design with sensors.

Project Videos