Project

# Title Team Members TA Documents Sponsor
33 HelpMeRecall
 Michael Jiang
Sravya Davuluri
William Li
 Hossein Ataee design_document2.pdf
proposal3.pdf
# HelpMeRecall

Team Members:
- Sravya Davuluri (sravyad2)
- William Li (wli202)
- Michael Jiang (mbjiang2)

# Problem

Many individuals have difficulty remembering recent activities and completing routine tasks like eating or taking medication.

# Solution

A standalone assistive device that supports activity recall using sensor-gated voice interaction. It allows users to verbally log activities they have completed, and later query if a specific activity has been performed. It uses an onboard microphone and on-device audio processing on a microcontroller to perform keyword detection.

This device is always on and will be verifiable with an LED, but the voice input is only accepted if the device is worn (capacitive touch sensor) and specific words from a limited vocabulary is said to avoid accidental logging. To address the possibility of reduced correct detection of supported keywords, we will have various keywords targeted for an activity. So in the case of taking medicine, it might be medicine, medication, pill, drug, and prescription. This also simplifies the problem and prevents confidence rate issues. To validate a completed action, the action is logged only if an accelerometer detects physical movement around the time in order to reduce false logging. If a voice log is accepted, haptic feedback is provided by the device. Activities are also timestamped and stored in local memory. If the device notes that a specific activity has been completed, it affirms it including the timestamp using an integrated speaker.

The logs reset at midnight automatically since the activities repeat on the daily. There is also an option of a hard reset button to clear logs. There will also be a button to delete the latest log in case of a logging mistake by the user.

# Solution Components

## Subsystem 1: Microcontroller Unit and Controls

Acts as the central unit for logic. Manages the sensor inputs, and executes a finite state machine. The FSM states are start, idle, listening, logging, and replying.

Components: ESP32-S3-WROOM-1

## Subsystem 2: Audio input processing unit

Captures the voice input from the user and performs keyword detection on a limited vocabulary, where each action can be mapped to multiple set keywords to improve detection.

Components: Digital MEMS microphone (INMP441), ESP32-S3-WROOM-1

## Subsystem 3: Sensor gating and activity validation

Uses a capacitive touch sensor and an accelerometer to detect motion, which ensures that voice input is only received and accepted if the device is worn and recent movement is detected by the accelerometer instead of continuous voice recognition. A "cooldown" period is enforced where the microphone will be disabled for 10 seconds if there's motion but no logging during the listening period multiple times in a row to help conserve some battery.

Components: Capacitive touch sensor (AT42QT1010), Accelerometer (MPU-6050)

## Subsystem 4: Feedback and Output

Uses a speaker for audio feedback as a response to the user’s query. This subsystem also provides haptic feedback as an indication of an accepted user voice log. To indicate if the device is on, the LED is green. If the device is listening, the LED is yellow. If the device is low on power, the LED will be red.

Components: Speaker (8 ohm speaker), amplifier (MAX98357A), coin vibration motor, transistor (2N3904), RGB LED

## Subsystem 5: Time logging and local storage

Stores the activity voice logs along with timestamps. Allows automatic reset at midnight to support daily repetitive tasks. Timekeeping is done using ESP32’s internal RTC.

Components: ESP32-S3-WROOM-1

## Subsystem 6: Power

Supplies power to the device.

Components: Battery (Li-Po battery)

# Criterion For Success
- Correctly detects supported keywords with an accuracy of at least 80% in a quiet environment
- Device will only log upon verifying physical activity and hearing a keyword from the user within a 5 second window
- Upon successful logging, the speaker will output audibly and haptic feedback can be felt by the user with a 2 second vibration
- While querying logs, speaker will output and LED will be solid
- Logs will be automatically cleared at midnight and can be manually reset with the reset button
- Latest log will be deleted upon pushing a separate button
- LED stays solid while device is powered
- False log rate < 1 per hour in normal conversation when worn.

Recovery-Monitoring Knee Brace

Dong Hyun Lee, Jong Yoon Lee, Dennis Ryu

Featured Project

Problem:

Thanks to modern technology, it is easy to encounter a wide variety of wearable fitness devices such as Fitbit and Apple Watch in the market. Such devices are designed for average consumers who wish to track their lifestyle by counting steps or measuring heartbeats. However, it is rare to find a product for the actual patients who require both the real-time monitoring of a wearable device and the hard protection of a brace.

Personally, one of our teammates ruptured his front knee ACL and received reconstruction surgery a few years ago. After ACL surgery, it is common to wear a knee brace for about two to three months for protection from outside impacts, fast recovery, and restriction of movement. For a patient who is situated in rehabilitation after surgery, knee protection is an imperative recovery stage, but is often overlooked. One cannot deny that such a brace is also cumbersome to put on in the first place.

--------

Solution:

Our group aims to make a wearable device for people who require a knee brace by adding a health monitoring system onto an existing knee brace. The fundamental purpose is to protect the knee, but by adding a monitoring system we want to provide data and a platform for both doctor and patients so they can easily check the current status/progress of the injury.

---------

Audience:

1) Average person with leg problems

2) Athletes with leg injuries

3) Elderly people with discomforts

-----------

Equipment:

Temperature sensors : perhaps in the form of electrodes, they will be used to measure the temperature of the swelling of the knee, which will indicate if recovery is going smoothly.

Pressure sensors : they will be calibrated such that a certain threshold of force must be applied by the brace to the leg. A snug fit is required for the brace to fulfill its job.

EMG circuit : we plan on constructing an EMG circuit based on op-amps, resistors, and capacitors. This will be the circuit that is intended for doctors, as it will detect muscle movement.

Development board: our main board will transmit the data from each of the sensors to a mobile interface via. Bluetooth. The user will be notified when the pressure sensors are not tight enough. For our purposes, the battery on the development will suffice, and we will not need additional dry cells.

The data will be transmitted to a mobile system, where it would also remind the user to wear the brace if taken off. To make sure the brace has a secure enough fit, pressure sensors will be calibrated to determine accordingly. We want to emphasize the hardware circuits that will be supplemented onto the leg brace.

We want to emphasize on the hardware circuit portion this brace contains. We have tested the temperature and pressure resistors on a breadboard by soldering them to resistors, and confirmed they work as intended by checking with a multimeter.

Project Videos