Project
| # | Title | Team Members | TA | Documents | Sponsor |
|---|---|---|---|---|---|
| 44 | A World-Model based Infant Interaction Robot |
Ruijin Xu Zishuo Feng |
design_document2.pdf final_paper1.pdf presentation1.pptx video |
Gaoang Wang | |
| # Problem Interactive robots designed for infant-oriented companionship, play, and sensorimotor stimulation have potential applications in safe and adaptive human-robot interaction. However, a key challenge is ensuring safe and robust interaction, as infant-like users or proxy moving objects may unpredictably grab, hit, or collide with the robot. Traditional reactive systems are often insufficient because they respond only after contact occurs, potentially too late to prevent harm to the robot or the infant. Current robotic systems lack the ability to anticipate infant-like actions and proactively adjust their behavior in real time, creating a safety gap in close-range human-robot interaction. # Solution Overview This project aims to develop a world-model-based intelligent mobile robot system that can operate safely and proactively in infant-like interaction scenarios. The system perceives nearby motion and approach behavior through onboard sensors such as ToF proximity sensors, bumper switches, IMU, and optionally a camera, utilizes a lightweight world model to predict short-term future states such as imminent contact or grab risk, and adjusts robot motion strategies in real time to enable dynamic and safe interaction. The emphasis is on predictive avoidance rather than reactive response, allowing the robot to anticipate and avoid potentially harmful situations before they occur. # Solution Components 1. Sensing Subsystem * Utilize ToF distance sensors, proximity sensors, bumper switches, and IMU to capture infant motion, approach patterns, and environmental context. * Detect approaching and possible grabbing-like behaviors in real time through multi-sensor fusion. 2. Embedded Processing & World Model * Run on an embedded platform (ESP32 or similar) for sensor data acquisition, timestamping, and real-time processing. * Implement a lightweight world model (small GRU/LSTM or 1D CNN) that learns latent dynamics from time-series sensor data and predicts future states (e.g., sensor values, grab risk) over a 0.5-1 second horizon. 3. Decision Logic & Robot Control * Use world model predictions to make decisions: if predicted future distance falls below a threshold, trigger proactive avoidance maneuvers. * Control robot chassis (2WD with motor driver) to execute behaviors such as retreating or changing direction. * Integrate basic interaction capabilities (speaker/buzzer for audio cues) for infant engagement. 4. Data Logging & Evaluation Pipeline * Implement data logging to synchronize sensor data with ground truth labels for model training and evaluation. * Enable comparison between reactive baseline (threshold-based) and prediction-based avoidance strategies. # Criterion for Success 1. The robot can collect stable real-time sensor data and move reliably on a mobile chassis. 2. A reactive baseline avoidance system is implemented and demonstrated. 3. A lightweight prediction model is trained and deployed for short-term risk prediction. 4. The robot can use model predictions to trigger proactive avoidance in real time. 5. In controlled tests, the prediction-based system demonstrates measurable improvement over the reactive baseline in at least one metric, such as lower contact rate, higher avoidance success rate, faster response, or greater maintained separation distance. |
|||||