Project
| # | Title | Team Members | TA | Documents | Sponsor |
|---|---|---|---|---|---|
| 57 | Digital Audio Player with Onboard AI Equalization |
Advaith Anand Andrew Cheng Siddharth Salapaka |
Denghan Xiong | design_document1.pdf proposal2.pdf |
|
| # Digital Audio Player with Onboard AI Equalization **Team Members:** - Siddharth Salapaka (svs9) - Andrew Cheng (ac158) - Advaith Anand (aanand10) ## Problem Portable digital audio players let users carry a personal music collection and listen without a phone. However, adjusting their sound usually means choosing a fixed equalizer preset or changing individual frequency bands by hand. A fixed preset applies the same correction to every song. A "bass boost" that helps a thin acoustic recording can make an already bass-heavy track muddy, and the balance can change between sections of the same song. Listeners often describe what they want in simple terms, such as "more bass," "clearer vocals," or "softer treble," without knowing which frequencies to adjust. We want to build a player that turns these listening goals into continuous EQ adjustments that depend on the music being played. The player should keep convenient physical controls and high-quality wired playback. We also need to show with an objective measurement that this adaptive EQ does better than a fixed preset. ## Solution We propose a battery-powered digital audio player with microSD storage, a rotary wheel, physical buttons, an OLED display, a dedicated stereo DAC, a headphone amplifier, and a custom control PCB. Users load songs onto the microSD card, select a track, and choose a listening goal on the device. Each listening goal is defined as a **target tonal balance**: a specific shape for the relative energy across five frequency bands. The target is derived from a reference curve measured on a large, independent music corpus rather than from our own listening. During playback, a compact neural network analyzes the last two seconds of the original, unprocessed audio. It then predicts gains for a five-band equalizer that move the current passage toward the selected goal's target. Because the correction depends on the passage, a thin recording receives more bass boost than one that already has plenty of bass. Only the adaptive controller can make this distinction. A fixed preset cannot. Digital signal processing applies the predicted gains continuously. Gain changes are ramped to avoid clicks, headroom is reserved for boosts, and a final peak limiter protects the output. The same gains are applied to both channels to preserve stereo balance. All analysis, inference, and filtering run on the player without an internet connection. Model training happens offline during development. The required modes are "more bass," "clearer vocals," and "softer treble," plus an EQ bypass mode. The central engineering claim of the project is measurable: **for each goal, the onboard controller brings held-out music closer to the goal's target tonal balance than the best possible fixed preset, by a committed margin** (see Criterion for Success). The training targets, dataset splits, metric, and pass thresholds will be committed to our repository before training begins, and the commit hash will be shared with our TA. ### System Overview Power enters through a 5 V USB-C port and goes to a BQ24074 charger, which charges a protected single-cell lithium-ion battery and supplies the system at roughly 3.0 to 4.4 V. A TPS63070 buck-boost regulator converts this to 3.6 V. Two low-dropout regulators then produce the final rails: a TLV75733P supplies the digital 3.3 V rail for the microcontroller, microSD card, and OLED, and a TPS7A2033 supplies a separate low-noise analog 3.3 V rail for the DAC and headphone amplifier. Audio flows from the microSD card to the STM32H743ZIT6 microcontroller over a 4-bit SDMMC interface. The microcontroller sends processed audio over I2S (using the SAI peripheral with DMA) to a PCM5102A DAC. The DAC output passes through an input attenuator to a TPA6132A2 headphone amplifier, which drives the 3.5 mm jack. Inside the firmware, the WAV reader fills a ring buffer. Audio then passes through feature extraction, the int8 neural network, gain smoothing, the five-band biquad equalizer, a headroom pre-gain, 32-bit digital volume control, and a peak limiter before it is sent to the DAC. The rotary encoder and four buttons connect to the microcontroller's GPIO and external interrupt pins. The microcontroller drives an SSD1306 128 × 64 OLED over SPI and reads the battery voltage through a resistor divider on an ADC input. The board also includes an SWD header, a UART logging header, and test points. ## Solution Components ### Main Control PCB Subsystem The custom PCB connects the processor, storage, audio output, controls, and power circuitry. We plan to use an STM32H743ZIT6 (Arm Cortex-M7, up to 480 MHz, 2 MB flash, 1 MB RAM). Audio leaves the processor through the SAI peripheral in I2S mode using DMA, so model inference, display updates, and SD reads cannot stall the output stream. The audio pipeline is **sample-indexed and deterministic**. Features are computed every 4,800 samples (100 ms). The resulting gain update is applied at a fixed sample boundary one hop later. An inference that is not finished by that boundary is logged as a deadline miss. Because of this, the firmware produces bit-identical output whether it plays in real time or runs in the offline evaluation mode described below. We will use a four-layer board with a continuous ground plane. The analog output section will be placed away from the switching regulator, SD lines, and OLED lines. Final assembly will use ICs and supporting circuitry on our PCB, with a connected display and controls. Development boards will be used only for early bring-up. Components: - STM32H743ZIT6 microcontroller IC - 25 MHz HSE crystal, 32.768 kHz LSE crystal, decoupling capacitors, and reset circuitry - SWD programming/debugging header, UART logging header, and test points - Storage, display, control, and audio connectors Requirements: - Maintain at least 40 ms of buffered output audio. Worst-case audio processing time per 10 ms block must stay below 5 ms (under 50% of the real-time budget). - Log buffer underruns, inference deadline misses, per-block CRC-32 of processed audio, and predicted gains over UART. - Measured output sample rate within ±100 ppm of 48 kHz. ### AI and Equalization Subsystem #### Equalizer bands The EQ uses five fixed biquad filters (RBJ cookbook forms). Only the gains change. Fixed frequencies and bandwidths keep the learned control space small and make the combined response easy to predict for headroom management. The five bands are: - **Band 1, Low:** low shelf at 100 Hz, Q of 0.7, measured over 20 to 150 Hz. - **Band 2, Low-mid:** peaking filter at 300 Hz, Q of 1.0, measured over 150 to 500 Hz. - **Band 3, Mid:** peaking filter at 1 kHz, Q of 1.0, measured over 500 Hz to 2 kHz. - **Band 4, Presence:** peaking filter at 3.2 kHz, Q of 1.0, measured over 2 to 5 kHz. - **Band 5, Treble:** high shelf at 8 kHz, Q of 0.7, measured over 5 to 16 kHz. Each band is limited to ±6 dB. The treble measurement region stops at 16 kHz to avoid encoder low-pass artifacts in the training material. #### Goal definitions and target curves **Reference curve.** For every non-silent 3-second segment of the training split, we compute the energy in each of the five measurement regions in dB and subtract the mean across the five bands. This leaves only the tonal shape, independent of loudness. The reference curve R is the median of these vectors over the training split. It describes a typical tonal balance for commercially distributed music, and no team member's hearing is involved in computing it. **Goal targets.** Each goal's target T is the reference curve plus a fixed set of offsets, re-normalized to zero mean. The offsets are frozen before training and are not changed after we see any results. All offsets are in dB relative to R, and any band not mentioned has an offset of 0 dB. - **More bass:** +4.0 dB in the low band and +1.5 dB in the low-mid band. The low-band gain is constrained to be zero or positive. - **Clearer vocals:** −2.0 dB in the low-mid band, +1.0 dB in the mid band, and +3.0 dB in the presence band. The presence gain is constrained to be zero or positive, and the low-mid gain is constrained to be zero or negative. - **Softer treble:** −1.5 dB in the presence band and −4.0 dB in the treble band. The treble gain is constrained to be zero or negative. The sign constraints keep each goal faithful to what the user asked for. For example, "more bass" never cuts bass, even on a track that is already bass-heavy; the controller simply applies little or no boost there. "Clearer vocals" changes the tonal balance of the whole stereo mix. It does not isolate or separate the singer. #### Training data and dataset separation - **Source material:** Full-length, Creative Commons–licensed tracks from the Free Music Archive (FMA) dataset, a public research corpus of over 100,000 tracks with artist and genre metadata. We will sample about 2,000 tracks stratified across top-level genres. We will exclude tracks shorter than 60 s or longer than 10 min and files encoded below 256 kbps. Tracks are decoded and resampled to 48 kHz, 16-bit stereo WAV with a high-quality resampler (SoX). Audio is not redistributed. - **Split by artist:** Tracks are divided 70/15/15 into training, validation, and test splits (about 1,400, 300, and 300 tracks), with **no artist appearing in more than one split**. This is stricter than separating by song, because tracks from the same artist often share mixing and mastering choices. - **Freezing the test set:** The split lists and their SHA-256 hashes are committed before training. The validation split is used for all development decisions. The test split is used only for the final evaluation. - **Augmentation (training split only):** Random low-shelf, high-shelf, and spectral-tilt changes of up to ±6 dB are applied to training excerpts before labels are generated. This teaches the model to correct a wide range of starting balances. #### How training targets are generated No person sets gains for individual songs. The reference gains are produced by an offline optimization script (the "oracle"), and the team's only choices are the frozen goal offsets above. For each training track, each goal, and each 100 ms hop, the oracle: 1. Takes a **centered** 3-second window of the input (1.5 s before and 1.5 s after the current hop) and computes its magnitude spectrum. 2. Applies the exact magnitude response of the device's five-biquad cascade for a candidate set of gains to that spectrum, and measures the resulting mean-normalized band levels. 3. Uses bound-constrained optimization (SciPy L-BFGS-B) to find the gains that minimize the sum of three terms: the squared difference between the resulting band levels and the target, a smoothness penalty on the change from the previous hop's gains, and a small penalty on the size of the gains. The gains are limited to ±6 dB and must satisfy the goal's sign constraints. The weights of the smoothness and size penalties are tuned on the validation split. 4. Stores the resulting gains as the training label for that hop. The oracle is **non-causal**: it uses 1.5 s of future audio and an iterative solver, so it cannot run on the player. The neural network has to predict the oracle's gains from **past audio only**, within a microcontroller's compute budget. This is the learning problem. The model has to recognize how the current passage's spectrum relates to the target and anticipate where the music is heading. #### Model and on-device deployment - **Features:** Every 100 ms, a 4,096-sample Hann-windowed FFT (CMSIS-DSP) of the mono sum of the *unprocessed* input produces 24 log-spaced band energies. The model input is the last 20 frames (2 s of context), normalized with training-set statistics, plus a one-hot goal vector. - **Network:** A small 1-D temporal convolutional network (depthwise-separable layers, about 25,000 parameters). A tanh output is scaled to ±6 dB, and sign constraints are enforced by clamping. - **Training loss:** Mean squared error to the oracle gains, plus an auxiliary tonal-balance loss computed through a differentiable model of the EQ filters. - **Quantization:** Quantization-aware training to int8. The model is deployed with STM32Cube.AI / ST Edge AI Core, with TensorFlow Lite for Microcontrollers as a fallback. Weights are stored in internal flash, with a target of under 150 KB of flash and under 64 KB of activation RAM. - **Strength control:** The user's strength setting (0 to 100%) scales the predicted gains linearly after inference. All evaluations use 100%. #### Baselines - **Best fixed preset (graded baseline):** For each goal, the single constant five-band gain setting that minimizes mean tonal-balance error over the *entire training split*, under the same ±6 dB limits and sign constraints. No constant preset can score better on the training data, so a network that learned to output constant gains could at most tie this baseline. - **Oracle (upper bound, reported):** The non-causal optimizer's gains, which show how much improvement is possible. - **Causal rule-based controller (reported):** Gains computed directly from the past 2 s of band levels with the same smoothing. This shows how much the network adds beyond a hand-written rule. We will report this comparison whatever the outcome. #### Evaluation metric For each non-silent 3-second segment of processed output, we measure the output's energy in each of the five band regions in dB and subtract the mean across the five bands. The segment's tonal-balance error is the root-mean-square difference, in dB, between these mean-normalized levels and the goal's zero-mean target curve. Segments whose input is below −60 dBFS are excluded. A track's error is the average of its segment errors. Because the metric removes overall level, loudness differences between conditions do not affect the score. #### Evaluation procedure - **Device evaluation (pass/fail):** 100 test-split tracks, stratified by genre and fixed at the design freeze, are processed **on the player** in offline evaluation mode. This mode runs the identical sample-indexed firmware path, reads WAV files from microSD, and writes processed WAV files back to microSD. Each track is processed for each goal with the neural controller and with the best fixed preset, using the same filters, headroom logic, and limiter. - **Host simulation (reported):** The same C DSP code and quantized model, compiled for a PC, evaluate the full 300-track test split. - **Live equivalence:** For 10 tracks, per-block CRC-32 values logged over UART during real-time playback must match offline mode exactly. - **Analog confirmation:** For the same 10 tracks, the headphone output into a 32 Ω load is recorded with a USB audio interface at 48 kHz and 24 bits. After correcting for the analog chain response measured in bypass, the error reduction must agree with the digital result within 5 percentage points. #### Runtime protection - Gain changes are smoothed with a one-pole filter with a time constant of about 300 ms. Filter coefficients are updated every 1 ms (48 samples), with no gain step larger than 0.05 dB per update. - A headroom pre-gain equal to the peak of the combined filter response (evaluated at 64 log-spaced frequencies) plus 0.5 dB keeps boosts from raising overall level. A look-ahead peak limiter enforces a −1 dBFS ceiling. - Silence detection (input below −60 dBFS) holds the current gains. - Non-finite, out-of-range, or stuck model outputs trigger a 500 ms ramp to bypass. A missed inference deadline keeps the previous gains, and filtering continues uninterrupted. Components: - Compact int8 temporal convolutional network, with weights in onboard flash - STM32Cube.AI / ST Edge AI Core deployment tools and CMSIS-DSP routines - Spectral feature extraction and five stereo-linked biquad EQ bands - Gain smoothing, headroom management, and look-ahead peak limiter - Offline tools: dataset preparation, oracle label generator, training, host simulator, and evaluation scripts ### Audio Output Subsystem The processor sends filtered stereo audio to a PCM5102A DAC over I2S in 32-bit frames. The PCM5102A generates its clocks from the bit clock, so no separate master clock line is required. Because the PCM5102A has no internal volume control, volume is applied digitally in the 32-bit domain before transmission. This preserves the 16-bit source resolution across the usable volume range. A TPA6132A2 ground-referenced headphone amplifier drives the 3.5 mm jack without output coupling capacitors. A resistive attenuator between the DAC's 2.1 Vrms full-scale output and the amplifier input, together with the amplifier gain setting, will be chosen so that a full-scale DAC signal cannot drive the amplifier into clipping. The design targets conventional 32 Ω headphones. To prevent pops, the DAC soft-mute (XSMT) pin and the amplifier enable pin are sequenced by firmware at power-up, power-down, and track changes. Both analog ICs run from a dedicated low-noise 3.3 V LDO rail. Analog traces are kept short, and digital return currents are routed away from the analog section. Components: - Texas Instruments PCM5102A stereo DAC - Texas Instruments TPA6132A2 headphone amplifier - 3.5 mm stereo headphone jack - Input attenuation, coupling, and output-filter components per the reference circuits - Local supply filtering and audio test points Requirements (measured at the headphone jack into 32 Ω, EQ bypassed): - Frequency response flat within ±1 dB from 20 Hz to 20 kHz - THD+N below 0.1% at 1 kHz, 0.2 Vrms output - Dynamic range of at least 90 dB (A-weighted), relative to maximum unclipped output - Maximum unclipped output of at least 0.5 Vrms - No audible pops at power-up, power-down, or track change, with transients below 10 mV peak at the jack Measurements will use a lab audio analyzer or a USB audio interface with Room EQ Wizard (REW), with the interface's own noise floor measured and reported. ### Music Storage and Playback Subsystem A removable microSD card stores the music library. Users copy songs onto the card with a computer and insert it into the player. The required format is stereo, 16-bit, 48 kHz PCM WAV, which keeps decoding overhead low. The card is accessed through the SDMMC peripheral in 4-bit mode with the FatFs file system (FAT32 and exFAT). The processor reads ahead into a buffer and supports track selection, play/pause, and previous/next. Files in unsupported formats are skipped, and a message is shown on the display. The same subsystem supports the offline evaluation mode, which writes processed WAV files back to the card. Components: - microSD card socket and removable card - SD interface pull-up resistors and local decoupling - FatFs file-system software, WAV header parser, and buffered playback Requirements: - Read-ahead buffer of at least 500 ms of audio - Library of at least 500 tracks, with file names up to 64 characters - Zero underruns caused by storage during the 60-minute playback test ### User Interface Subsystem A rotary wheel and physical buttons control the player without a phone. Depending on the selected menu, the wheel scrolls through tracks and listening goals, adjusts volume, or sets EQ strength. The OLED shows the track name, active goal, strength, battery status, and the current gain of each of the five EQ bands, so the adaptation is visible during use and during the demo. A bypass button switches between adaptive EQ and unprocessed playback with a 50 ms crossfade. Bypass output is level-matched to the EQ path so the comparison is not biased by loudness. Components: - Incremental rotary encoder with push switch and wheel - Play/pause, previous/next, and EQ bypass buttons - 128 × 64 OLED display with SSD1306 controller (SPI) - Button pull-ups, RC debounce where needed, and enclosure hardware Requirements: - Every control input is reflected on the display within 100 ms. - No missed or double-counted encoder steps at rotation speeds up to 3 detents per second. ### Power Subsystem A protected single-cell lithium-ion polymer battery powers the player. It charges from a 5 V USB-C input. A BQ24074 linear charger with power-path management handles charging and shares input power with the system. The USB-C port uses 5.1 kΩ CC pull-down resistors, and input current is limited to 500 mA, which any compliant source supports by default. Charge current is set to no more than 0.5C of the chosen cell. An NTC thermistor on the charger's TS pin blocks charging outside the cell's temperature window. A TPS63070 buck-boost regulator produces 3.6 V across the full battery range. Two LDOs derive the final rails: a TLV75733P for the digital 3.3 V rail and a TPS7A2033 low-noise LDO for the analog 3.3 V rail. The LDO stage keeps switching ripple off the audio supply. The regulator will be placed and filtered to minimize coupling into the headphone output. A battery-voltage divider to an ADC input provides a low-battery warning at 3.5 V and a controlled shutdown at 3.3 V. Controlled shutdown mutes audio, closes open files, and turns off the regulator. Our preliminary power estimates, which will be replaced with measurements from the first board, are: - STM32H743 running the audio pipeline and inference: 100 to 160 mA on the digital rail - microSD card, averaged during playback: 20 to 40 mA on the digital rail - SSD1306 OLED: 10 to 25 mA on the digital rail - PCM5102A DAC: 10 to 15 mA on the analog rail - TPA6132A2 amplifier with 32 Ω headphones: 5 to 15 mA on the analog rail The estimated total is 145 to 255 mA at 3.3 V, or about 0.5 to 0.85 W. At about 85% conversion efficiency, the battery supplies roughly 0.6 to 1.0 W. A cell of at least 1,200 mAh (about 4.4 Wh) therefore gives an estimated 4.4 to 7 hours of playback, which leaves margin above the 3-hour requirement. The final capacity will be chosen after measuring the first board. Components: - Protected 3.7 V nominal lithium-ion polymer cell (at least 1,200 mAh, final capacity after power measurements) - Texas Instruments BQ24074 charger with NTC temperature sensing - Texas Instruments TPS63070 buck-boost regulator - Texas Instruments TLV75733P (digital) and TPS7A2033 (analog) LDOs - USB-C connector, CC resistors, and input TVS protection - Power switch, battery connector, voltage-sensing divider, inductors, and capacitors ## Additional Features (if time allows) ### Typed Description Control Users can enter short descriptions such as "a little more bass" or "less sharp" with an on-screen character selector operated by the wheel. A small onboard intent classifier maps supported phrases to a listening goal and strength. Unrecognized requests prompt the user to pick a supported goal. This feature does not require a cloud language model. Components: - On-screen text-entry interface using the existing controls - Compact intent classifier and supported phrase vocabulary ### Additional Audio Formats If processing and memory allow, we will add FLAC playback and 44.1 kHz support. These additions will be tested with adaptive EQ enabled to confirm that decoding and sample-rate changes do not interrupt playback. ### Supplementary Blinded Listening Check As supporting evidence only (not a success criterion), we may run a blinded, loudness-matched A/B comparison between the adaptive controller and the best fixed preset with volunteer listeners who are not on the team. ## Criterion For Success ### Core player 1. **Reliable playback:** Plays stereo, 16-bit, 48 kHz WAV files from microSD for 60 continuous minutes on battery with AI EQ enabled. UART logs must show zero resets, zero buffer underruns, and zero inference deadline misses, with gains updating ten times per second and every inference finishing within 50 ms. 2. **Standalone control:** Track navigation, play/pause, volume, all three listening goals, EQ strength, and bypass all work through the wheel and buttons, with no phone or computer. Each input is reflected on the display within 100 ms. 3. **Output quality:** At the headphone jack into 32 Ω with EQ bypassed, the player measures a frequency response within ±1 dB from 20 Hz to 20 kHz, THD+N below 0.1% at 1 kHz and 0.2 Vrms, and at least 90 dB A-weighted dynamic range. 4. **Battery and thermal:** Plays for at least 3 hours with AI EQ enabled at a fixed volume setting that produces 0.2 Vrms for a −12 dBFS 1 kHz tone. Recharges from a 5 V USB-C source while drawing no more than 500 mA. All ICs stay within their rated temperatures, and the board surface stays below 45 °C during playback and charging. ### AI equalization 5. **Outperforms the best fixed preset (primary AI criterion):** For **each** of the three goals, on the 100-track held-out test subset processed on the device at full strength, the neural controller must: - achieve a mean tonal-balance error at least **20% lower** than the best fixed preset; - achieve lower error than the preset on at least **65 of the 100 tracks**; and - show a statistically significant improvement (one-sided Wilcoxon signed-rank test, **p < 0.01**). We will also report the oracle upper bound, the causal rule-based controller, and host-simulation results on the full 300-track test split. 6. **Content-dependent adaptation:** On spliced test clips that join two held-out passages whose band levels differ by at least 4 dB in the goal's primary band, the controller's gain in that band moves at least 1 dB toward its new steady-state value within 1 second of the splice, with continuous playback. Across the test subset, the primary-band gain must also vary with a standard deviation of at least 1 dB between segments, confirming that the output is not effectively constant. 7. **Safe, click-free transitions:** Across the evaluation playlist and 100 goal and bypass changes, processed peaks never exceed the −1 dBFS limiter ceiling. Logged gain trajectories show no step larger than 0.05 dB between 1 ms coefficient updates, and bypass crossfades last at least 50 ms. No playback interruption occurs. 8. **The evaluation represents the real device:** On-device quantized gains match the floating-point host model within 0.25 dB RMS on the validation split. Live-playback CRC-32 logs match offline evaluation mode exactly on 10 tracks. Analog recordings of those tracks reproduce the digital error reduction within 5 percentage points. ## Development Plan Work on the AI component starts in parallel with the hardware, and a go/no-go checkpoint is placed before board integration. - **Weeks 1–2:** On the hardware side, we will complete the schematic and part selection and bring up audio and power on a development board with a DAC breakout. On the AI side, we will download the dataset, create the artist-disjoint splits, and commit the frozen targets, split hashes, metric code, and thresholds. We will then compute the best fixed presets and the oracle labels. - **Weeks 3–4:** We will lay out the PCB and place the first-round order. In parallel, we will train the first model and run host simulation on the validation split. **Checkpoint:** the model must show at least a 30% error reduction versus the best fixed preset on validation for all three goals, which leaves margin above the 20% test requirement. - **Weeks 5–6:** We will assemble the board and measure power, noise, and audio output. In parallel, we will run quantization-aware training, port the model to the STM32, and check that the quantized model matches the floating-point model. - **Weeks 7–8:** We will make second-round PCB fixes and build the enclosure. In parallel, we will integrate real-time playback, the offline evaluation mode, and CRC equivalence logging. - **Weeks 9–10:** We will run battery runtime and thermal tests. In parallel, we will run the final on-device test-set evaluation (once), perform the analog confirmation, and work on stretch features. - **Weeks 11–12:** We will complete final measurements, prepare the demo, and write the final report with all baseline comparisons. |
|||||