Date Lecture (and links to advance videos when applicable) Links and
Notes		 Reading Suggestions Comments
Intro and Basics 1/20 Introduction and Logistics Intro & logistics Slide set
1/22 Edge AI and Mitigating Resource Bottlenecks
 Bottlenecks Slide set  
1/27 Overfitting and Self-supervised Learning  Group requests due
1/29 Edge AI and Mitigating the Data Bottleneck All groups assigned
The Data Bottleneck: Self- Supervised Data- Efficient Learning for IoT 2/3 Class Project Ideas Introduction. Projects Slide set  
2/5 Fundamentals of Self-Supervised Learning: Tokenization, Pre-training, Fine-tuning, Backbone Architectures (e.g., auto-encoders, transformers, etc), and Issues with Scaling Laws for IoT Applications Self-Supervised Learning (SSL) Slide set
2/10 Self-supervised Learning Architectures for Time-Series Data: RNNs, LSTMs, and State Space Models SSL Models for Time- Series Data 1. Schmidt, Robin M. "Recurrent neural networks (rnns): A gentle introduction and overview." arXiv preprint arXiv:1912.05911 (2019).
2. Kexin Zhang, Qingsong Wen, Chaoli Zhang, Rongyao Cai, Ming Jin, Yong Liu, James Y. Zhang et al. "Self-supervised learning for time series analysis: Taxonomy, progress, and prospects." IEEE transactions on pattern analysis and machine intelligence 46, no. 10 (2024): 6775-6794.
3. Albert Gu, Karan Goel, and Christopher Ré. "Efficiently modeling long sequences with structured state spaces." arXiv preprint arXiv:2111.00396 (2021).
4. Albert Gu, and Tri Dao. "Mamba: Linear-time sequence modeling with selective state spaces." In First conference on language modeling. 2024.		 Note: Project title and abstract due
2/12 Representation Learning from Multimodal Sensor Data Multimodal Intro

 HW1 Out		 1. Chao Zhang, Zichao Yang, Xiaodong He, and Li Deng. "Multimodal intelligence: Representation learning, information fusion, and applications." Journal of Selected Topics in Signal Processing, 2020.
2. Dave Vedant, Fotios Lygerakis, and Elmar Rueckert. "Multimodal visual-tactile representation learning through self-supervised contrastive pre-training." In 2024 IEEE ICRA, pp. 8013-8020. IEEE, 2024.
3. Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. "Videobert: A joint model for video and language representation learning." In Proceedings of the IEEE/CVF international conference on computer vision, pp. 7464-7473. 2019. 		 
2/17 Representation Learning from Multimodal Sensor Data (Student Led) G4

Multimodal Papers		 1. Shengzhong Liu, Tomoyoshi Kimura, Dongxin Liu, Ruijie Wang, Jinyang Li, Suhas Diggavi, Mani Srivastava, and Tarek Abdelzaher. "Focal: Contrastive learning for multimodal time-series sensing signals in factorized orthogonal latent space." Advances in Neural Information Processing Systems 36 (2023): 47309-47338.
2. Chen, Yatong, Chenzhi Hu, Tomoyoshi Kimura, Qinya Li, Shengzhong Liu, Fan Wu, and Guihai Chen. "SemiCMT: Contrastive cross-modal knowledge transfer for iot sensing with semi-paired multi-modal signals." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, no. 4 (2024): 1-30.
3. Xiaomin Ouyang, Jason Wu, Tomoyoshi Kimura, Yihan Lin, Gunjan Verma, Tarek Abdelzaher, and Mani Srivastava. "MMbind: Unleashing the potential of distributed and heterogeneous data for multimodal learning in iot." In Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems, pp. 491-503. 2025.
4. Tomoyoshi Kimura, Xinlin Li, Osama Hanna, Yatong Chen, Yizhuo Chen, Denizhan Kara, Tianshi Wang et al. "InfoMAE: Pair-efficient cross-modal alignment for multimodal time-series sensing signals." In Proceedings of the ACM on Web Conference 2025, pp. 3084-3095. 2025.
5. Li, Zechen, Shohreh Deldari, Linyao Chen, Hao Xue, and Flora D. Salim. "SensorLLM: Aligning large language models with motion sensors for human activity recognition." In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 354-379. 2025.
6. Yuwei Zhang, Kumar Ayush, Siyuan Qiao, A. Ali Heydari, Girish Narayanswamy, Maxwell A. Xu, Ahmed A. Metwally et al. "SensorLM: Learning the Language of Wearable Sensors." arXiv preprint arXiv:2506.09108 (2025).
Debate #1 (20 min)

Student led talk (45 min + 10 min Q&A)

See note @48 for debate concluding remarks on Piazza.
2/19 Self-supervised Learning from Frequency Domain Data
Frequency Domain Intro

HW2 Out		 Slide set Debate #2 (20 min)
2/24 Self-supervised Learning from Frequency Domain Data (Student Led) G3

Frequency Domain Papers		 1. Shuochao Yao, Ailing Piao, Wenjun Jiang, Yiran Zhao, Huajie Shao, Shengzhong Liu, Dongxin Liu, Jinyang Li, Tianshi Wang, Shaohan Hu, Lu Su, Jiawei Han and Tarek Abdelzaher, "STFNets: Learning Sensing Signals from the Time-Frequency Perspective with Short-Time Fourier Neural Networks," In Proc. The Web Conference (WWW), San Francisco, CA, May 2019.
2. Dongxin Liu, Tianshi Wang, Shengzhong Liu, Ruijie Wang, Shuochao Yao, and Tarek Abdelzaher. "Contrastive self-supervised representation learning for sensing signals from the time-frequency perspective." In 2021 International Conference on Computer Communications and Networks (ICCCN), pp. 1-10. IEEE, 2021.
3. Yuan Gong, Cheng-I. Lai, Yu-An Chung, and James Glass. "Ssast: Self-supervised audio spectrogram transformer." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 10, pp. 10699-10709. 2022.
4. Setareh Rahimi Taghanaki, Michael Rainbow, and Ali Etemad. "Self-supervised human activity recognition with localized time-frequency contrastive representation learning." IEEE Transactions on Human-Machine Systems 53, no. 6 (2023): 1027-1037.
5. Denizhan Kara, Shengzhong Liu, Jinyang Li, Dongxin Liu, Tianshi Wang, Ruijie Wang, Yizhuo Chen, Yigong Hu, Tarek Abdelzaher, "FreqMAE: Frequency-Aware Masked Autoencoder for Multi-Modal IoT Sensing," In Proc. The Web Conference (WWW), May 2024.
6. Denizhan Kara, Tomoyoshi Kimura, Yatong Chen, Jinyang Li, Ruijie Wang, Yizhuo Chen, Tianshi Wang, Shengzhong Liu, Lance Kaplan, Joydeep Bhattacharyya, Tarek Abdelzaher, "PhyMask: An Adaptive Masking Paradigm for Efficient Self-Supervised Learning in IoT," In Proc. 22nd ACM Conference on Embedded Networked Sensor Systems (SenSys), Hangzhou, China, November 2024.
 HW2 Debate (20 min)

Student led talk (45 min + 10 min Q&A)
2/26 Handling Spatial-Temporal IoT Data Spatial Temporal Intro

HW3 Out		 Slide Set 2-page project proposal due
3/3 Handling Spatial-Temporal IoT Data (Student Led) G2 1. Yinghui Zhang, Hu An, Yaxuan Xing, Yang Liu, and Tiankui Zhang. "Learning temporal and spatial features jointly: A unified framework for space-time data prediction in industrial IoT networks." IEEE Sensors Journal 23, no. 16 (2023): 18752-18764.
2. Liu, Jing, et al. "Distributional and spatial-temporal robust representation learning for transportation activity recognition." Pattern Recognition (2023).
3. Porter Jenkins, Ahmad Farag, Suhang Wang, and Zhenhui Li. "Unsupervised representation learning of spatial data via multimodal embedding." In Proceedings of the 28th ACM international conference on information and knowledge management, pp. 1993-2002. 2019.
4. Yizhuo Chen, Tianchen Wang, You Lyu, Yanlan Hu, Jinyang Li, Tomoyoshi Kimura, Hongjue Zhao, Yigong Hu, Denizhan Kara, and Tarek Abdelzaher. "Spar: Self-supervised placement-aware representation learning for multi-node iot systems." arXiv e-prints (2025): arXiv-2505.
5. Tianchen Wang, Yizhuo Chen, Hongjue Zhao, You Lyu, Jinyang Li, Tomoyoshi Kimura, Yigong Hu et al. "On Network-Efficient Multimodal Multi-Vantage Foundation Models for Distributed Sensing." In 2025 IEEE 22nd International Conference on Mobile Ad-Hoc and Smart Systems (MASS), pp. 19-27. IEEE, 2025.
6. Pengrui Quan, Brian Wang, Kang Yang, Liying Han, and Mani Srivastava. "Benchmarking Spatiotemporal Reasoning in LLMs and Reasoning Models: Capabilities and Challenges." arXiv preprint arXiv:2505.11618 (2025).
HW3 Debate (20 min) br>
Student led talk (45 min + 10 min Q&A)
Data Curation and "Faking" 3/5 Physical Data Curation and Augmentation HW4 Out Slide Set  
3/10 Physical Data Curation and Augmentation (Student Led) G6 1. Chenzhi Hu, Yatong Chen, Denizhan Kara, Shengzhong Liu, Tarek Abdelzaher, Fan Wu, Guihai Chen, “OpenMAE: Efficient Masked Autoencoder for Vibration Sensing with Open-Domain Data Enrichment,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (ACM IMWUT), also presented in UbiComp, Espoo, Finland, October 2025.
2. Jeongwoo Ju, Heechul Jung, Yoonju Oh, and Junmo Kim. "Extending contrastive learning to unsupervised coreset selection." IEEE Access 10 (2022): 7704-7715
3. Sungnyun KIm, Sangmin Bae, and Se-Young Yun. "Coreset sampling from open-set for fine-grained self-supervised learning." In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7537-7547. 2023.
4. Haizhong Zheng, Elisa Tsai, Yifu Lu, Jiachen Sun, Brian R. Bartoldson, Bhavya Kailkhura, Atul Prakash, “ELFS: LABEL-FREE CORESET SELECTION WITHPROXY TRAINING DYNAMICS,” ICLR 2025
5. Tianshi Wang, Jinyang Li, Ruijie Wang, Denizhan Kara, Shengzhong Liu, Davis Wertheimer, Antoni Martin, Raghu Ganti, Mudhakar Srivatsa, and Tarek Abdelzaher, “SudokuSens: Enhancing Deep Learning Robustness for IoT Sensing Applications using a Generative Approach,” In Proc. ACM Sensys, Istanbul, Turkey, November 2023.
6. Tianshi Wang, Qikai Yang, Ruijie Wang, Dachun Sun, Jinyang Li, Yizhuo Chen, Yigong Hu, Chaoqi Yang, Tomoyoshi Kimura, Denizhan Kara, Tarek F. Abdelzaher, “Fine-grained Control of Generative Data Augmentation in IoT Sensing,” In Proc. 38th Annual Conference on Neural Information Processing Systems (NeurIPS), Vancouver, Canada, December 2024. 		HW4 Debate (20 min)

Student led talk (45 min + 10 min Q&A)
3/12 Project Elevator Talks      
Break 3/17  Spring Break
3/19
The Compute Bottleneck: Efficient Inference at the IoT Edge 3/24 Input Data Filtering  
 
3/26 Model Reduction: Pruning, Quantization, Distillation HW5 Out
G1		   Instructor intro (20 min)

Student led talk (45 min + 10 min Q&A)
3/31 Neural Network Architecture Search     HW5 Debate (20 min)
4/2 Mixture of Experts Cascades HW6 Out
G7		   Instructor intro (20 min) 

Student led talk (45 min + 10 min Q&A)
4/7 Timing Guarantees   HW6 Debate (20 min)
4/9 Energy Consumption and Thermal Issues HW7 Out
    
4/14 Federated Learning, Distributed Fine-Tuning, and Test-Time Adaptation G8   HW7 Debate (20 min)

Student led talk (45 min + 10 min Q&A)
4/16 Closed loop control and related foundation models (RT-2, RT-X, etc) HW8 Out
Ethics 4/21 Ethical and Societal Considerations HW8 Debate (20 min)
4/23  
Student Projects 4/28 Student-led Final Project Presentations      
4/30 Student-led Final Project Presentations      
5/5 Recap