These figures show, in order, (1) a peer to peer system (Gnutella), (2) a datacenter, (3) an IoT/sensor network, (4) the Internet ISP topology.


                                  

CS 525 Spring 2024

Advanced Distributed Systems

[Wednesdays and Fridays] @ [11:00 AM - 12.15 PM] @ [1310 DCL]

 


Instructor: Dr. Indranil Gupta ("Indy"), indy at illinois dawt edu, Zoom/3112 SC.

Indy's Office hours: Right after class until 1 PM (chat right after class or in his office 3112 SC).

TAs: Anna Karanika (annak8) and Beomyeol Jeon (bj2); Office hours by appointment only.

Class Discussion Forum: Piazza



Best Project Reports, in arbitrary order:

(only teams are mentioned, no project names)
Top Project Groups
Special Mention Project Groups
Second List: Special Mention Project Groups
  • Mittal, Zhang, Shah
  • Phan, Zhang




Important Links and Handouts

>> Paper Selections <<
>> Schedule and Sign-up Sheet <<
 

 
 
Course Information [pdf]  

Doing Presentations, Reviews, and Scribes

 

 





Project Idea Starters, Resources and Interesting Links [htm]
 
  writing your term paper/project survey/midterm/final report [htm]

Structuring Your Project Code [pdf]

 

 

This is a course on distributed systems. It brings together research centered in cloud computing, peer-to-peer systems, distributed algorithms and on sensor networks. The course reviews classical work in these scattered areas of distributed computing research, and explores overlaps across them. The course has one semester-long project that is aimed at producing either an entrepreneurial effort for industry or  a conference/workshop-quality research paper. The entrepreneurial effort must make a technological case and a reasonable business case for innovation in today's world. The paper must address an open research problem, prove bounds and/or propose new algorithms, and contain analytical and experimental evaluation. Previous course projects have had high success rates at conferences/workshops (see past CS525/CS598IG projects at the bottom of this page). Three to five "best projects'' at the end of the semester will be earmarked for expedited submission to a renowned conference, with the help of the instructor's involvement even after the semester is over. 

This is an in-person course only. Alas due to the discussion-oriented nature of the class, we do not have online or hybrid sections.

Experimental Testbeds: All students will get access to a few VMs in the CS VM server farm (details forthcoming in Feb). Students can also request course staff for a CloudLab (formerly Emulab) project (there are limited number of resources available for the first few groups that ask). Please note: (1) all requests will be granted on an as-needed basis, and (2) you will be given an account on typically only one of these last two testbeds, so please choose carefully depending on your project requirements! In general, there are plenty of cluster resources available in the department, so try to procure some of those! (alternately, AWS and sometimes Google cloud/Azure give out student grants, which you have to apply for individually).





 

Latest News

1/3/23: Course web page online. Happy New Year 2023! All updates will be posted on Piazza (see link above).

Policies on Ethics, Attribution and Cheating

  • Reviews: You may discuss papers with others in the class, but reviews should be written independently. In other words, if two reviews are found to have been copied from each other, both students will be penalized. The best way to ensure that you are safe is by mentioning right at the top of your review that "This review was written after discussion with Mr. John Doe and Ms. Jane Joe".

  • Presentation: Your work (presentations and reviews) should be original and independent, although you are allowed to cooperate with your partner for your session. Reusing slides from someone else's presentation should be done only if it is absolutely necessary, but should be acknowledged right at the beginning of the presentation. 

  • The standard university policies on original work, cheating and attribution apply to all work in the course. Violation of these may result in either lowering of course grade by one letter, or failing the course, or a different final decision left to the instructor.


A Sampling of Past CS525/CS598IG Projects that were Subsequently Published in Conferences/Journals (check http://dprg.cs.uiuc.edu for more recent papers since 2016, and for PDFs of the papers below)

There have been many papers published from CS525 sessions subsequent to Spring 2016, but I have sadly not been able to keep the (long) list below updated since 2016! Find them all at: http://dprg.cs.uiuc.edu

Select Spring 2016 CS525 Project Papers that became conference/journal papers

  • Varun Badrinath Krishna, Michael J. Rausch, Ben E. Ujcich, Indranil Gupta, William H. Sanders. “REMAX: Reachability-Maximizing P2P Detection of Erroneous Readings in Wireless Sensor Networks,” Proc. IEEE/IFIP Dependable Systems and Networks (DSN), 2017.

Select Spring 2015 CS525 Project Papers that became conference/journal papers

  • (Awarded Best Paper!) Mayank Pundir, Manoj Kumar, Luke Leslie, Indranil Gupta, Roy Campbell. "Supporting On-demand Elasticity in Distributed Graph Processing," Proc. IEEE International Conference on Cloud Engineering (IC2E), 2016.
  • Mayank Pundir, Luke Leslie, Indranil Gupta, Roy Campbell. "Zorro: Zero-Cost Reactive Failure Recovery in Distributed Graph Processing," ACM Symposium on Cloud Computing (ACM SoCC), 2015.
  • Chris X. Cai, Shayan Saeed, Indranil Gupta, Roy Campbell, Franck Le. "Phurti: Application and Network-Aware Flow Scheduling for Multi-Tenant MapReduce Clusters," Proc. IEEE International Conference on Cloud Engineering (IC2E), 2016.
  • Le Xu Boyang Jerry Peng, Indranil Gupta. "Stela: Enabling Stream Processing Systems to Scale-in and Scale-out On-demand," Proc. IEEE International Conference on Cloud Engineering (IC2E), 2016.

Select Spring 2014 CS525 Project Papers that became conference/journal papers

  • (Awarded Best Paper!) Md Tanvir Al Amin, Shen Li, Muntasir Raihan Rahman, Panindra Tumkur Seetharamu, Shiguang Wang, Tarek F. Abdelzaher, Indranil Gupta, Mudhakar Srivatsa, Raghu K Ganti, Reaz Ahmed, Hieu Le. "SocialTrove: A Self-summarizing Storage Service for Social Sensing," Proc. IEEE International Conference on Autonomic Computing (ICAC), 2015.
  • Rajath Subramanyam, Luke Leslie, Wenting Wang, Indranil Gupta. "Idempotent Distributed Counters Using a Forgetful Bloom Filter," Proc. IEEE International Conference on Cloud and Autonomic Computing (ICCAC), 2015.
  • Wenting Wang, Le Xu, Indranil Gupta. "Scale up Vs. Scale out in Cloud Storage and Graph Processing Systems," Proc. IEEE Workshop on Cloud Analytics (IWCA) (colocated with IEEE International Conference on Cloud Engineering (IC2E)), 2015.
Select Spring 2013 CS525 Project Papers that became conference/journal papers
  • Shen Li, Shaohan Hu, Shiguang Wang, Lu Su, Tarek F. Abdelzaher, Indranil Gupta, Richard Pace. "WOHA: Deadline-Aware Map-Reduce Workflow Scheduling Framework over Hadoop Cluster," Proc. 34th International Conference on Distributed Computing Systems (ICDCS), 2014.
  • Furquan Shaikh, Fangzhou Yao, Indranil Gupta, Roy Campbell, "VMDedup: Memory De-duplication in Hypervisor," Proc. IEEE International Workshop on Cloud Analytics (IWCA), 2014.
  • Shannon Chen, Klara Nahrstedt, Indranil Gupta, "3DTI Amphitheater: A Manageable 3DTI Environment with Hierarchical Stream Prioritization," Proc. ACM Multimedia Systems Conference (MMSys), 2014.

Select Spring 2010 CS525 Project Papers that became conference/journal papers

  • (Awarded Best Paper!) Liangliang Cao, Hyun Duk Kim, Min-Hsuan Tsai, Brian Cho, Zhen Li, Indranil Gupta, ChengXiang ZHai, Thomas S. Huang, "Delta-SimRank Computing on MapReduce," Proc. 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining (BigMine), 2012.

Select Spring 2009 CS525 Project Papers that became conference/journal papers

  • Abhishek Verma, Nicolas Zea, Brian Cho, Indranil Gupta, Roy Campbell, "Breaking the MapReduce Stage Barrier," Proc. IEEE International Conference on Cluster Computing, 2010. (Extended version in Springer Journal of Cluster Computing, 2011)

Select Spring 2008 CS525 Project Papers that became conference/journal papers

  • Md. Ahsan Arefin, Yusuf Sarwar, Indranil Gupta, Klara Nahrstedt. "Q-Tree: A Multi-Attribute Based Range Query Solution for Tele-Immersive Framework," Proc. International Conference on Distributed Computing Systems (ICDCS), 2009.
  • Mirko Montanari, Riccardo Crepaldi, Indranil Gupta, Robin Kravets. "Using Failure Models for Controlling Data Availability in Wireless Sensor Networks," Proc. IEEE Infocom Minisymposium, 2009.
  • Rahul Malik, Chandrasekhar Ramachandran, Sangkyun Kim, Xin Jin, Jiawei Han, Indranil Gupta, Klara Nahrstedt. "MLR-Index: An Index Structure for Fast and Scalable Similarity Search in High Dimensions," Proc. 2009 Int. Conf. on Scientific and Statistical Database Management (SSDBM'09), 2009.
  • Rahul Malik, Chandrasekhar Ramachandran, Indranil Gupta, Klara Nahrstedt. "A Scalable and Memory-Efficient Feature Extraction Algorithm for Short 3D Video Segments," Proc. IMMERSCOM, 2009.

Select Spring 2007 CS525 Project Papers that became conference/journal papers

  • I-Hong Hou, Yu-En Tsai, Tarek Abdelzaher, Indranil Gupta. "AdapCode: Adaptive Network Coding for Code Updates in Wireless Sensor Networks," Proc. INFOCOM, 2008
  • Vijay Anand Korthikanti, Prateek Mittal, Indranil Gupta. "Fair K Mutual Exclusion Algorithm for Peer to Peer Systems," Proc. International Conference on Distributed Computing Systems (ICDCS), 2008.
  • Ramses Morales, Brian Cho, Indranil Gupta. "AVMEM - Availability-Aware Overlays for Management Operations," Proc. ACM/IFIP/USENIX Middleware, pp. 266-286, (Springer LNCS 4834), November, 2007.
  • Wanmin Wu. Zhenyu Yang, Indranil Gupta, Klara Nahrstedt. "Towards Multi-Site Collaboration in 3D Tele-Immersive Environments," Proc. International Conference on Distributed Computing Systems (ICDCS), 2008.

Select Spring 2006 CS598IG Project Papers that became conference/journal papers

  • Long Vu, Indranil Gupta, Jin Liang, Klara Nahrstedt. "Measurement and Modeling of a Large-scale Overlay for Multimedia Streaming," Invited Paper in Proc. Intnl. Conf. on Heterogeneous Networking for Quality, Reliability, Security and Robustness (QShine) , 2007. Also in TOMCCAP journal.
  • Ercan Ucan, Nat Thompson, Indranil Gupta. "A piggybacking approach to reduce overhead in sensor network gossiping," Proc. 2nd Intnl Wshop on Middleware for Sensor Networks (MidSens), 2007.

Select Fall 2004 CS 598IG Project Papers that became conference/journal papers

  • Jin Liang, Indranil Gupta, Klara Nahrstedt. "Reliable On-Demand Management Operations for Large-scale Distributed Applications," ACM SIGOPS Operating Systems Review (OSR) Special Issue on Gossip-Based Networking, vol. 41, no. 5, pp. 82-88, October, 2007.
  • Steve Ko, Indranil Gupta, Yookyung Jo. "A New Class of Nature-Inspired Algorithms for Self-Adaptive Peer-to-Peer Computing," ACM Transactions on Autonomous and Adaptive Systems (TAAS), to appear, September, 2008.
  • James Newell, Indranil Gupta. "The P2P MultiRouter: a Black Box Approach to Run-time Adaptivity for P2P DHTs," IEEE Conference on Collaborative Computing (CollaborateCom), 2005.
  • Romit Roy Choudhury, Pradeep Kyasanur, Indranil Gupta. "Smart Gossip: An Adaptive Gossip-based Broadcasting Service for Sensor Networks", Proc. IEEE Conf. Mobile Ad-Hoc and Sensor Systems (MASS), pp. 91-100, 2006.
  • Jin Liang, Indranil Gupta, Klara Nahrstedt. "MON: On-demand Overlays for Distributed System Management," Usenix WORLDS, 2005.
  • And more!

Academic Integrity Policy (Please Read):

We adhere by the CS academic integrity policies outlined at this webpage. It is the course policy that all of the work you submit for grading, or in support of graded material, as an individual or project group, shall be your own product, from inception to completion. You cannot copy or appropriate code, ideas, written text, figures, or anything else (no, not even from papers we have read in class). As always, the golden rule is -- cite the paper! Always attribute to the original source. In some cases, it may be possible for you to reproduce some of these (e.g., a figure in your paper), but always with the proper attribution. We rigorously check every submitted assignment (including code) for violations of academic integrity.

We rigorously check every submitted HW and MP (including code) for violations of academic integrity.

All violations of this academic integrity policy are treated seriously in this course. Don't risk it - just avoid cheating and the temptation to! That way, you'll learn more and years later you will be happier about standing up for yourself.

Mental Health Resources (Please Read):

Diminished mental health, including significant stress, mood changes, excessive worry, substance/alcohol abuse, or problems with eating and/or sleeping can interfere with optimal academic performance, social development, and emotional wellbeing. The University of Illinois offers a variety of confidential services including individual and group counseling, crisis intervention, psychiatric services, and specialized screenings at no additional cost. If you or someone you know experiences any of the above mental health concerns above, it is strongly encouraged to contact or visit any of the University’s resources provided below. Getting help is a smart and courageous thing to do -- for yourself and for those who care about you.

Counseling Center: 217-333-3704, 610 East John Street Champaign, IL 61820.

McKinley Health Center:217-333-2700, 1109 South Lincoln Avenue, Urbana, Illinois 61801.

Anti-Racism/Anti-Bias/Anti-Microaggression Policy for CS525 (Please Read):

(This statement borrows heavily from a draft statement from the Anti-Racism Task Force in the Grainger College of Engineering). This course believes in a truly inclusive Illinois that is free from overt racism, prejudice, bias, as well as from micro-aggressions (commonplace hostile, derogatory, or negative prejudicial slights and insults)--these have no place in this course, in the classroom or outside, or on campus. We believe that Black Lives Matter. We believe that students (and faculty) who are African American/Black, Hispanic/Latino(a), Native American/Alaskan Native, and women, can live and learn without marginalization and without suffering racism, bias, and prejudice--from anyone else in the class, on campus, in our community, or in the world.

In this course, if you face any kind of bias, prejudice, or racism, from fellow students or course staff, please be courageous and speak up. If you are apprehensive to do so, or face sustained incidents, please contact the instructor (Indy) via email. If you have feedback or constructive suggestions on how to improve the anti-racist environment in this course, please contact the instructor (Indy) via email. You can also report these incidents to the BART Center (Bias Assessment and Response Team) on campus. Each of us is unique, due to our backgrounds and experiences, and so each of us makes our campus environment richer. At the same time, each of us is continually learning to be a better human being and become more aware of our own biases. A famous quote by Dr. Mae Jemison (first Black astronaut in space) captures this last sentiment beautifully, "Never limit yourself because of others' limited imagination; never limit others because of your own limited imagination."

This course also supports and adheres by the Anti-Racism and Inclusivity Stance/Statement from Grainger College of Engineering (Prof. Gupta was also involved in drafting the college's statement, as part of his Anti-Racism Task Force (ARTF) membership.).

Here are Other important syllabus statements from the Grainger College of Engineering and campus.


(C) Indranil Gupta, 2003 - Present.