These figures show, in order, (1) a peer to peer system (Gnutella), (2) a datacenter, (3) a sensor network, (4) the Internet ISP topology.

CS 525 Spring 2016

Advanced Distributed Systems

[Tuesdays and Thursdays] @ [2.00 PM - 3.15 PM] @ [DCL 1310]

Instructor: Dr. Indranil Gupta ("Indy"), indy at illinois dawt edu, 3112 SC.

Office hours: Tuesdays and Thursdays 3.15 pm - 4.00 pm (class days only) in 3112 SC.

TAs: Luke Leslie (lmlesli2) and Le Xu (lexu1). Office hours by appointment.

Class Discussion Forum: Piazza

Important Links and Handouts

This is a course on distributed systems. It brings together research centered in cloud computing, peer-to-peer systems, distributed algorithms and on sensor networks. The course reviews classical work in these scattered areas of distributed computing research, and explores overlaps across them. The course has one semester-long project that is aimed at producing either an entrepreneurial effort for industry or  a conference/workshop-quality research paper. The entrepreneurial effort must make a technological case and a reasonable business case for innovation in today's world. The paper must address an open research problem, prove bounds and/or propose new algorithms, and contain analytical and experimental evaluation. Previous course projects have had high success rates at conferences/workshops (see past CS525/CS598IG projects at the bottom of this page). Three to five "best projects'' at the end of the semester will be earmarked for expedited submission to a renowned conference, with the help of the instructor's involvement even after the semester is over. 

Experimental Testbeds: We are trying to get all students access to a few VMs in the CS VM server farm (details forthcoming by Jan-end). Students can also request course staff for a PlanetLab slice, or Emulab project (there are limited number of slots available for each: about 5 projects on each testbed). Therefore, (1) all requests will be granted on an as-needed basis, and (2) you will be given an account on typically only one of these testbeds, so please choose carefully depending on your project requirements! In general, there are plenty of cluster resources available in the department, so try to procure some of those! (alternately, AWS and sometimes Google cloud/Azure give out student grants, which you have to apply for individually).

• BEEER..., Demetriou, Hojjati, Long
  
• Distributed validation..., Krishna, Rausch, Ujcich
  
• Efficient guaranteed..., Goyal, Gangapuram, Jain
  
• Update Atomic ..., Liu, Santhanam, Mukerji
  
• Web Service on the ..., Zhao, Shao, Yu
  

• CollabOnCode, Davids, Dao
• On-demand Games, Teng, McKenna, Nie
  

Latest News

5/11: This course is now over. You should have received (or will soon receive) comments on your final report from Indy via email. You will hear about your course grade directly from the University. Have a great summer!

1/13: Course web page online. Happy New Year 2016! All updates will be posted on Piazza.

Policies on Ethics, Attribution and Cheating

• Reviews: You may discuss papers with others in the class, but reviews should be written independently. In other words, if two reviews are found to have been copied from each other, both students will be punished. The best way to ensure that you are safe is by mentioning right at the top of your review that "This review was written after discussion with Mr. John Doe and Ms. Jane Joe".

• Presentation: Your work (presentations and reviews) should be original and independent, although you are allowed to cooperate with your partner for your session. Reusing slides from someone else's presentation should be done only if it is absolutely necessary, but should be acknowledged right at the beginning of the presentation. 

• The standard university policies on original work, cheating and attribution apply to all work in the course. Violation of these may result in either lowering of course grade by one letter, or failing the course, or a different final decision left to the instructor.

A Sampling of Past CS525/CS598IG Projects that were Subsequently Published in Conferences/Journals (check http://dprg.cs.uiuc.edu for copies of papers)

Select Spring 2015 CS525 CS525 Project Papers that became conference/journal papers

• Mayank Pundir, Luke Leslie, Indranil Gupta, Roy Campbell. "Zorro: Zero-Cost Reactive Failure Recovery in Distributed Graph Processing," ACM Symposium on Cloud Computing (ACM SoCC), 2015.
  
• Chris X. Cai, Shayan Saeed, Indranil Gupta, Roy Campbell, Franck Le. "Phurti: Application and Network-Aware Flow Scheduling for Multi-Tenant MapReduce Clusters," Proc. IEEE International Conference on Cloud Engineering (IC2E), 2016.
• Le Xu Boyang Jerry Peng, Indranil Gupta. "Stela: Enabling Stream Processing Systems to Scale-in and Scale-out On-demand," Proc. IEEE International Conference on Cloud Engineering (IC2E), 2016.
• Mayank Pundir, Manoj Kumar, Luke Leslie, Indranil Gupta, Roy Campbell. "Supporting On-demand Elasticity in Distributed Graph Processing," Proc. IEEE International Conference on Cloud Engineering (IC2E), 2016.
  

Select Spring 2014 CS525 CS525 Project Papers that became conference/journal papers

• (Awarded Best Paper!) Md Tanvir Al Amin, Shen Li, Muntasir Raihan Rahman, Panindra Tumkur Seetharamu, Shiguang Wang, Tarek F. Abdelzaher, Indranil Gupta, Mudhakar Srivatsa, Raghu K Ganti, Reaz Ahmed, Hieu Le. "SocialTrove: A Self-summarizing Storage Service for Social Sensing," Proc. IEEE International Conference on Autonomic Computing (ICAC), 2015.
• Rajath Subramanyam, Luke Leslie, Wenting Wang, Indranil Gupta. "Idempotent Distributed Counters Using a Forgetful Bloom Filter," Proc. IEEE International Conference on Cloud and Autonomic Computing (ICCAC), 2015.
  
• Wenting Wang, Le Xu, Indranil Gupta. "Scale up Vs. Scale out in Cloud Storage and Graph Processing Systems," Proc. IEEE Workshop on Cloud Analytics (IWCA) (colocated with IEEE International Conference on Cloud Engineering (IC2E)), 2015.

• Shen Li, Shaohan Hu, Shiguang Wang, Lu Su, Tarek F. Abdelzaher, Indranil Gupta, Richard Pace. "WOHA: Deadline-Aware Map-Reduce Workflow Scheduling Framework over Hadoop Cluster," Proc. 34th International Conference on Distributed Computing Systems (ICDCS), 2014.
• Furquan Shaikh, Fangzhou Yao, Indranil Gupta, Roy Campbell, "VMDedup: Memory De-duplication in Hypervisor," Proc. IEEE International Workshop on Cloud Analytics (IWCA), 2014.
• Shannon Chen, Klara Nahrstedt, Indranil Gupta, "3DTI Amphitheater: A Manageable 3DTI Environment with Hierarchical Stream Prioritization," Proc. ACM Multimedia Systems Conference (MMSys), 2014.

Select Spring 2010 CS525 CS525 Project Papers that became conference/journal papers

• (Awarded Best Paper!) Liangliang Cao, Hyun Duk Kim, Min-Hsuan Tsai, Brian Cho, Zhen Li, Indranil Gupta, ChengXiang ZHai, Thomas S. Huang, "Delta-SimRank Computing on MapReduce," Proc. 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining (BigMine), 2012.

Select Spring 2009 CS525 Project Papers that became conference/journal papers

• Abhishek Verma, Nicolas Zea, Brian Cho, Indranil Gupta, Roy Campbell, "Breaking the MapReduce Stage Barrier," Proc. IEEE International Conference on Cluster Computing, 2010. (Extended version in Springer Journal of Cluster Computing, 2011)

Select Spring 2008 CS525 Project Papers that became conference/journal papers

• Md. Ahsan Arefin, Yusuf Sarwar, Indranil Gupta, Klara Nahrstedt. "Q-Tree: A Multi-Attribute Based Range Query Solution for Tele-Immersive Framework," Proc. International Conference on Distributed Computing Systems (ICDCS), 2009.
• Mirko Montanari, Riccardo Crepaldi, Indranil Gupta, Robin Kravets. "Using Failure Models for Controlling Data Availability in Wireless Sensor Networks," Proc. IEEE Infocom Minisymposium, 2009.
• Rahul Malik, Chandrasekhar Ramachandran, Sangkyun Kim, Xin Jin, Jiawei Han, Indranil Gupta, Klara Nahrstedt. "MLR-Index: An Index Structure for Fast and Scalable Similarity Search in High Dimensions," Proc. 2009 Int. Conf. on Scientific and Statistical Database Management (SSDBM'09), 2009.
• Rahul Malik, Chandrasekhar Ramachandran, Indranil Gupta, Klara Nahrstedt. "A Scalable and Memory-Efficient Feature Extraction Algorithm for Short 3D Video Segments," Proc. IMMERSCOM, 2009.

Select Spring 2007 CS525 Project Papers that became conference/journal papers

• I-Hong Hou, Yu-En Tsai, Tarek Abdelzaher, Indranil Gupta. "AdapCode: Adaptive Network Coding for Code Updates in Wireless Sensor Networks," Proc. INFOCOM, 2008
• Vijay Anand Korthikanti, Prateek Mittal, Indranil Gupta. "Fair K Mutual Exclusion Algorithm for Peer to Peer Systems," Proc. International Conference on Distributed Computing Systems (ICDCS), 2008.
• Ramses Morales, Brian Cho, Indranil Gupta. "AVMEM - Availability-Aware Overlays for Management Operations," Proc. ACM/IFIP/USENIX Middleware, pp. 266-286, (Springer LNCS 4834), November, 2007.
• Wanmin Wu. Zhenyu Yang, Indranil Gupta, Klara Nahrstedt. "Towards Multi-Site Collaboration in 3D Tele-Immersive Environments," Proc. International Conference on Distributed Computing Systems (ICDCS), 2008.

Select Spring 2006 CS598IG Project Papers that became conference/journal papers

• Long Vu, Indranil Gupta, Jin Liang, Klara Nahrstedt. "Measurement and Modeling of a Large-scale Overlay for Multimedia Streaming," Invited Paper in Proc. Intnl. Conf. on Heterogeneous Networking for Quality, Reliability, Security and Robustness (QShine) , 2007. Also in TOMCCAP journal.
• Ercan Ucan, Nat Thompson, Indranil Gupta. "A piggybacking approach to reduce overhead in sensor network gossiping," Proc. 2nd Intnl Wshop on Middleware for Sensor Networks (MidSens), 2007.

Select Fall 2004 CS 598IG Project Papers that became conference/journal papers

• Jin Liang, Indranil Gupta, Klara Nahrstedt. "Reliable On-Demand Management Operations for Large-scale Distributed Applications," ACM SIGOPS Operating Systems Review (OSR) Special Issue on Gossip-Based Networking, vol. 41, no. 5, pp. 82-88, October, 2007.
• Steve Ko, Indranil Gupta, Yookyung Jo. "A New Class of Nature-Inspired Algorithms for Self-Adaptive Peer-to-Peer Computing," ACM Transactions on Autonomous and Adaptive Systems (TAAS), to appear, September, 2008.
• James Newell, Indranil Gupta. "The P2P MultiRouter: a Black Box Approach to Run-time Adaptivity for P2P DHTs," IEEE Conference on Collaborative Computing (CollaborateCom), 2005.
• Romit Roy Choudhury, Pradeep Kyasanur, Indranil Gupta. "Smart Gossip: An Adaptive Gossip-based Broadcasting Service for Sensor Networks", Proc. IEEE Conf. Mobile Ad-Hoc and Sensor Systems (MASS), pp. 91-100, 2006.
• Jin Liang, Indranil Gupta, Klara Nahrstedt. "MON: On-demand Overlays for Distributed System Management," Usenix WORLDS, 2005.
• And more!