ECE 598 RE: Dynamical Systems and Neural Networks

About This Course

Welcome! This course is a deep dive into the fascinating intersection of two fields that are shaping the future of intelligent systems: the elegant mathematics of dynamical systems and the powerful machinery of neural networks. We will move beyond treating neural networks as "black boxes" and instead use the tools of chaos, stability, and attractor dynamics to understand *why* they work the way they do, and how we can make them better. You'll not only learn to analyze these systems but also to build and simulate them, gaining a better intuition for their behavior. My goal is to give you a flavor of the exciting research happening in this space and equip you with the theoretical and computational skills to contribute. If you're curious about the fundamental principles of learning and computation in both brains and machines, this course is for you.

Weekly Schedule (Approximate)

The course is divided into four main modules.

• Weeks 1-3: Module I - Foundations of Dynamical Systems
  Topics: Discrete/continuous time systems, stability, phase portraits, chaos, Lyapunov exponents, bifurcations.
• Weeks 4-8: Module II - Dynamical Systems in Machine Learning
  Topics: Optimization as dynamics, Transformers and RNNs as dynamical systems, vanishing/exploding gradients, Neural ODEs.
• Weeks 9-12: Module III - Dynamical Systems in Neural Circuit Models
  Topics: Single neuron dynamics, rate-based and spiking networks, the balanced state, chaos in neural network models.
• Weeks 13-14: Module IV - Advanced Topics & Project Presentations
  Topics: Discovering dynamics from data, random dynamical systems, final project work.

Lectures

• Lecture 1: Introduction to the Course - [Slides]

Final Project

The final project is a significant research-oriented project, completed individually. The goal is to apply the concepts from the course to a problem in machine learning or computational neuroscience, culminating in a written report (NeurIPS/PRL style) and a presentation.

Course Description

This graduate course explores the interplay between dynamical systems theory and neural networks (artificial and biological). Students analyze how stability, attractors, bifurcations, and chaos govern behavior, learning, and computation. The course covers foundational discrete and continuous-time dynamical systems, applied to analyzing machine learning models (including training dynamics) and modeling complex neural circuits (rate-based and spiking). Emphasis is on hands-on computational analysis and developing a theoretical understanding. The course culminates in a research project applying these interdisciplinary concepts.

Prerequisites

Required: Multivariable calculus and linear algebra (e.g., UIUC MATH 415 & MATH 416); basic proficiency in Python or Julia.
Recommended: Familiarity with ordinary differential equations (e.g., MATH 285/286) and foundational machine learning concepts (e.g., ECE 449/CS 446).

Grading

• Homework Assignments (Analytical & Computational): 30%
• Mini-Quizzes: 15%
• Project Pitch & Written Proposal: 15%
• Final Project (Report and Presentation): 40%

Resources & Reading Material

Primary Texts (Excerpts)

• Pikovsky & Politi (2016). Lyapunov Exponents: A Tool to Explore Complex Dynamics.
• Izhikevich E. M. (2007). Dynamical Systems in Neuroscience.
• Gerstner W., et al. (2014). Neuronal Dynamics. [Online Version]
• Strogatz, S. H. (2018). Nonlinear Dynamics and Chaos.

A more extensive list of research papers for discussion will be provided during the course.

Software

We will primarily use Python or Julia for computational assignments. Familiarity with libraries like NumPy, SciPy, and Matplotlib is expected. We may also use machine learning libraries like PyTorch, Flux or JAX.