Teaching

I teach reinforcement learning, theoretical computer science, and cyber-physical systems. Across these courses, we examine how algorithms work, which assumptions they rely on, and what can be proved about their behavior. Mathematical analysis, concrete examples, and implementation are all part of the process.

Current and Recent Courses

Deep Reinforcement Learning

CSCI 4932 / CSCI 5932 / ROBO 5329 · Fall 2026

An upper-level undergraduate and graduate course on reinforcement learning with function approximation. Topics include multi-armed bandits, Monte Carlo and temporal-difference learning, deep Q-networks, policy-gradient methods, actor–critic algorithms, and proximal policy optimization. The course combines mathematical foundations with implementation and experimentation.

Theory of Computation

CSCI 5444 · Fall 2025, Fall 2026

A mathematically rigorous introduction to automata theory, computability, and computational complexity. The course develops formal models of computation and examines both the capabilities and fundamental limitations of algorithms.

Online Teaching

Theoretical Foundations of Reinforcement Learning

The three-course Foundations of Reinforcement Learning Specialization, launched through CU Boulder and Coursera in June 2026, introduces reinforcement learning from a mathematical and conceptual perspective:

  1. Foundations of Reinforcement Learning
    Markov decision processes, value functions, Bellman equations, dynamic programming, and foundational learning methods.
  2. Deep Reinforcement Learning
    Function approximation, deep value-based methods, policy gradients, actor–critic algorithms, and modern deep-RL techniques.
  3. Reward Programming
    Reward design, temporal objectives, reward machines, specification of agent behavior, and the relationship between rewards and intended outcomes.

The specialization is designed to make reinforcement learning accessible while preserving the mathematical structure needed to understand why its algorithms work.

Reinforcement Learning

Theoretical Foundations of Reinforcement Learning

CSCI 7000 · Summer 2023, Spring 2024

A graduate course on the mathematical foundations of reinforcement learning. Topics included Markov decision processes, value functions, Bellman equations, dynamic programming, tabular learning algorithms, reward design, convergence guarantees, and the challenges introduced by function approximation.

Theory and Algorithms

Theory of Computation

CSCI 5444 · Spring 2017; Fall 2018, 2020–2023, 2025, 2026

Graduate-level study of finite automata, context-free languages, computability, and complexity. Some offerings also incorporated material from computational learning theory.

Theory of Computation

CSCI 3434 · Fall 2016, Fall 2023

An undergraduate introduction to formal languages, automata, computability, and complexity, designed for students encountering theoretical computer science for the first time.

Design and Analysis of Algorithms

CSCI 5454 · Fall 2017

A graduate course on the design and rigorous analysis of algorithms.

Data Structures

CSCI 2270 · Spring 2019–2023

A core undergraduate course connecting the mathematical foundations of data structures with their practical implementation and analysis.

Cyber-Physical Systems

Foundations of Cyber-Physical Systems

CSCI 5854 · Spring 2018

An introduction to the modeling, design, and verification of safety-critical cyber-physical systems. Topics included discrete, continuous, and hybrid dynamical models, formal specification, reachability, and verification.

Earlier Teaching

Before joining the University of Colorado Boulder, I taught the following courses at the Indian Institute of Technology Bombay: