Fall 2025
CS 198-126: Deep Learning for Visual Data
University of California, Berkeley
Semester: Fall 2025 (2 units) Instructors: Tejas Prabhune, Tim Xie, Rishi Athavale, Saathvik Selvan Lecture Time & Location: Physics 2, 7-8pm, Wednesday & Thursday Office Hours Time & Location: TBD
Links
Ed Link: https://edstem.org/us/join/sUaKqc Enrollment Form: https://forms.gle/S6rLfenSRVWQDn9XA (maximum capacity is 100) Enrollment Process: Codes will be sent out after the application due date but before the first lecture Gradescope Code: 3XK78B Spring 2025 Lecture Recordings: Link Fall 2024 Lecture Recordings: Link
Course Mission
Welcome to CS 198-126: Deep Learning for Visual Data, offered by Machine Learning at Berkeley!
This course is designed to introduce students to a subset of computer vision that relies on deep learning, spanning both introductory and recent state-of-the-art methods. Our goal is to give students a breadth of understanding of how different computer vision systems can be applied to a wide variety of tasks, as well as a depth of understanding for a certain subset of such systems. Students will ideally leave with:
Wide exposure to different systems used for solving different computer vision problems and a high-level understanding of how and why they work
An understanding of what these systems like these look like under the hood and how they are translated from high-level ideas to low-level details in practice
Experience in implementing end-to-end deep learning systems from scratch in PyTorch
Immediately after completing this course, it is our hope that students will have the knowledge and practical experience necessary to undertake independent projects in the area of computer vision and continue their education on their own.
Prerequisites
The prerequisite for this course is the general minimum background required for understanding both concepts in and the tools used for deep learning. We will likely not have the staff bandwidth to help with prerequisite content so we strongly recommend having at least the following foundational knowledge before taking this class:
Basic Python
At the level of CS 61A or from other experiences working on programming projects in the past
You need to be comfortable with object-oriented programming in Python and writing well-organized modular clean code
Basic linear algebra
Familiarity with vector and matrix operations (ex. vector dot products, matrix-vector products, vector norms, etc.)
An optional 3Blue1Brown playlist is linked here to refresh your memory on this (chapters 1, 2, 3, 4, 5 and 8 might be helpful… but no need to worry about something like eigenvectors — just become comfortable with vector and matrix notation)
Multivariable Calculus
At the level of Math 53
Familiarity with the concept of a partial derivative, a gradient and the chain rule for derivatives
Probability and Statistics
At the level of CS 70
Familiarity with basic statistics and probability distributions (ex. bayes rule, random variables, expectation, variance, normal distribution, etc.)
Methods of Instruction
Every week, there will be two 1 hour live lectures, and an associated concept-check quiz due on the following Monday. Lectures will be held in-person, and attendance is mandatory. In addition, there will be four programming-heavy homework assignments spread across the semester, where students will get an opportunity to implement and interact with the concepts learned in class.
All of the materials, including the lecture videos, slides, and assignments, will be updated here during the progression of the course.
Office Hours
We will host weekly in-person office hours / homework party to provide assistance with the quizzes and programming assignments, or to clarify any concepts from class. This will also be a great opportunity for students to work together in groups (if they wish to).
Course Outline
Lectures will aim to strike a balance between surveying a wide breadth of content as well as exploring architectural details and the underlying math. Slide decks and lecture videos will keep getting linked below as the semester progresses.
Cluster 1: Intro to Deep Learning
Week 1-2
What is Machine Learning? (slides)
Deep Learning 1: Introduction to Neural Networks (slides)
Deep Learning 2: Neural Network Optimization (slides)
Deep Learning 3: Representation Learning (slides)
Cluster 2: Essential Computer Vision
Week 3-5
Images and Convolutions (slides)
Convolutional Neural Networks (slides)
Advanced CV Architectures
Object Detection
Segmentation
Cluster 3: Transformers
Week 5-8
Sequence Modeling
Attention and Transformers
Vision Transformers
Multimodality and CLIP
Advanced Detection and Segmentation
Self-Supervised Learning
Cluster 4: Generative Modeling
Week 8-11
Variational Autoencoders
Generative Adversarial Networks
Vector Quantization
Diffusion Part 1
Diffusion Part 2
Cluster 5: Special Topics
Week 12
3D Reconstruction
Office hours for Diffusion HW
Assignments
Syllabus
We will have mandatory quizzes and assignments to ensure retention of the course material and to give you practical experience with deep learning. We will try to make sure that the workload is appropriate for a 2 unit course and that every work assigned is meaningful and critical to your learning. Our hope is that all of the assignments combined will take, on average, around 6 hours per week total.
Course Components
Weekly Lecture Quizzes
We will assign weekly concept-check quizzes on Gradescope. While these quizzes are not meant to be intense, they will require you to pay attention during the lectures. Quizzes will be graded on completion but we still expect you to make an honest attempt at them.
We anticipate that there will be a total of 8-10 quizzes throughout the semester.
Programming Assignments
There will be 4 programming assignments this semester, all of them graded on correctness. Each of them will walk you through implementing some end-to-end deep learning pipeline, all the way from model training to evaluation. Since these models can be fairly compute-intensive to train, we will utilize the free GPU services provided by Google Colab and/or Kaggle. You are welcome to set up your own hardware, but course staff may be unable to help you in the event that you run into any issues with it.
We may also have you go over parts of different research papers during some of the assignments in order to give you more practice with reading and extracting relevant information from academic literature.
Extensions
We are adopting an extension policy inspired by CS 161:
Extensions of up to 3 days will be automatically approved and will require no additional actions on your part. In particular, each Gradescope assignment will have a due date (the original deadline) and a late due date (the original deadline + 3 days) — requesting an extension of ≤ 3 days is equivalent to submitting between the due date and late due date.
Extensions beyond 3 days but up to 7 days will need to be approved by a facilitator — please fill out the Extensions Request Form linked here.
Extensions beyond 7 days will not be considered. While this course is only a DeCal, we don’t want you to fall behind:
The free GPU services mentioned above have usage quotas that are automatically applied to your account, that could possibly take a while (from a few hours to a couple of days) to refresh. If multiple programming assignments are attempted in a relatively short period of time, say during the last week of instruction, we cannot guarantee if you will receive enough GPU resources to successfully complete them. This reason may seem far-fetched but there were multiple students in past semesters who were unable to submit the homeworks because they started very late and got blocked by these GPU quotas.
Several lectures will build upon previous lectures. In order to ensure that you are keeping up with the content, we cannot delay a concept-check quiz by more than a week.
Passing
This course is graded on a P/NP scale and a 70% is considered a passing grade. Here is the grading breakdown that we will follow this semester. Note that Homework 6 is not listed here, but can be used to replace the grade of any of Homeworks 2-5.
Quizzes
20%
Homework 1
10%
Homework 2
15%
Homework 3
15%
Homework 4
15%
Homework 5
15%
Attendance
10%
Resources
Stanford offers a similar course: CS231n: Deep Learning for Computer Vision.
We recommend checking out 3Blue1Brown’s Youtube series on linear algebra and multivariable calculus if you need a quick refresher on those subjects. He also has an amazing series explaining gradient descent and backpropagation for neural networks, something that we will cover in our introductory deep learning lectures as well. We also recommend CS 70 lecture notes (note 13 and beyond) as a quick reference for probability and statistics concepts.
Communication
We will use Edstem for all announcements, questions and discussion. We will monitor Ed pretty regularly so please make a private post on it if you need to communicate with us directly.
If you can’t access Ed, please email [email protected]. Please do not email anybody else individually.
Acknowledgements
We would like to thank Aryan Jain, Jake Austin, Arvind Rajaraman, Ryan Alameddine, Val Rotan, Verona Teo, Vanessa Teo, Saathvik Selvan, April Zhang, Derek Xu, John So, Arjun Sripathy, Alexander Wan, Sohom Paul and Harshika Jalan for their help in developing the course and/or facilitating its previous iterations!
Last updated