Algorithms for Machine Learning

CAP 4611 (Fall 2025)

Department of Computer Science
University of Central Florida
Orlando, FL, USA

Instructor and TA information

Instructor: Ali Siahkoohi
TAs:

See webcourses for the office hour location.

Description

We introduce basic principles and techniques in the field of machine learning. These are some of the key tools behind the emerging field of data science. These techniques now run behind the scenes to discover patterns and make predictions in various applications in our daily lives. We focus on many of the core machine learning technologies, with motivating applications from a variety of disciplines.

The material for this course—including slides, homework assignments, and the overall website structure—is adapted from Mark Schmidt’s CPSC 340: Machine Learning and Data Mining course at the University of British Columbia, used with his permission. We are grateful for his contribution in providing foundational material that helped shape this course.

Textbooks

There is no required textbook for the class. A introductory book that covers many (but not all) the topics we will discuss is the Artificial Intelligence book of Rusell and Norvig (AI:AMA) or the Artificial Intelligence book of Poole and Mackworth. More advanced books include The Elements of Statistical Learning (ESL) by Hastie et al., Murphy’s Machine Learning: A Probabilistic Perspective (ML:APP) which can be accessed through the library, and Bishop’s Pattern Recognition and Machine Learning (PRML).

Grading

  • Assignments 30%, Midterm 20%, Final 50%.
  • There will be 2% extra credit for submitting teaching evaluation, if more than 80% of students submitted the evaluation.

Schedule

Lecture slides will be posted on the course website before each lecture. The instructor reserves the right to alter this schedule as needed.

Dates Lecture Topics Resources Assignments/Notes
Aug 19, Aug 21 Motivation and Syllabus
Exploratory Data Analysis
Decision Trees		 What is Machine Learning?
Machine Learning
Rise of the Machines
Talking Machine Episode 1
Mathematics for Machine Learning
Gotta Catch’em all
Why Not to Trust Statistics
Visualization Types
Google Chart Gallery
Other Tools
A Visual Introduction to Machine Learning
Decision Trees
Entropy
What is Big O Notation?
AI:AMA 19.2-3, ESL: 9.2, ML:APP 16.2		 Assignment 1 a1.zip
Assignment instructions
Big-O Notes
Aug 26, Aug 28 Fundamentals of Learning
Probability Slides		 7 Steps of Machine Learning
IID
Cross-validation
Bias-variance		 Assignment 1 due: Aug 29
Course Notation Guide
Probability Notes
Sep 2, Sep 4 Probabilistic Classifiers
Non-Parametric Models
Ensemble Methods
Clustering		 No Free Lunch
AI:AMA 19.4-5, ESL 7.1-7.4, 7.10, ML:APP 1.4, 6.5
Conditional probability (demo)
Naive Bayes
Probabilities and Battleship
AI:AMA 12.6, ESL 4.3, ML:APP 2.2, 3.5, 4.1-4.2
K-nearest neighbours
Decision Theory for Darts
Norms
AI:AMA 19.7, ESL 13.3, ML:APP 1.4
Ensemble Methods
Random Forests
Empirical Study
Kinect
AI:AMA 19.8, ESL: 7.11, 8.2, 15, 16.3, ML:APP 6.2.1, 16.2.5, 16.6
Clustering
K-means clustering (demo)
K-Means++ (demo)
IDM 8.1-8.2, ESL: 14.3
DBSCAN (video, demo)
Hierarchical Clustering
Phylogenetic Trees
IDM 8.4		 Assignment 2 a2.zip
Sep 9, Sep 11 Outlier Detection
Linear Regression
Nonlinear Regression		 Empirical Study
IDM 8.3, ESL 14.3.12, ML:APP 25.5
Linear Regression (demo, 2D data, 2D video)
Least Squares
Essence of Calculus
Partial Derivative
Gradient
ESL 3.1-2, ML:APP 7.1-3, AI:AMA 19.6
Why should one learn machine learning from scratch?
Essence of Linear Algebra
Matrix Differentiation
Fluid Simulation (video)
ESL 5.1, 6.3		 Linear Algebra Notes
Linear/Quadratic Gradients
Sep 16, Sep 18 Gradient Descent
Robust Regression
Feature Selection		 Gradient Descent
Convex Functions
ML:APP 7.4
Genome-Wide Association Studies
AIC
BIC
ESL 3.3, 7.5-7		 Assignment 2 due: Sep 19
Sep 23, Sep 25 Regularization
More Regularization
Linear Classifiers		 ESL 3.4., ML:APP 7.5, AI:AMA 19.4
RBF video
RBF and Regularization video
ESL 6.7, ML:APP 13.3-4
Perceptron
ESL 4.5, ML:APP 8.5		 Assignment 3 a3.zip
Sep 30, Oct 2 More Linear Classifiers
Feature Engineering
Convolutions		 Support Vector Machines
ESL 4.4, 12.1-2, ML:APP 8.1-3, 9.5 14.5, AI:AMA 19.6
Gmail Priority Inbox
But what is a convolution?		 Assignment 3 due: Oct 3
Oct 7, Oct 9 Kernel Trick
Stochastic Gradient Descent
Boosting		 ESL 12.3, ML:APP 14.1-4
Stochastic Gradient Descent
Theory and Practice
ML:APP 8.5
AdaBoost (video)
XGBoost (video)
ML:APP 16.4		 Assignment 4 a4.zip
Max and Argmax Notes
Oct 14, Oct 16 MLE and MAP Maximum Likelihood Estimation
ML:APP 9.3-4		 Midterm - Oct 16
Oct 21, Oct 23 Principal Component Analysis
More PCA
Beyond PCA		 Principal Component Analysis
ESL 14.5, IDM B.1, ML:APP 12.2
Making Sense of PCA
SVD
Eigenfaces
Non-Negative Matrix Factorization (original - access from UCF)
Recommender Systems
Netflix Prize
ESL 14.6, ML: APP 13.8		 Assignment 4 due: Oct 24
Oct 28, Oct 30 Multi-Dimensional Scaling
Neural Networks		 Nonlinear Dimensionality Reduction
t-SNE demo
ESL 14.8-9, IDM B.2
Google Video
What is a Neural Network?
Interactive Guide
ML:APP 16.5, ESL 11.1-4, AI:AMA 21.1		 Assignment 5 a5.zip
Nov 4, Nov 6 Over-Parameterization
Deep Neural Networks
Convolutional Neural Networks		 Fortune Article
Deep Learning References
Alchemy
ML:APP 28.3, ESL 11.5, AI:AMA 21.2 and 21.4-5
Convolutional Neural Networks
ML:APP 28.4, ESL 11.7, AI:AMA 21.3		 Assignment 5 due: Nov 7
Nov 11, Nov 13 Autoencoders and Multi-Label AI:AMA 21.6-8 Assignment 6 a6.zip
Veterans Day - No Class Tuesday
Nov 18, Nov 20 Fully-Convolutional Networks
Recurrent Neural Networks		 Assignment 6 due: Nov 21
Nov 25, Nov 27 What do we Learn?
LSTMs and Transformers		 Thanksgiving - No Class Thursday
Dec 2 Final Exam