Algorithms for Machine Learning
CAP 4611 (Fall 2026)
Department of Computer Science
University of Central Florida
Orlando, FL, USA
Instructor and TA information
- Instructor: Ali Siahkoohi
-
- Email: alisk@ucf.edu
- Office hours: Tuesdays at 3–4pm
TAs: TBA
See webcourses for the office hour location.
Description
We introduce basic principles and techniques in the field of machine learning. These are some of the key tools behind the emerging field of data science. These techniques now run behind the scenes to discover patterns and make predictions in various applications in our daily lives. We focus on many of the core machine learning technologies, with motivating applications from a variety of disciplines.
The material for this course—including slides, homework assignments, and the overall website structure—is adapted from Mark Schmidt’s CPSC 340: Machine Learning and Data Mining course at the University of British Columbia, used with his permission. We are grateful for his contribution in providing foundational material that helped shape this course.
Textbooks
There is no required textbook for the class. A introductory book that covers many (but not all) the topics we will discuss is the Artificial Intelligence book of Rusell and Norvig (AI:AMA) or the Artificial Intelligence book of Poole and Mackworth. More advanced books include The Elements of Statistical Learning (ESL) by Hastie et al., Murphy’s Machine Learning: A Probabilistic Perspective (ML:APP) which can be accessed through the library, and Bishop’s Pattern Recognition and Machine Learning (PRML).
Grading
- Assignments 30%, Midterm 20%, Final 50%.
- There will be 2% extra credit for submitting teaching evaluation, if more than 80% of students submitted the evaluation.
Schedule
Lecture slides will be posted on the course website before each lecture. The instructor reserves the right to alter this schedule as needed.
| Dates | Lecture Topics | Resources | Assignments/Notes |
|---|---|---|---|
| Aug 25, Aug 27 | Motivation and Syllabus Exploratory Data Analysis Decision Trees |
What is Machine Learning? Machine Learning Rise of the Machines Talking Machine Episode 1 Mathematics for Machine Learning Gotta Catch’em all Why Not to Trust Statistics Visualization Types Google Chart Gallery Other Tools A Visual Introduction to Machine Learning Decision Trees Entropy What is Big O Notation? AI:AMA 19.2-3, ESL: 9.2, ML:APP 16.2 |
Assignment 1 a1.zip Big-O Notes |
| Sep 1, Sep 3 | Fundamentals of Learning Probability Slides |
7 Steps of Machine Learning IID Cross-validation Bias-variance |
Assignment 1 due: Sep 4 Course Notation Guide Probability Notes |
| Sep 8, Sep 10 | Probabilistic Classifiers Non-Parametric Models Ensemble Methods |
No Free Lunch AI:AMA 19.4-5, ESL 7.1-7.4, 7.10, ML:APP 1.4, 6.5 Conditional probability (demo) Naive Bayes Probabilities and Battleship AI:AMA 12.6, ESL 4.3, ML:APP 2.2, 3.5, 4.1-4.2 K-nearest neighbours Decision Theory for Darts Norms AI:AMA 19.7, ESL 13.3, ML:APP 1.4 Ensemble Methods Random Forests Empirical Study Kinect AI:AMA 19.8, ESL: 7.11, 8.2, 15, 16.3, ML:APP 6.2.1, 16.2.5, 16.6 |
Assignment 2 a2.zip |
| Sep 15, Sep 17 | Clustering More Clustering Outlier Detection |
Clustering K-means clustering (demo) K-Means++ (demo) IDM 8.1-8.2, ESL: 14.3 DBSCAN (video, demo) Hierarchical Clustering Phylogenetic Trees IDM 8.4 Empirical Study IDM 8.3, ESL 14.3.12, ML:APP 25.5 |
Linear Algebra Notes Linear/Quadratic Gradients |
| Sep 22, Sep 24 | Linear Regression Nonlinear Regression Gradient Descent |
Linear Regression (demo, 2D data, 2D video) Least Squares Essence of Calculus Partial Derivative Gradient ESL 3.1-2, ML:APP 7.1-3, AI:AMA 19.6 Why should one learn machine learning from scratch? Essence of Linear Algebra Matrix Differentiation Fluid Simulation (video) ESL 5.1, 6.3 Gradient Descent Convex Functions |
Assignment 2 due: Sep 25 |
| Sep 29, Oct 1 | Robust Regression Feature Selection Regularization |
ML:APP 7.4 Genome-Wide Association Studies AIC BIC ESL 3.3, 7.5-7 ESL 3.4., ML:APP 7.5, AI:AMA 19.4 |
Assignment 3 a3.zip |
| Oct 6, Oct 8 | More Regularization Linear Classifiers More Linear Classifiers |
RBF video RBF and Regularization video ESL 6.7, ML:APP 13.3-4 Perceptron ESL 4.5, ML:APP 8.5 Support Vector Machines ESL 4.4, 12.1-2, ML:APP 8.1-3, 9.5 14.5, AI:AMA 19.6 |
Assignment 3 due: Oct 9 |
| Oct 13, Oct 15 | Feature Engineering Kernel Trick Stochastic Gradient Descent |
Gmail Priority Inbox ESL 12.3, ML:APP 14.1-4 Stochastic Gradient Descent Theory and Practice ML:APP 8.5 |
Assignment 4 a4.zip |
| Oct 20, Oct 22 | Boosting | AdaBoost (video) XGBoost (video) ML:APP 16.4 |
Midterm - Oct 22 |
| Oct 27, Oct 29 | MLE and MAP Principal Component Analysis |
Maximum Likelihood Estimation ML:APP 9.3-4 Principal Component Analysis ESL 14.5, IDM B.1, ML:APP 12.2 |
Assignment 4 due: Oct 30 Max and Argmax Notes |
| Nov 3, Nov 5 | More PCA Beyond PCA |
Making Sense of PCA SVD Eigenfaces Non-Negative Matrix Factorization (original - access from UCF) Recommender Systems Netflix Prize ESL 14.6, ML: APP 13.8 |
Assignment 5 a5.zip |
| Nov 10, Nov 12 | Multi-Dimensional Scaling Neural Networks Over-Parameterization |
Nonlinear Dimensionality Reduction t-SNE demo UMAP demo ESL 14.8-9, IDM B.2 Google Video What is a Neural Network? Interactive Guide ML:APP 16.5, ESL 11.1-4, AI:AMA 21.1 |
Assignment 5 due: Nov 13 |
| Nov 17, Nov 19 | Deep Neural Networks | Fortune Article Deep Learning References Alchemy ML:APP 28.3, ESL 11.5, AI:AMA 21.2 and 21.4-5 |
Assignment 6 a6.zip |
| Nov 24, Nov 26 | Convolutions Convolutional Neural Networks Autoencoders Recurrent Neural Networks LSTMs and Transformers |
But what is a convolution? Convolutional Neural Networks ML:APP 28.4, ESL 11.7, AI:AMA 21.3 AI:AMA 21.6-8 |
Assignment 6 due: Nov 27 Thanksgiving - No Class Thursday |
| Dec 1, Dec 3 | What do we Learn? | ||
| TBA | Final Exam |