Chapter 3 of Foundations

Beyond linear models soon

Trees, ensembles and support vector machines for the cases a straight line cannot handle; unsupervised learning for finding structure without labels; and how to interpret what a model has learned.

soon This chapter is planned and not yet written. The outline below is the current plan; it may change.

What it will cover

• Decision trees and why they overfit; random forests and gradient boosting
• Support vector machines and the kernel idea
• Unsupervised learning: principal component analysis and k-means clustering
• Model interpretation: permutation importance, partial dependence, SHAP-style explanations
• Choosing between model families with cross-validation
• Lab 3: a tabular modelling bake-off on the solubility and breast cancer data

Planned learning outcomes

1. Train and tune tree ensembles and SVMs with scikit-learn, and say when each is appropriate.
2. Use PCA and clustering to explore a dataset without labels and interpret the components.
3. Explain a fitted model’s behaviour to a scientist using permutation importance and partial dependence.

In the meantime

Chapters 0, 1 and 2 are complete and are the prerequisites for this chapter.

Previoususerdata: 0 Chapter 2 key pointsNextChapter 4 Deep learning essentials with PyTorch

Edit this pageReport a problemLast updated 2026-11-02