Chapter 2 of Foundations

Machine learning fundamentals

What a model is, how to fit one, and above all how to find out honestly whether it works. You will build regression and classification models with scikit-learn on real chemistry and medical data, and learn the evaluation habits that separate a useful model from a misleading one.

6 episodes3 h 45 min of teaching2 h 05 min of exercises1 lab, 1 h 30 minabout 7 h 20 min in all

What you will be able to do

Explain supervised learning as fitting a function \(f(x;\theta)\) by minimising a loss, and say where it helps and where it fails in chemistry and biology.
Fit linear regression from scratch with the normal equation and gradient descent, then with scikit-learn’s estimator API.
Evaluate a model honestly against a baseline with a held-out test set, MAE, RMSE, \(R^2\), parity plots and residual analysis.
Diagnose overfitting with learning and validation curves, and control it with ridge, lasso and cross-validated hyperparameter search.
Build and judge classifiers with confusion matrices, precision, recall, F1, ROC and precision–recall curves, including imbalanced classes.
Assemble leak-free pipelines with scaling, imputation and encoding, and recognise the leakage patterns that plague scientific datasets.

Schedule

Episode	Cumulative time	Questions
2.1What is machine learning? 25 min teaching + 10 min exercises	35 min	What does it mean for a computer to learn from data, and how is that different from fitting a curve? When is a question a regression, a classification or an unsupervised problem, and when is it not a machine-learning problem at all? Where does machine learning genuinely help in chemistry and biology, and where does it quietly fail?
2.2Linear regression from scratch, then scikit-learn 40 min teaching + 20 min exercises	1 h 35 min	How does a computer actually find the best straight line through data? What are the normal equation and gradient descent, and when would I use each? What do scikit-learn’s `fit`, `predict` and `score` do, and what is in `coef_` and `intercept_`? Can I read a regression coefficient as the effect of that feature?
2.3Evaluating models honestly 35 min teaching + 20 min exercises	2 h 30 min	Why can’t I judge a model on the data it was trained on? What is a baseline, and why must every model beat one? Which numbers (MAE, RMSE, R²) tell me how good a regression model is, and what do they hide?
2.4Overfitting, regularisation and cross-validation 45 min teaching + 25 min exercises	3 h 40 min	Why does a more flexible model often predict worse on new data? How do I tell whether my model is too simple or too flexible, and what do I do about it? What does regularisation actually change in a linear model? How do I choose a setting like the regularisation strength without cheating with the test set?
2.5Classification 40 min teaching + 25 min exercises	4 h 45 min	How does a model predict a category instead of a number, and where do the probabilities come from? Why is accuracy a poor summary of a classifier, and what should I report instead? What changes when the class I care about is rare, as active compounds are in a screen?
2.6Pipelines, preprocessing and data leakage 40 min teaching + 25 min exercises	5 h 50 min	Why does the same model score differently when I scale the features, and which models care? How do I handle missing values and text categories without writing fragile glue code? What is data leakage, how does it sneak into real scientific datasets, and how do I keep it out? How do I save a fitted model and make sure a colleague gets the same numbers?
Lab 2First end-to-end ML project: predicting solubility 90 min	7 h 20 min	Predict the aqueous solubility of 1,128 small molecules from five descriptors, from a reproducible split and a baseline through cross-validated ridge and kNN pipelines to a parity plot, an error analysis of the worst-predicted molecules and a written conclusion. Optional extension: turn it into a soluble/insoluble classifier with a confusion matrix and ROC curve.
QuizChapter 2 checkpoint	7 h 20 min	Ten questions covering the whole chapter.

Lab

Lab 2, 90 min

First end-to-end ML project: predicting solubility

Predict the aqueous solubility of 1,128 small molecules from five descriptors, from a reproducible split and a baseline through cross-validated ridge and kNN pipelines to a parity plot, an error analysis of the worst-predicted molecules and a written conclusion. Optional extension: turn it into a soluble/insoluble classifier with a confusion matrix and ROC curve.

Dataset: ESOL / Delaney aqueous solubility (downloaded at runtime with checksum; see data/README.md)

Open in Colab Download .ipynb View solutions

Before you start

Prerequisite

You should be comfortable with the material of Chapter 1: NumPy arrays and the \(X\) / \(y\) convention, pandas DataFrames, and matplotlib’s fig, ax interface. If you skipped it, do at least episodes 1.2 and 1.5, because this chapter picks up the solubility question exactly where 1.5 leaves it.

Maths corner

If you want the derivations behind the methods, the maths refresher covers vectors, dot products, gradients and the normal distribution. Each episode also has collapsible deep dive boxes with the mathematics; they are optional.