Chapter 2 key points

Every key point of the chapter, generated from the episode front matter.

Supervised learning fits a function f(x; θ) from examples that come with answers: regression predicts a number, classification a category. Unsupervised learning looks for structure without labels.
Features go in X (one row per sample, one column per feature); labels go in y. Almost every tool in this course expects that layout.
Training means choosing the parameters θ that make a loss function small on the training data; for regression the loss is usually the mean squared error.
The goal is generalisation, performance on samples the model has never seen. A curve that passes through every training point can be a worse model than one that misses them all slightly.
Machine learning pays off with large, pattern-rich data and expensive-to-measure targets. It fails on tiny datasets, under distribution shift, with noisy labels and with confounded data.

A linear model predicts ŷ = Xθ; fitting means choosing θ to minimise the mean squared error on the training data.
The normal equation gives the exact least-squares solution in one line: solve (X^⊤X)θ = X^⊤y with np.linalg.solve, never with an explicit inverse.
Gradient descent reaches the same answer step by step: compute the gradient, move against it, repeat. The learning rate decides whether it converges, crawls or explodes.
Standardise features before gradient descent. The normal equation does not care about scale; gradient descent does.
scikit-learn wraps all of this in the estimator API: fit, predict, score, coef_, intercept_. Every model in this course uses the same names.
Coefficients are not effects. They depend on the units of the feature and on which other, correlated features are in the model.
Everything in this episode is fitted and scored on the same data. Whether the model is any good is the question of episode 2.3.

Performance on training data is not evidence. Always report metrics on data the model has never seen.
Split once, reproducibly, with train_test_split(..., random_state=42), and keep the test set untouched until the end.
A model that cannot beat a DummyRegressor (or DummyClassifier) has learned nothing useful.
MAE is the typical error in target units; RMSE punishes large errors more; R² says how much variance is explained. Report at least two of them.
Metrics summarise, plots explain. A parity plot and a residual plot show where a model goes wrong.

For nested least-squares models, the best training error cannot increase with flexibility; test error can rise as the model starts fitting noise.
Bias is error from being too rigid; variance is error from following noise. Every model choice trades one for the other.
A learning curve says whether more data would help; a validation curve says how a hyperparameter should be set.
Ridge shrinks coefficients smoothly; lasso sets some exactly to zero. Both need scaled features, so put a StandardScaler in the pipeline.
Cross-validation replaces one lucky or unlucky split with k systematic ones; report the mean and the spread.
Choose hyperparameters with cross-validation on the training data; the test set is touched once, at the very end, and never used to choose anything.

Logistic regression is a linear model whose output passes through the sigmoid to give a probability; the class is a threshold on that probability.
Always compare against DummyClassifier; on imbalanced data a useless model can have impressive accuracy.
The confusion matrix is the ground truth. Precision, recall and F1 are read from it; report them for the class you care about.
ROC/AUC measures ranking across all thresholds; the precision–recall curve is the honest picture when positives are rare.
The threshold is a decision, not a constant: move it to trade recall against precision.

Scale features for kNN, logistic regression, ridge and lasso; fit the scaler on the training data only.
A Pipeline bundles preprocessing and model into one estimator: fit learns everything from the training set, predict applies it unchanged to new data.
ColumnTransformer sends numeric columns through imputation and scaling and categorical columns through one-hot encoding; get_feature_names_out() tells you what came out.
Leakage is any way information about the test set reaches the model before evaluation. The test score then measures memory, not generalisation.
Duplicates, repeated patients or batches, preprocessing fitted on all data and after-the-fact columns are the four leaks to check for in every scientific dataset.
GroupKFold keeps every sample of a group on one side of the split; scaffold and sequence-identity splits are the same idea for molecules and proteins.
Save the whole pipeline with joblib.dump, fix every seed, pin versions and never touch the test set twice.