Chapter 3 checkpoint

Ten questions about flexible models, unsupervised learning and interpretation.

Choose an answer before opening its feedback. Aim to explain the reason as well as select the option before starting Lab 3.

Checkpoint

A perfectly fitted tree

A deep tree has zero training RMSE and much worse validation RMSE. What is the most useful next step?

1. A Restrict tree complexity and compare candidates using development folds

   Correct. Correct: depth, minimum leaf size or pruning can reduce overfitting without consulting the final test.

2. B Report the training result because it uses more rows

   Not quite. Training observations influenced fitting; their error does not estimate performance on new rows.

3. C Try test splits until one also gives zero RMSE

   Not quite. Selecting a split by its result invalidates the evaluation.

Checkpoint

What makes a forest useful?

Why can averaging a collection of trees outperform a single tree?

1. A Averaging guarantees removal of all systematic error

   Not quite. Shared biases, omitted features and data leakage remain shared problems.

2. B Diverse tree errors can partly cancel, reducing prediction variability

   Correct. Correct: resampled rows and feature subsampling encourage diversity; identical trees would add no benefit.

3. C Every tree is trained only on test observations

   Not quite. All fitting uses training data. Test observations must remain outside model development.

Checkpoint

A boosting schedule

You halve the learning rate but keep a small iteration count fixed. What might happen?

1. A The ensemble may underfit because it has taken smaller total correction steps

   Correct. Correct: learning rate and the number of iterations work together and should be assessed jointly.

2. B The result is guaranteed to stay identical

   Not quite. Changing correction size changes both the path and subsequent trees.

3. C Validation is unnecessary because smaller learning rates cannot overfit

   Not quite. No learning rate guarantees a useful fit or an honest evaluation.

Checkpoint

SVM geometry

Why should standardisation be inside the SVR cross-validation pipeline?

1. A To use the final test distribution to define the distances

   Not quite. Test statistics must not influence the learned transformation.

2. B Because trees and SVMs require identical feature scales for mathematical validity

   Not quite. Tree threshold ordering usually does not need standardisation; the reason here is distance geometry.

3. C To learn comparable feature scales using only each training fold

   Correct. Correct: RBF similarity depends on distance, and fitting the scaler outside the folds leaks validation information.

Checkpoint

The complete selection procedure

You plan to choose between ridge, forest and SVR and tune each. What must be inside the inner loop of nested CV?

1. A Both the family choice and its hyperparameter selection

   Correct. Correct: the outer fold must remain excluded from every choice the procedure makes.

2. B Only the final model fit after selecting the family from outer scores

   Not quite. Then outer outcomes influenced selection and do not independently evaluate that new choice.

3. C The final test labels, to improve the search

   Not quite. The final test is outside all selection and fitting.

Checkpoint

Variance is not accuracy

Two principal components retain 75% of input variance. What follows?

1. A A classifier using them will be 75% accurate

   Not quite. Explained input variance and agreement with target labels are different quantities.

2. B Those two directions retain 75% of variation under the fitted scaling convention

   Correct. Correct: PCA did not optimise the target, so prediction must be evaluated separately.

3. C The omitted components contain only measurement noise

   Not quite. Low-variance directions can contain useful target information.

Checkpoint

Unsupervised leakage

PCA is fitted on every row before a classifier is cross-validated. Is the evaluation leak-free because PCA never saw labels?

1. A Yes, only label access can cause leakage

   Not quite. Input-distribution information can leak through fitted preprocessing too.

2. B No; PCA and its scaler must be refitted within each training fold

   Correct. Correct: the evaluation should reproduce learning the complete procedure from training data alone.

3. C Yes, if the first two components look sensible

   Not quite. A plausible plot does not establish independence of the evaluation.

Checkpoint

A well-separated cluster

A group has a high silhouette score. What is the justified claim?

1. A It is a newly discovered biological category

   Not quite. Biological interpretation requires independent evidence and experimental context.

2. B Cluster number 0 is more fundamental than cluster number 1

   Not quite. The identifiers are arbitrary labels.

3. C Its assignments are relatively compact and separated under the chosen distance

   Correct. Correct: the geometric diagnostic depends on representation, scaling and algorithm.

Checkpoint

Correlated descriptors

Two strongly correlated descriptors each have low individual permutation importance. What is a plausible explanation?

1. A The model can substitute one for the other, hiding their shared predictive information

   Correct. Correct: low individual importance does not prove absence of useful information; grouped analyses answer a different question.

2. B Neither descriptor can ever predict the target

   Not quite. Importance is conditional on this fitted model and the other available features.

3. C Permutation importance proves both have no causal effect

   Not quite. It measures predictive reliance, not causation.

Checkpoint

An explanation versus an intervention

A partial-dependence curve rises with polar surface area. What can you conclude?

1. A Increasing the polar surface area of any real molecule will cause the same increase in solubility

   Not quite. Real structural changes affect multiple properties and may leave the observed domain.

2. B The model’s average prediction rises under the plotted hypothetical feature replacements

   Correct. Correct: that is model behaviour; correlated features and unrealistic combinations limit its interpretation.

3. C The curve is a calibrated prediction interval for future compounds

   Not quite. A response curve provides neither interval coverage nor calibration.

PreviousLab 3 Tabular model comparison: select, explain, then testNextChapter 3 key points

Edit this pageReport a problemLast updated 2026-11-04