Chapter 1 of Foundations

Scientific Python for data work

The tools every later chapter assumes: NumPy arrays for fast numerical work, pandas DataFrames for tables of measurements, and matplotlib and seaborn for figures you can trust. You will practise on real data, from Antarctic penguins to a medical imaging dataset, and finish with a full exploratory analysis of the aqueous solubility of small molecules, the question Chapter 2 then tries to predict.

5 episodes2 h 45 min of teaching1 h 40 min of exercises1 lab, 1 h 15 minabout 5 h 40 min in all

What you will be able to do

Write short, readable Python for scientific data: lists and dictionaries as lookup tables, functions with docstrings, f-strings for formatted output, pathlib for files, and tracebacks read from the bottom up.
Store measurements in NumPy arrays, select them with slices and boolean masks, replace loops with vectorised arithmetic and broadcasting, and generate reproducible random numbers with a seeded generator.
Recognise the \(X\) (samples × features) / \(y\) (targets) convention that every model in this course consumes.
Load, inspect, filter, group, merge and reshape tables with pandas, and handle missing values and categorical columns deliberately rather than by accident.
Draw histograms, scatter plots, box plots and small multiples with matplotlib and seaborn, with units on every axis and colours that work for everyone.
Carry out an exploratory data analysis of the ESOL solubility dataset end to end, and repeat the workflow on the breast cancer Wisconsin dataset in the guided lab.

Schedule

Episode	Cumulative time	Questions
1.1Python refresher for scientists 25 min teaching + 15 min exercises	40 min	Which parts of Python do I actually need for data work, and which can wait? When should I reach for a list and when for a dictionary? How do I print a number with two decimals, or a table that lines up? An error message just filled my screen. Where do I start reading?
1.2NumPy arrays 35 min teaching + 20 min exercises	1 h 35 min	Why does scientific Python use arrays instead of lists, and what do I gain? How do I pick out the rows, columns or readings I want from a grid of numbers? What does it mean that NumPy is ‘vectorised’, and how much faster is it really? How do I get random numbers that are the same every time I run the notebook? What is the X / y convention that every model in this course expects?
1.3pandas DataFrames 40 min teaching + 25 min exercises	2 h 40 min	My data has text, numbers and gaps in the same table. How do I hold that in Python? How do I pick out the rows and columns I care about, and compute something per group? How do I combine two tables, or reshape one from wide to long? What should I do about missing values, and when is dropping them wrong? What makes a table ‘tidy’, and why does every later chapter insist on it?
1.4Visualising data with matplotlib and seaborn 30 min teaching + 20 min exercises	3 h 30 min	Why should I draw a picture before I compute a statistic? What are a Figure and an Axes, and why does the course always write `fig, ax = plt.subplots()`? Which plot do I use for one variable, two variables, or a whole table of them?
1.5EDA case study: aqueous solubility 35 min teaching + 20 min exercises	4 h 25 min	What do I actually do with a new dataset before I model anything? What is logS, and why do chemists model the logarithm of solubility rather than solubility itself? Which of the descriptors in the ESOL file are related to solubility, and how strongly?
Lab 1Guided EDA: the breast cancer Wisconsin dataset 60–90 min	5 h 40 min	Run the EDA checklist from episode 1.5 on the breast cancer Wisconsin dataset, from shape and class balance to a feature ranking and a written summary, with checks after every task.

Lab

Lab 1, 60–90 min

Guided EDA: the breast cancer Wisconsin dataset

Run the EDA checklist from episode 1.5 on the breast cancer Wisconsin dataset, from shape and class balance to a feature ranking and a written summary, with checks after every task.

Dataset: Breast cancer Wisconsin (diagnostic), via scikit-learn; no download needed

Open in Colab Download .ipynb View solutions

Before you start

Prerequisite

You need a working Python environment and the habit of running code in a notebook. If you have not done so, work through Chapter 0 and make sure you pass its self-check: it confirms that NumPy, pandas, matplotlib and scikit-learn import correctly and that data/penguins.csv is where this chapter expects it.

For programmers

If you already write Python for a living, skim episode 1.1 for the scientific examples and the pathlib and f-string conventions this course uses, then start properly at episode 1.2. NumPy’s broadcasting rules and pandas’ label-based indexing are where experienced programmers most often trip, so slow down there.