From Rules to Recall

1.1 The definition

That still leaves a question: mimic what, exactly? The textbook (Russell and Norvig) splits the answers into four goals, and the slide draws them as four cards.

Thinking like a human is hard to study, because we barely know how we think. Acting exactly like a human isn't even what you want from a calculator. So most of computer science settled on the fourth card, acting rationally: given what you can see, do whatever gets the best outcome, or the best expected outcome when you can't be sure.

That definition is more useful than it sounds, because "best outcome" is something you can measure. Chapter 7 is built on that.

1.2 Where you've already met it

All six of these do one specific job well. None of them can do another one's job. Keep that in mind for section 1.5.

Once a system acts in the world, "the best outcome" stops being a purely technical question. The next slide makes that uncomfortable.

1.3 Intelligent agents

An intelligent agent is anything that perceives its environment and acts to reach a goal. It's the frame the whole field uses.

Notice "past experiences" feeding into the box in the picture. An agent that improves from its own experience is exactly what machine learning builds.

1.4 Fields of AI

1.5 Narrow, general, super

• Artificial narrow intelligence does one kind of task. Everything in section 1.2 is this.
• Artificial general intelligence would handle any intellectual task a person can. It doesn't exist yet, and people argue about how close we are.
• Artificial super intelligence would beat humans at everything. It's hypothetical.

1.6 The AI lab: questions worth having an opinion on

2.1 Can a machine learn?

Normal programs do what they're told. Machine learning is what you reach for when you can't say what to tell them. It sits inside AI, and deep learning sits inside it.

There are two classic definitions. The first is the idea and the second is the test.

• Arthur Samuel, 1959: the field of study that gives computers the ability to learn without being explicitly programmed.
• Tom Mitchell, 1998: a program learns from experience \(E\) with respect to task \(T\) and performance measure \(P\) if its performance on \(T\), measured by \(P\), improves with \(E\).

Mitchell's version is great for exams because you can fill it in. For a spam filter: T is classifying emails as spam or not. E is watching you label emails. P is the fraction it gets right.

2.2 Why machine learning?

The slide puts two problems side by side, and it's the most important picture in this chapter.

So we flip the job around. Instead of writing \(h(x)\) ourselves, we hand over examples of inputs and correct outputs, and an algorithm finds \(h(x)\) for us.

2.3 Supervised learning

In supervised learning every training example comes with the right answer, the label, like a student working with a teacher.

There are two kinds of task, and the only difference is the type of answer you want.

Regression: a number

Classification: a category

2.4 Unsupervised learning

In unsupervised learning there are no labels. Nobody says what the groups are, and the algorithm finds structure on its own.

A catch the slides show nicely: "similar" depends on what you measure. The same basket of fruit groups differently by colour, by shape or by size. The algorithm can't tell you which grouping is the useful one. You decide that from the business problem.

The other unsupervised task is dimensionality reduction: fewer columns, as little lost as possible. It matters because data gets lonely fast. The slide's picture shows 10 positions in one dimension, 100 in two, and 1,000 in three, so the same number of examples covers less and less of the space.

2.5 Semi-supervised and reinforcement learning

Semi-supervised learning uses a few labels and many unlabelled examples. It exists because labelling costs real money and time: unlabelled scans are cheap, but a radiologist's hour isn't.

Reinforcement learning has no labels either. An agent tries actions in an environment and gets rewards or penalties. It's the agent loop from Chapter 1 with a score attached.

2.6 Sort the problems

The slides ask you to list your own examples of each type. Warm up here first. The test is always: is there a right answer in the data, and what shape does the answer take?

2.7 The machine learning life cycle

Students often think ML means training. Training is one of eight steps, and in a real project it isn't where most of the time goes.

Press Run and follow one project all the way round: a spam filter for a company's email.

2.8 As demonstrated in the Teachable Machine lab

3.1 NumPy: why not just use a list?

Because a Python list is slow, and machine learning does the same operation on millions of numbers. The notebook's first test multiplies a million numbers by 2:

my_arr  = np.arange(1_000_000)
my_list = list(range(1_000_000))
%timeit my_arr2  = my_arr * 2
%timeit my_list2 = [x * 2 for x in my_list]

NumPy:    4.54 ms per loop
Python: 238    ms per loop

Three ideas cover most of what you'll use. Arithmetic is element-wise, a 2-D array is a table with a .shape, and slicing picks out rows and columns.

a = np.array([1, 2, 3]);  b = np.array([9, 8, 7])
a + b                     # [10 10 10], no loop needed
a * b                     # [ 9 16 21]
A = np.array([[1,2,3],[4,5,6],[7,8,9]])
A.shape                   # (3, 3): rows, columns
A[0, :]                   # first row
A[:, 0]                   # first column

You'll meet .shape again right away. Scikit-learn insists on a 2-D input, which is why the labs keep writing X_train.reshape(-1, 1).

3.2 Pandas: your data's home

Real data has column names and mixed types, and it has holes in it. A DataFrame handles all of that. The notebook's picture says it best: a Series is one labelled column, and a DataFrame is several of them side by side.

movies_df = pd.read_csv("IMDB-Movie-Data.csv", index_col="Title")
movies_df.head()             # look at it
movies_df.info()             # types and non-null counts
movies_df.shape              # (1000, 11)
movies_df.isnull().sum()     # where are the holes?
movies_df.describe()         # mean, std, quartiles

Revenue (Millions)    128 missing
Metascore              64 missing
Rating: mean 6.72, std 0.95, min 1.9, max 9.0

3.3 Try a DataFrame

This small table comes from the preprocessing notebook, and we'll clean the same table in Chapter 5. Click a line of pandas and see which cells it touches.

4.1 Where it came from

The odd name has a story. In 1877 Francis Galton, Charles Darwin's cousin, measured pea seeds against the seeds of their parents. Then he did the same with human heights.

Tall parents tended to have tall children, but not as tall. Short parents had short children, but not as short. Heights were "regressing" toward the average, and the name stuck to the method even though the method now does far more.

4.2 When to use it

Linear regression predicts a number when the input and the output have a roughly straight-line relationship. So look at the data before you fit anything.

4.3 The model

The slides price land. You know the area of a plot and want its price.

The notation is short, and you'll see it everywhere:

Training turns those examples into a hypothesis \(h\), and \(h\) is the thing you actually use afterwards.

For one input, \(h\) is a straight line:

\(a\) is the weight: how much the price rises per extra m². \(b\) is the bias: where the line starts. Together they're the model's parameters, and training is nothing more than choosing them.

With more than one input, the same idea is called multiple linear regression: \(h(x) = w_1x_1 + w_2x_2 + \dots + b\). A price really does depend on size and location and age.

4.4 The cost function: scoring a line

To pick the best line we need one number that says how bad a line is. For each example, take the gap between the prediction and the truth.

Average the squared gaps. Squaring makes every gap positive, so misses in opposite directions can't cancel out, and it makes big misses count far more than small ones.

Now you be the learning algorithm. Move the sliders and try to get \(J\) as low as you can.

The lab plots what you just did, for every possible \(w\) and \(b\). The surface is a bowl: one bottom, and every direction from anywhere leads down to it.

4.5 Gradient descent: walking downhill

You found a good line by nudging sliders and watching the number. Gradient descent is that same nudging as an algorithm. Start anywhere, work out which direction is downhill, take a small step, and repeat.

\(\eta\) is the learning rate, your step size. The derivative is the slope under your feet. A positive slope means downhill is to the left, so the minus sign moves you left, and a negative slope moves you right. The lab's picture shows exactly that.

For our line, the slopes work out to:

Here \(w_1\) is the slope \(a\) and \(w_0\) is the bias \(b\). As a flowchart, the loop is:

The slides then show it running. Step through the frames: on the left the bowl with the current position, on the right the line that position stands for.

←→

One caution from the slides: a real cost surface isn't always a clean bowl. On a bumpy one, gradient descent can settle into a local minimum, depending on where it starts. Linear regression's squared error has a single minimum, so that can't happen here. It will matter in Chapter 6.

4.6 The learning rate

Step size is the one setting you have to choose yourself. Run all three below and compare them. The left chart shows the line being fitted. The right one shows the same run as a ball on the bowl.

4.7 As demonstrated in the labs

The labs use the smallest dataset possible: two houses. A 1,000 sqft house sold for $300k and a 2,000 sqft house for $500k. Two points fix one line, so you can check every answer by hand: \(w = 200\), \(b = 100\).

def compute_gradient(x, y, w, b):
    m = x.shape[0]
    dj_dw, dj_db = 0, 0
    for i in range(m):
        f_wb = w * x[i] + b
        dj_dw += (f_wb - y[i]) * x[i]
        dj_db += (f_wb - y[i])
    return dj_dw / m, dj_db / m

Iteration    0: Cost 7.93e+04   w: 6.500e+00, b: 4.00000e+00
Iteration 1000: Cost 3.41e+00   w: 1.949e+02, b: 1.08228e+02
Iteration 9000: Cost 2.90e-05   w: 2.000e+02, b: 1.00024e+02
(w,b) found by gradient descent: (199.9929, 100.0116)
1200 sqft house prediction 340.0 Thousand dollars

linear_model = LinearRegression()
linear_model.fit(X_train.reshape(-1, 1), y_train)
linear_model.coef_, linear_model.intercept_

w = [200.], b = 100.00                                  # the two houses
w = [0.27 -32.9 -67.29 -1.47], b = 221.50               # houses.txt, four features
predicted price (1200 sqft, 3 bed, 1 floor, 40 yrs) = $318936.36

5.1 Types of data and variables

First, know what you're holding, because the type decides which operations even make sense.

What you can do with each type:

The type also decides which chart to draw:

And a pair of words you already know under other names: the independent variable is your input \(x\), and the dependent variable is your output \(y\). It depends on \(x\), hence the name.

5.2 Cleaning: low variation and missing values

Low variation. A column where every row says the same thing can't tell rows apart. It carries no information, so drop it. The notebook drops Marital_Status because its only value is "Single".

Missing values. These don't always look missing. They might be NA, n/a, an empty string, ?, or worse, a number like -1 or -999.

You have two moves: delete the rows (fine if you have plenty of data) or impute, which means fill them in. Numbers get the mean, median or mode. Categories get the mode. A rule of thumb from the slides (credited to Vishal Patel): impute if under 50% is missing, and throw the feature away above 95%.

Mean or median? Try both. The Age tab shows why the choice matters.

df.isnull().sum()                     # Sex 1, Salary 1, Education 2
imp = SimpleImputer(strategy="median")
df[numeric_cols] = imp.fit_transform(df[numeric_cols])

5.3 Outliers

An outlier sits an abnormal distance from everything else. It might be a typo, a broken sensor, or data mixed in from the wrong source. It might also be real, and that's the hard part. Maths can flag a value, but a person who knows the domain has to decide.

Method 1: Z-score

How many standard deviations from the mean is it? For bell-shaped data, about 99.7% of values fall within ±3, so anything beyond that is suspicious.

\[ z = \frac{x-\bar x}{\sigma} \]

Method 2: IQR

Sort the data and cut it into quarters. The middle half has width \(IQR = Q_3 - Q_1\). Anything further than \(1.5 \times IQR\) past the box is flagged. It's what a boxplot draws.

\[ x < Q_1 - 1.5\,IQR \ \text{ or }\ x > Q_3 + 1.5\,IQR \]

Run the slide's two examples, then type your own numbers.

In the lab

Data preprocessing notebook · removing Age = 200

Before and after removing the one outlier. On the left, everyone else is squashed into a sliver. On the right you can finally see the real spread.

5.4 Imbalanced data

In many real problems the thing you're looking for is rare. The model can then score well by ignoring it.

Real world

A well-known credit card dataset has 284,807 transactions and only 492 frauds, or 0.17%. A "model" that says "not fraud" every time is 99.83% accurate and catches nothing. That's why these methods exist, and why Chapter 7 won't let you trust accuracy alone.

Both have a cost. Undersampling throws away real data, and oversampling teaches the model the same few examples again and again, which invites overfitting. SMOTE is the middle road: it makes new minority examples instead of copies, by placing them on the line between a minority point and one of its nearest minority neighbours.

5.5 Feature scaling

To an algorithm, numbers don't have units. House size runs into the thousands and bedrooms stop around 4. So when an algorithm measures distance, size drowns out bedrooms completely. Try it: which house is most like house A?

5.6 Feature encoding

Algorithms do arithmetic, so words have to become numbers. The trap is that the numbers you pick can invent a meaning that was never there.

small, medium, large → 0, 1, 2

5.7 Feature correlation

Pearson's \(r\) measures how close two numeric columns are to a straight-line relationship, from −1 through 0 to +1. Two strongly correlated inputs tell the model nearly the same thing, so you can often drop one of them.

The slide's example is a shop's daily temperature against sales. Run all five steps of the calculation. Then press Add a strange day and see what one row does to \(r\).

In the lab

Data preprocessing notebook · heatmap

pearson_corr = df.corr(method='pearson', numeric_only=True)
sns.heatmap(pearson_corr, cmap='PiYG', annot=True, vmin=-1, vmax=1, center=0)

5.8 The Delhi houses lab

6.1 From a number to a decision

Chapter 4 predicted a price. A lot of real decisions aren't numbers, though. Spam or not? Fraud or not? Which digit is this? Answers from a fixed set of options are classification.

Logistic regression is the standard first tool. Despite the name it does classification. It works in two moves: estimate a probability between 0 and 1, then compare it to a threshold to give a yes or a no.

6.2 Why not just use a straight line?

The slides try the obvious thing first. Predict obesity from weight: label "not obese" as 0 and "obese" as 1, fit a line, and call it obese wherever the line passes 0.5. It works, until you add a few very heavy people, who are clearly obese and should change nothing. Press the button and watch the threshold.

The line tilts toward the extreme examples, so its 0.5 crossing slides right and real obese people get called not obese. Those are the red crosses on the slide:

6.3 The sigmoid

We need a function that takes any number and squeezes it between 0 and 1. That's the sigmoid, or logistic function.

So logistic regression keeps the weighted sum from linear regression and runs it through the S:

6.4 The decision boundary

Use a threshold of 0.5. The sigmoid crosses 0.5 exactly when \(\mathbf w\mathbf x = 0\), and that gives a neat result: the line \(\mathbf w\mathbf x = 0\) splits the plane in two. One side is class 1 and the other is class 0.

Press Run and watch gradient descent find that line by itself. It starts out wrong, and each step rotates and shifts it to reduce the loss.

6.5 Cost: why cross-entropy instead of squared error

Squared error worked beautifully in Chapter 4. Put a sigmoid inside it, though, and the bowl turns bumpy, so gradient descent can get stuck in a local dip.

So we swap the cost for one that stays bowl-shaped and also matches what we want. It's called cross-entropy, or negative log-likelihood:

It looks long, but for any one example only one half is alive. If the truth is 1, the cost is \(-\log(p)\). If the truth is 0, it's \(-\log(1-p)\).

Try it

What does a mistake cost?

Truth: spam (y = 1)Truth: not spam (y = 0)

Model says p =

Gradient descent then runs as before. The derivative even comes out in the same form as linear regression's, with the sigmoid hidden inside \(h\):

6.6 More than two classes

Logistic regression is yes-or-no. What about three iris species, or ten digits? There are two ways.

One vs. rest

Train one yes-or-no classifier per class: "is it class 1, or anything else?", "is it class 2, or anything else?" and so on. Then ask them all, and the most confident one wins.

←→

Softmax regression

Or do it all in one model. Compute a raw score for each class, then turn the scores into probabilities that add up to 1:

6.7 Pros and cons

7.1 Learning phase and prediction phase

A model has two lives: first it learns from data you have, then it's used on data you don't have yet.

7.2 Batch, iteration, epoch, and hyperparameters

7.3 Splitting the data

Grading a model on its own training data is like marking an exam after handing out the answers beforehand. So before training starts, we hide some data away.

• Training set: learns the weights.
• Validation set: compares models and tunes hyperparameters (optional).
• Test set: gives one final, honest grade. It must play no part in learning.

A random split has a weakness: your score depends on which rows happened to land in the test set. A sample that doesn't represent the population is called selection bias.

7.4 K-fold cross-validation

Why trust one split? Split into \(k\) parts and give each part a turn as the test set. Then average the scores.

This demo really trains a line on each fold, using 30 noisy points. Look at how much the score moves between folds. That spread is what a single split would have hidden from you.

The same trick tunes hyperparameters: for each candidate setting, run k-fold, then keep the setting with the best average. Taken to the limit, \(k = n\) is called leave-one-out.

7.5 Underfitting and overfitting

• Underfitting: bad on training data too. The model is too simple to catch the pattern.
• Overfitting: great on training data and bad on new data. The model memorised the noise.

Drag the slider and watch what happens. A higher degree means a more flexible curve. Training error keeps falling no matter what, but test error falls and then shoots back up. Where they part ways is overfitting.

Real world

Google Flu Trends tried to predict flu from search terms, out of 50 million candidates. It looked excellent on past years, partly because some terms just happened to match the winter pattern. In the 2012–13 season it predicted more than double the actual doctor visits for flu (Science, 2014), and it was later retired. Lots of flexibility plus a limited number of seasons is the same setup as your slider at degree 9.

Bias and variance

These are the same two failures seen from the statistics side. Bias is being consistently off target. Variance is being inconsistent from one sample to another. The dartboard makes it obvious.

Learning curves

Is it worth collecting more data? Plot training and test error against the training set's size.

7.6 Regression metrics

The slides work one set of five points. Run it, then turn on the big miss and compare how MSE and MAE react.

On the slide's numbers you should get MSE = 6.08, MAE = 2.16 and R² = 0.1739. Slide 28 labels its MAE calculation "MSE", but the sum there is MAE.

7.7 The confusion matrix

For classification, the question isn't only "how often is it right?" but "what kind of wrong?". The confusion matrix sorts every prediction into four boxes.

The trade-off you can't escape

A classifier gives a probability, and you choose the threshold. Slide it and watch precision and recall pull against each other.