rAs Kids🟢🟢 ╱ 🔴🔴 ❓
🎒 What you need
- your plotted data from module 1, or chalk to redraw it
- stones in two colours, or stones and bottle caps
- a long straight stick or string
✋ Do this
- Redraw your two axes big, on the ground. Place a stone for every good grain and a cap for every damaged one, at its measured position.
- Lay your stick down as a straight line that best separates stones from caps. Move it until you cannot do better.
- Count the mistakes: stones on the wrong side, caps on the wrong side. Write that number down. A perfect split is rare and slightly suspicious.
- Take a NEW grain, measure it, place it. Which side of the stick? That is your prediction. Now check the real answer.
- Second method: for that same new grain, find its three nearest neighbours on the ground and take a vote. Did the two methods agree?
- Now curve your line using string instead of a stick. Can you get zero mistakes? And do you trust a line that twists around every single dot?
💡 Why it works
You have just built two real classifiers by hand. The straight stick is a linear classifier; the vote of nearest points is k-nearest-neighbours, still used in production today. And the twisting string is overfitting: a boundary so contorted to fit your exact examples that it will do badly on the next grain it meets. That tension between fitting your data and generalising to new data is the central problem in machine learning.
🔥 Challenge
Split your data in half. Fit your line using only the first half, then test it on the second half you never looked at. Report both error counts. That gap is what the field really cares about.
📖 New words
classifiera system that assigns a new example to one of several classes
decision boundarythe line or surface separating one class's region from another's
overfittingmatching your training examples so exactly that you fail on new ones
📏
Boundary Drawer
Tap when you have finished this module.