Neural Networks: First Intuition¶
Why this matters¶
The full neural-network lesson uses terms like weights, bias, loss, gradient, learning rate, forward propagation, backpropagation, and chain rule.
This page gives the basic idea first:
You do not need calculus, matrices, or Python code yet. Start with the learning loop.
Mental model¶
Imagine a cafe wants to predict whether an order will be small or large.
| order | items ordered | drink size | includes food | real answer |
|---|---|---|---|---|
| A | 1 | small | no | small |
| B | 3 | large | yes | large |
| C | 2 | medium | yes | large |
| D | 1 | large | no | small |
The model starts with a rough rule. It might care too much about drink size and not enough about whether food is included.
Training means improving those internal settings after seeing examples.
Plain version:
Core ideas¶
- A feature is one input fact the model can use.
- A prediction is the model's answer for one example.
- Weights are adjustable settings that control how strongly features matter.
- A bias is an adjustable starting push before the features are considered.
- A loss is a wrongness score.
- Learning means changing weights and biases to reduce loss.
- A forward pass makes the prediction.
- A backward pass figures out which settings should change.
- Backpropagation is efficient blame assignment in a multi-layer network.
- A learning rate controls how big each adjustment is.
Walkthrough¶
Start with features and a prediction¶
For the cafe order example, the inputs are:
These inputs are features.
The model uses the features to make a prediction:
At first, the prediction may be bad. That is normal. Neural networks usually begin with poor settings and improve through repeated feedback.
Weights are knobs¶
A model can treat each feature as more or less important.
For example:
Those influence settings are weights.
If the model keeps predicting large whenever the drink is large, it may be giving the drink-size weight too much influence.
If the model ignores whether food is included, that weight may need more influence.
Bias is a starting push¶
The bias is another adjustable setting.
Think of it as the model's starting lean before it looks at the features.
For example:
That starting lean can be useful, but it should not overpower the actual features. Training adjusts the bias too.
Loss is the wrongness score¶
After a prediction, the model compares its answer with the real answer.
Example:
Another example:
Loss is useful because it gives the model a signal to improve from.
The goal is not just:
The goal is:
Learning is repeated adjustment¶
Training repeats the same basic loop many times:
1. Look at an example.
2. Make a prediction.
3. Measure the loss.
4. Adjust the weights and bias.
5. Try again.
The model does not understand cafe orders like a person does. It improves its settings because the loss tells it whether the current settings are working.
The gradient is a direction hint¶
The full lesson introduces gradients.
For now, read gradient as:
Since training wants lower loss, the model moves the other way.
Plain version:
You do not need the derivative formula yet. The important idea is direction.
Learning rate is step size¶
Once the model knows a direction, it still needs to decide how far to move.
That is the learning rate.
If the learning rate is too small:
If the learning rate is too large:
So learning rate is not a tiny detail. It controls whether the model adjusts steadily.
Forward pass and backward pass¶
Neural-network training has two directions.
The forward pass goes from inputs to prediction:
The backward pass goes from loss back toward the earlier settings:
The forward pass answers:
The backward pass answers:
Backpropagation is the backward pass method¶
In a tiny model, it is easy to imagine adjusting one or two knobs.
Neural networks can have many layers and many weights. A later mistake may depend on many earlier settings.
Backpropagation is the efficient way to work backward through those layers and find useful correction signals.
People sometimes describe this as assigning credit or blame.
Plain version:
How backpropagation follows influence paths¶
The word blame is only a shortcut. Backpropagation is really following paths of influence.
Imagine the model made this path during the forward pass:
But the real answer was:
Now the model knows the final answer was pushed too far toward large.
Backpropagation works backward through the same path:
output was too large
-> which output settings pushed it upward?
-> which hidden signals fed those settings?
-> which input weights made those hidden signals strong?
So the logic is not random guessing.
It is closer to tracing a chain:
If a setting had a strong path to the mistake, it gets a larger correction. If a setting barely affected the mistake, it gets a small correction or no useful correction.
Backpropagation does not update the weights by itself. It computes the information needed for the update.
Gradient descent uses that information to change the weights.
How this connects to digit recognition¶
The full lesson switches to handwritten digits.
There, the features are not cafe-order facts. They are pixel values.
The prediction is not small or large. It is one of ten digit classes:
But the training loop is the same:
That is why a simple cafe example can prepare you for the harder neural-network lesson.
Term Decoder¶
Use this table when reading the full lesson.
| Full lesson term | Read it as |
|---|---|
| feature | one input fact the model can use |
| prediction | the model's answer |
| weight | an adjustable influence knob |
| bias | an adjustable starting push |
| parameter | a trainable weight or bias |
| loss | wrongness score |
| gradient | direction hint for how loss changes |
| gradient descent | stepping toward lower loss |
| learning rate | step size |
| forward propagation | computing the prediction |
| backpropagation | efficient blame assignment |
| chain rule | the math rule that lets blame move backward through layers |
Common traps¶
Do not start with the formulas
The formulas explain the mechanism later. First, understand the loop: predict, measure loss, adjust, repeat.
Do not think weights are fixed rules
Weights are learned settings. Training changes them.
Do not treat loss and accuracy as the same thing
Loss is the wrongness signal used for updates. Accuracy is a human-readable count of correct predictions.
Do not think backpropagation is the whole training process
Backpropagation computes gradient information. Training also needs predictions, loss calculation, and weight updates.
Do not treat larger steps as always better
A large learning rate can overshoot useful settings.
Check yourself¶
What is the basic neural-network training loop?
Make a prediction, measure the loss, adjust the weights and biases, and repeat.
In the cafe example, what are the features?
Inputs such as items ordered, drink size, and whether the order includes food.
What does a weight control?
How strongly one input influences the model's later computation.
Why does the model need a loss?
Loss tells the model how wrong the prediction was, which gives it a signal for improvement.
What is the learning rate?
The step size used when adjusting weights and biases.
What is the difference between a forward pass and a backward pass?
The forward pass computes the prediction. The backward pass works from the loss back through the network to find which settings should change.
Why is backpropagation useful in a neural network?
It efficiently assigns credit or blame through many layers and many weights.
Now Read the Full Lesson¶
Next: How Neural Networks Learn
When you read the full lesson, translate the main technical terms back to these ideas:
weights and biases -> adjustable knobs
loss -> wrongness score
gradient -> direction hint
learning rate -> step size
forward pass -> prediction direction
backpropagation -> blame-assignment direction
The full lesson adds perceptron, ADALINE, gradient descent, chain rule, and the digit-recognition example.
Source anchors¶
- Supports:
study-guide/docs/lessons/09a-how-nns-learn.md - Source file:
notebooks/Module2/09a-How NNs Learn.pdf - Key source concepts prepared here: features, weights, bias, loss, gradient, gradient descent, learning rate, forward propagation, backpropagation, chain rule, digit-recognition inputs