Neural Networks and Deep Learning

After this lesson, you will be able to: Understand neural networks, backpropagation, and what 'deep' learning actually means.

A neural network is a stack of matrix multiplications and non-linearities. That's it. The magic is backpropagation, how it adjusts millions of parameters to fit data. This lesson builds the mental model.

Prerequisites:Machine Learning Fundamentals

The neuron

Inputs × weights, summed, plus bias, run through an activation function (ReLU, sigmoid). Stack neurons in layers. Connect layers. That's a neural network.

How a network learns

1
1. Forward pass, input → output (a guess).
2
2. Loss, how wrong was the guess?
3
3. Backpropagation, gradient of loss w.r.t. each weight.
4
4. Gradient descent, nudge weights to reduce loss.
5
5. Repeat over millions of examples.

A neural net in PyTorch

Tiny network, real training loop:

python

import torch
import torch.nn as nn

model = nn.Sequential(
    nn.Linear(4, 16),
    nn.ReLU(),
    nn.Linear(16, 3),
)

loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

for epoch in range(100):
    optimizer.zero_grad()
    outputs = model(X_train)
    loss = loss_fn(outputs, y_train)
    loss.backward()  # backprop
    optimizer.step()  # gradient descent step

Architecture families

CNN (Convolutional), vision. Filters scan local regions. Used in image classifiers. RNN/LSTM, sequence. Pre-2017 standard for language. Mostly retired now. Transformer, attention-based. Powers all modern LLMs (ai-04 dives deep).

←Machine Learning Fundamentals

Back to Artificial Intelligence

What are Large Language Models?→