# topic/ai-ml/deep

Activation Functions

Activation functions introduce nonlinearity after the linear transformation in each neuron.

Autoregressive Generation

Autoregressive generation produces output one token at a time, feeding each generated token back as input for the next step.

Backpropagation

Backpropagation computes the gradient of the loss with respect to every parameter in the network by applying the Chain Rule (Multivariable) on the computation graph.

Batch Normalization

Batch normalization normalizes activations within each mini-batch to stabilize and accelerate training.

Causal Masking

Causal masking restricts Self-Attention so that each position can only attend to itself and earlier positions — never future tokens.

Chain-of-Thought Prompting

Chain-of-thought (CoT) prompting improves LLM reasoning by eliciting intermediate steps before the final answer.

Computational Complexity of Attention

Standard Self-Attention has time and memory in sequence length . This is the fundamental bottleneck of transformers.

Convolutional Neural Networks

CNNs exploit spatial structure through three key ideas: local connectivity, parameter sharing, and translation equivariance.

Deep Q-Network

DQN extends Q-Learning to high-dimensional state spaces by approximating with a neural network.

Direct Preference Optimization

DPO (Rafailov et al., 2023) trains directly on preference pairs without a separate reward model or RL loop.

Dropout

Dropout randomly sets each neuron's output to zero with probability during training:

Embeddings

An embedding maps discrete objects (words, tokens, users, items) into a continuous vector space where geometric relationships encode semantic relationships.

KL Penalty

The KL penalty in RLHF constrains the trained policy from drifting too far from the reference (SFT) model.

Layer Normalization

Layer normalization normalizes across the feature dimension for each individual sample:

LLMs as RL Agents

The framing of LLM generation as a reinforcement learning problem reveals deep connections between language modeling and sequential decision-making.

LoRA

LoRA (Low-Rank Adaptation) makes fine-tuning large models practical by training only small low-rank matrices.

Loss Functions

The loss function measures how wrong the model's predictions are. Training minimizes it via Backpropagation.

LSTM and GRU

LSTMs and GRUs solve the vanishing gradient problem of Recurrent Neural Networks through gating mechanisms.

Multi-Head Attention

Multi-head attention runs Self-Attention multiple times in parallel, each head learning different relationships.

Multimodality

Multimodal models extend a language model to consume (and sometimes produce) images, audio, or video by mapping each modality into the same embedding space the LLM already…

Neural Networks

Neural networks learn composed functions from data. The useful mental model is: layers apply differentiable transformations, nonlinearities make the composition expressive, and…

Positional Encoding

Transformers have no built-in notion of order — Self-Attention is permutation-equivariant. Positional encodings inject position information.

Pretraining

Pretraining teaches an LLM to predict the next token on massive text corpora — the foundational stage of the LLM pipeline.

ReAct

ReAct (Yao et al., 2023) interleaves reasoning and acting, enabling LLMs to use external tools.

Recurrent Neural Networks

RNNs process sequences by maintaining a hidden state that is updated at each time step:

Residual Connections

A residual (skip) connection adds the input of a layer to its output:

RLHF Pipeline

RLHF (Reinforcement Learning from Human Feedback) aligns LLMs with human preferences by training against a learned reward model.

Sampling and Decoding Strategies

At each step, an autoregressive language model outputs logits over the vocabulary. The decoding strategy determines how to pick the next token from this distribution.

Scaling Laws

Scaling laws describe how LLM performance (loss) improves predictably as you increase model size, data, and compute.

Self-Attention

Self-attention is the core mechanism of the transformer. It computes relationships between all pairs of positions in a sequence.

Self-Improvement in LLMs

Self-improvement methods allow LLMs to improve their own capabilities by generating and learning from their own outputs.

Softmax

Softmax converts a vector of raw scores (logits) into a probability distribution:

Supervised Fine-Tuning

SFT fine-tunes a pretrained LLM on instruction-response pairs, teaching it to be a helpful assistant instead of a text completer.

Test-Time Compute

Test-time compute refers to strategies that spend more computation at inference to improve output quality, rather than scaling the model itself.

The Artificial Neuron

The fundamental unit of a neural network:

Tokenization

Tokenization converts raw text into the integer sequences that transformers process.

Transformers and LLMs

Transformers and LLMs model token sequences by repeatedly mixing contextual information across positions. The useful mental model is: tokenization creates discrete symbols…

Weight Initialization

Wrong initialization causes vanishing or exploding gradients before training even begins.