SFT fine-tunes a pretrained LLM on instruction-response pairs, teaching it to be a helpful assistant instead of a text completer.
Training data: curated examples of (instruction, response) pairs:
- "Summarize this article: ..." → "The article discusses..."
- "Write a Python function that..." → "
python\ndef ..."
Objective: same cross-entropy loss as Pretraining, but only on the response tokens (instruction tokens are masked from the loss).
What SFT teaches:
- The "format" of being helpful — follow instructions, answer questions, refuse harmful requests
- Not new knowledge — that comes from pretraining data
- How to structure outputs (markdown, code blocks, step-by-step reasoning)
SFT alone is not enough:
- The model imitates the training data, including mistakes
- Can't distinguish between good and bad responses beyond what's in the data
- Tends to be verbose, sycophantic, or confidently wrong
- → Needs RLHF or DPO to learn from preferences
Efficient SFT:
- Full fine-tuning: update all parameters (expensive)
- LoRA: freeze base weights, train small low-rank adapters (much cheaper)
- Both used in practice depending on compute budget
In the pipeline: Pretraining → SFT → RLHF/DPO
See also: Pretraining, LoRA, RLHF Pipeline