# topic/cs

Approximate Nearest Neighbor Search

Exact nearest neighbor search in high dimensions is per query (brute-force scan). For millions of vectors, this is too slow. Approximate methods trade a small accuracy loss for…

Big-O and Complexity Analysis

Big-O notation describes how an algorithm's time or space scales with input size , ignoring constants and lower-order terms.

Computation Graphs

A computation graph is a directed acyclic graph (DAG) where nodes are operations and edges carry values. It makes the Chain Rule (Multivariable) systematic.

Computational Complexity of Attention

Standard Self-Attention has time and memory in sequence length . This is the fundamental bottleneck of transformers.

Computer Science Foundations

Computer science foundations study the cost and structure of computation. The useful mental model is: algorithms transform inputs into outputs, data structures control access…

Distributed Training Strategies

When a model or dataset is too large for a single GPU, training must be distributed across multiple devices.

Dynamic Programming

Dynamic programming (DP) solves problems with optimal substructure (optimal solution built from optimal sub-solutions) and overlapping subproblems (same subproblems recur) by…

Floating Point and Quantization

Numbers in hardware have finite precision. Choosing the right format trades off range, precision, memory, and speed.

GPU Architecture and CUDA

GPUs achieve massive parallelism through thousands of simple cores executing the same instruction on different data (SIMT — Single Instruction, Multiple Threads).

Graphs and Traversals

A graph consists of vertices and edges. Directed graphs (digraphs) have ordered edges. A DAG (directed acyclic graph) has no cycles.

Hash Tables

A hash table maps keys to values via a hash function , giving average-case lookup, insert, and delete.

Memory Hierarchy and IO-Awareness

Modern hardware is memory-bound, not compute-bound for most ML operations. Understanding the memory hierarchy is the key to writing fast code.

P vs NP and Intractability

P = problems solvable in polynomial time. NP = problems whose solutions are verifiable in polynomial time. The question is open, but widely believed to be .

Randomized Algorithms

Randomized algorithms use random choices to achieve better average-case performance, simpler implementations, or solutions to problems where deterministic approaches are…

Sorting and Selection

Sorting arranges elements in order. Selection finds the -th smallest (or largest) element without fully sorting.

Systems and Scaling

Systems and scaling study how model training and inference behave on real hardware. The useful mental model is: performance is limited by compute, memory, communication, and…

Tokenization

Tokenization converts raw text into the integer sequences that transformers process.