08-11
What Is an LLM? Parameters, Sampling, and Training Costs
A beginner-friendly overview of large language models, framing a model as a compressed archive of 16-bit float parameters. It walks through inference, tokenization, common sampling strategies (greedy, top-k, top-p, temperature), and the two-phase training process, citing rough costs: 10TB of data, 6,000 GPUs for 12 days, around $2M, yielding a ~140GB parameter file. The author intentionally keeps things shallow and points to Karpathy's intro and Anthropic interpretability work. Experienced LLM engineers will find little new here; the piece is aimed at newcomers building their first mental model.