Inference batching Knowledge graph concept · also referred to as batching, batch size, batch, batched decoding Mentioned in Write-up of Reiner Pope's Lecture: How GPT, Claude, and Gemini Are Actually Trained and Served post · 2026-07-14 Theoretical Upper Bounds for LLM Performance post · 2026-07-14 Related concepts KV cache 2 LLM inference 2 Memory bandwidth 2 Roofline model 2 AI inference pricing 1 Memory capacity 1 Memory power 1 Mixture-of-experts model 1