
Speculative Decoding: The Inference Trick That Makes LLMs 3x Faster Without Sacrificing Quality
Speculative decoding cuts LLM inference latency by 2–3x by having a small draft model propose tokens that a large target model verifies in parallel. The output distribution is mathematically identical to the target model alone — here is how it works in production and where it is heading.







