LLM Speculative Decoding Speedup & Draft Model Efficiency Calculator
Speculative decoding speedup is calculated as: Speedup = 1 + E[# accepted tokens] / (1 + Cd/Cv), where Cd is drafting cost and Cv is verification cost. Speedup depends critically on acceptance rate (α ≥ 0.6 recommended for 2x+ gains), draft model efficiency, and batch size. Gains diminish at high batch sizes where GPU utilization is saturated. Small models (<3B) often experience slowdowns due to high relative drafting overhead.