Speculative Decoding Acceptance Rate & Speedup Calculator

Calculate the speedup from speculative decoding based on draft model acceptance rate and token speculation length. Optimize the draft model size vs acceptance rate tradeoff to maximize throughput for a given target model

Each component has a specific meaning:

• Speculative decoding based on draft model acceptance rate — The speculative decoding based on draft model acceptance rate recorded for the patient or scenario being assessed.
• Token speculation length — The token speculation length recorded for the patient or scenario being assessed.

Note: Interpret the speculative decoding result against the clinical thresholds and context described above.

Enter the speculative decoding based on draft model acceptance rate, token speculation length for the patient or scenario you are assessing. Calculate the speedup from speculative decoding based on draft model acceptance rate and token speculation length. Optimize the draft model size vs acceptance rate tradeoff to maximize throughput for a given target model. Use the speculative decoding result to inform your clinical assessment.