Real speculative decoding — running in your browser

distilgpt2 drafts γ tokens, gpt2 verifies them — quantized ONNX models executed client-side with transformers.js (WebGPU when available, WASM otherwise). No server, no GPU quota, no cold start: the rejection sampling below is the actual algorithm on actual model logits. ← back to the interactive explainer

Everything stays on your device — the models are fetched from the Hugging Face CDN and executed locally. Source: GitHub. The repo also ships a Gradio app (with KV caching, attention-trace diffing and a 70B cloud cross-check) you can run locally with python app.py.