inference-learnUnderstand LLM Inference Engines from Scratch

Load Qwen2.5-0.5B with ~2700 lines of pure C. No PyTorch, just libc. Every component written from scratch.

⚡

Pure C, Zero Dependencies

No PyTorch, no AI frameworks, just libc. ~2700 lines of code, every operator hand-written and readable.

🧠

RMSNorm, RoPE rotary positional encoding, GQA grouped attention, SwiGLU gated activation, KV Cache — each one broken down from math to code.

🔢

Forward pass results compared against the official PyTorch implementation. Error < 0.0002. Greedy generation matches HF token-by-token.

📖

From "what is an inference engine" to "how does it compare to vLLM" — 6700+ lines of docs with 13 Mermaid diagrams.

🔍

Tiered logging system + HTML visualization reports. Add -v for per-layer summaries, --report for a foldable execution trace.

🌍

Detailed comparison with llama.cpp / vLLM / SGLang. Understand what your engine is missing and why.