← Home
Writing
Notes on systems, ML infrastructure, and things I find interesting.
Kimi Delta Attention: How It Works, Step by Step
Aug 17, 2026Kimi Delta Attention (KDA) is the core linear-attention mechanism introduced in Moonshot AI’s **Kimi Linear** architecture. Its goal is ambitious: keep the inference efficiency of recurrent or linear
Transformer Inference Math
Jul 15, 2026A systems-level explainer: where computation comes from, why attention has an N² term, why dense layers scale as d², and why prefill and decode behave very differently on accelerators.
transformersinferencesystems
Transformer Inference FLOPs Calculator
Jul 15, 2026Interactive calculator that separates linear-layer work from quadratic attention and shows prefill versus one-token decode.
transformersinteractivecalculator
KV Cache Calculator
Jul 15, 2026Live KV-cache memory sizing for MHA, GQA, and MQA with configurable precision and batch size.
transformersinteractivecalculator