Two-thirds of a transformer block’s parameters sit in the feed-forward path. Here is how SwiGLU gates them, why the 8/3 width rule keeps compute flat, and where it fails.
SwiGLU Explained: Why Modern LLMs Replaced GELU in the Feed-Forward Block Read More »
Attention is limited by memory bandwidth, not arithmetic. Here is how tiling, online softmax and recomputation make FlashAttention exact and fast.
Why Attention Is Memory-Bound: Inside FlashAttention and IO-Aware Kernels Read More »
How reasoning models are trained with verifiable rewards and GRPO, why the critic network disappeared, and what can still go wrong.
How Reasoning LLMs Are Trained: Verifiable Rewards and GRPO Read More »
How constrained decoding makes a language model return valid JSON every time, and what it still cannot guarantee in a business process.
How LLMs Are Forced to Return Valid JSON: Constrained Decoding Explained Read More »
How quantization shrinks large language models to fit on fewer GPUs, makes them faster, and makes fine-tuning on your own data practical.
LLM Quantization Explained: Running Bigger Models on Fewer GPUs Read More »
The KV cache, PagedAttention and speculative decoding decide how fast a model answers and how many users your own GPUs can serve.
Serving LLMs on Your Own GPUs: KV Cache, PagedAttention and Speculative Decoding Read More »
Why the newest long-context language models keep full attention in only a few layers, and what it means for teams training models on their own GPUs.
Hybrid Attention: Why 2026’s Long-Context LLMs Use Less Full Attention Read More »
