Why We Still Need GPU Partitioning
Bigger models make GPU partitioning matter more.
LLM
From GPU to Token: The 8-Layer Observability Stack for AI Infrastructure
From GPU hardware, Kubernetes scheduling, inference engines to token cost — understanding the 8-layer observability architecture for modern AI infrastructure.
Kimi K2 Thinking: The True Awakening of China's Thinking Model
Kimi K2 Thinking’s open source marks China’s entry into thinking models. This article reviews its technical approach and compares it with Claude and Gemini.
Inference Traffic Control with Gateway API Inference Extension
Exploring how Gateway API Inference Extension brings model-aware inference traffic control through InferencePool, InferenceObjective, and metrics-driven routing.
Building Efficient LLM Inference with the Cloud Native Quartet: KServe, vLLM, llm-d, and WG Serving
Essential reading for cloud native and AI-native architects: how KServe, vLLM, llm-d, and WG Serving form the cloud native ‘quartet’ for large model inference, their roles, synergy, and …
From Kubernetes to Qwen: How "Open Source" Has Changed in the AI Era
Exploring the transformation of open source in the AI era, from Kubernetes to Qwen, and revealing the fundamental differences and new opportunities in open source strategies between China and the US.