Jimmy Song – Jimmy Song's Blog

Core Model Overview

Jimmy Song — Tue, 10 Feb 2026 13:56:12 +0000

The Yin-Yang - Five Elements - Yun - Qi Model views AI Infrastructure as an organic whole, revealing its operational mechanisms from four dimensions. Each layer focuses on different fundamental questions:

Four-Layer Model

Layer	Name	Focus Question
Yin-Yang	State Layer	The system’s internal unity of opposites tension structure, revealing how dual elements like performance vs. constraints, innovation vs. governance coexist
Five Elements	Role Layer	Five basic role elements in the system and their collaborative relationships, breaking down complex infrastructure into data, models, compute, platforms, and hardware
Yun	Time Layer	The development stage the system is in and its cyclical patterns, describing the evolution cycle from exploration to platformization, then scaling and rebalancing
Qi	Flow Layer	The effective “field” of flow within the system, characterizing the conduction and feedback of signals and resources, reflecting the overall smoothness of operation

Table 1: Four-Layer Model

Model Interactions

The four-layer model is not isolated but an interconnected organic whole:

The tension of Yin-Yang permeates the dynamic balance of Five Elements
The development of Five Elements roles is constrained by their Yun stage
The flow of Qi connects the above elements into a self-adaptive cyclic system

The overview diagram below illustrates each layer of the model and their interactions:

Figure 1: AI Infrastructure ‘Yin-Yang - Five Elements - Yun - Qi’ Model Overview. The Yin-Yang layer embodies the system’s internal tension and unity of opposites, the Five Elements layer defines core role elements, the Yun layer describes system stage cycles, and Qi as a flow element permeates and drives the entire system.

Model Application Value

This four-layer model provides a unique perspective for the design, operations, and governance of AI Infrastructure:

Holistic Cognitive Framework: Transcend the limitations of single technical metrics to grasp system state as a whole
Dynamic Balance Thinking: Understand unity of opposites relationships and avoid extremes
Evolutionary Stage Awareness: Grasp the system’s development stage and act in accordance with the situation
Flow Insights: Focus on energy flow within the system to anticipate problems

Next, we will delve into the connotation, engineering mapping, and mechanism of each layer.

The Yin-Yang Layer: Dynamic Balance of System States

Jimmy Song — Tue, 10 Feb 2026 13:56:33 +0000

Yin-Yang is originally a fundamental concept in Chinese philosophy, representing two opposing yet interdependent forces present in all things in the universe. Everything in the world can be classified as either Yin or Yang, and their continuous movement and change generate the various transformations we observe. In the context of systems, Yin-Yang represents the unity of opposites through tension—a pair of attributes or tendencies that pull against yet depend on each other.

Three Typical Pairs of Yin-Yang Tensions

In AI infrastructure, we identify three typical pairs of Yin-Yang tensions:

Expansion ↔ Constraint

Expansion ↔ Constraint: The tension between growth trends and limiting forces.

Yang (Expansion): System expansion speed, such as continuously adding tasks and scaling resources
Yin (Constraint): Limiting forces, such as cost controls, regulatory constraints, and hardware limits

System expansion speed and constraint intensity always coexist. For example, continuously adding tasks and scaling resources in GPU clusters (the Yang of expansion) is constrained by costs, regulations, or hardware limits (the Yin of constraint).

Imbalance manifestations:

Pursuing expansion without regard for constraints → Resource contention and crashes
Excessive constraint → Stifling system vitality

Innovation ↔ Governance

Innovation ↔ Governance: The tension between creative capability and control requirements.

Yang (Innovation): Technical innovation, introduction of new features
Yin (Governance): Security reviews, rule-making

The faster technical innovation progresses, the more easily governance gaps are exposed. For example, introducing new Agent features (innovation, Yang) may outpace security reviews and rule-making (governance, Yin), leading to potential risks.

Imbalance manifestations:

Innovation outpaces governance → Potential security risks
Excessively strict governance → Slowing innovation momentum

Speed ↔ Stability

Speed ↔ Stability: The tension between performance advancement and reliable operation.

Yang (Speed): Performance improvements, increased throughput
Yin (Stability): Reliable operation, system stability

When we pursue speed improvements single-mindedly, the cost to stability will eventually manifest. For example, pushing GPU utilization to the limit during model training (speed, Yang) easily leads to more frequent failures or delays (decline in stability, Yin).

Imbalance manifestations:

Extreme pursuit of speed → Decline in stability
Excessive conservatism → Performance waste

The Art of Yin–Yang Balance

The Yin–Yang poles described above are not simple trade-offs where you choose one and sacrifice the other, but rather inherent relationships of unity of opposites in systems. Both Yin and Yang sides are opposed yet complementary, neither can be dispensed with:

Expansion without constraints is difficult to sustain, constraints without expansion lose meaning

As the ancient saying goes, “One Yin and one Yang constitute the Way” (一阴一阳之谓道). Balancing Yin and Yang is the “Way” of healthy system operation. For architects, the key lies in:

Insight into dominant tensions: Determine which pair of tensions is currently dominant
Introducing the opposite: Introduce the complementary side at the right time to restore balance
Dynamic adjustment: Dynamically transform based on changes in system environment and stage

Practical Cases

Case: GPU Cluster Expansion

When the cluster is in a state of rapid expansion (Yang exuberant, Yin deficient):

✓ Add scheduling policies and resource quotas (supplement Yin)
✓ Establish cost control mechanisms (supplement Yin)
✗ Do not pursue expansion speed single-mindedly

Case: Agent Feature Innovation

When introducing new Agent features:

✓ Simultaneously establish monitoring and sandboxing mechanisms (supplement Yin)
✓ Improve security review processes (supplement Yin)
✗ Do not let innovation outpace governance

Case: Model Training Performance Optimization

When optimizing model training performance:

✓ Simultaneously strengthen fault tolerance mechanisms and testing (supplement the Yin of stability)
✓ Set performance baselines and rollback mechanisms (supplement Yin)
✗ Do not infinitely compress fault tolerance time

Dynamic Transformation of Yin–Yang States

It’s important to note that Yin–Yang states are not static and unchanging, but dynamically transform with system environment and stage.

The same capability may transform from an advantage to a risk at different stages

For example, a “rapid development” strategy that drives rapid iteration during the startup stage, if applied without restraint during the scaling stage, can instead become a major threat to stability.

The analysis of the Yin–Yang layer reminds us to constantly pay attention to the ebb and flow of these opposing forces, and to keep the system in a state of elastic tension through adjustments, rather than snapping or becoming slack and ineffective.

Five Elements Layer: Classification and Collaboration of System Roles

Jimmy Song — Tue, 10 Feb 2026 13:56:06 +0000

Five Elements (Wǔxíng, Five Elements or Five Phases) theory divides everything in the world into five basic elements: Wood, Fire, Earth, Metal, Water. Each element represents a fundamental attribute or functional role, with the five elements generating and overcoming each other in an endless cycle.

In AI infrastructure, we use “Five Elements” to characterize the system’s five core elements and their responsibilities:

Engineering Mapping of Five Elements

Five Elements	Symbol	Meaning	Engineering Correspondence
Water	🌊	Flow and containment	Data flow and quality: data pipelines, data assets, and quality control
Wood	🌲	Growth and creation	Model growth and capability expansion: model architecture iteration, parameter scale expansion
Fire	🔥	Energy and execution	Compute conversion and work efficiency: GPU/TPU computing, job scheduling efficiency
Earth	🏔️	Support and stability	Platform support and orchestration governance: distributed coordination, middleware, scheduling systems
Metal	⚙️	Strength and standardization	Hardware constraints and physical boundaries: GPU/CPU performance, storage capacity, network bandwidth

Table 1: Engineering Mapping of Five Elements

Water – Data Flow and Quality

Corresponds to data pipelines, data assets, and quality control in the system.

Water symbolizes flow and containment, analogous to the circulation and nourishing role of data in the system, including:

Training data acquisition
Real-time data input
Feedback signal transmission
Data cleaning and quality assurance

Wood – Model Growth and Capability Expansion

Corresponds to the evolution and growth of machine learning models and algorithms.

Wood represents growth and creation, mapped to:

Model architecture iteration
Parameter scale expansion
Cultivation of new capabilities
Algorithm optimization and improvement

Fire – Compute Conversion and Work Efficiency

Corresponds to computing processes and the utilization of compute resources.

Fire symbolizes energy and execution, reflected as:

Using GPU/TPU and other compute resources for calculation
Converting electrical energy into model training and inference work
Parallel computing capability
Job scheduling efficiency

Earth – Platform Support and Orchestration Governance

Corresponds to the support and governance capabilities of the platform layer.

Earth represents support and stability, analogous to:

Infrastructure platform support for upper-layer applications
Distributed system coordination and orchestration
Middleware services
Scheduling systems and policy management
Permission systems, service quality assurance

Metal – Hardware Constraints and Physical Boundaries

Corresponds to underlying hardware and system hard limits.

Metal represents strength and standardization, mapped to:

GPU/CPU hardware performance
Storage capacity
Network bandwidth
Physical conditions and hard rules (power consumption, safety specifications, etc.)

Five Elements Generation Relationships

The Five Elements form a positive cycle through “generation” relationships:

Data (Water) spawns model growth (Wood), model requirements stimulate compute investment (Fire), compute development drives platform thickening (Earth), platform capabilities utilize hardware to push the boundaries of (Metal), and hardware progress in turn supports greater data acquisition (Water)

Figure 1: Five Elements generation relationship diagram. Water generates Wood, Wood generates Fire, Fire generates Earth, Earth generates Metal, Metal generates Water, representing the mutually reinforcing cycle between data, models, compute, platforms, and hardware.

Five Elements Overcoming Relationships

At the same time, overcoming relationships also exist among the Five Elements, meaning when one element is too strong or imbalanced, it will suppress or weaken another element:

Wood overcomes Earth: Excessive model expansion increases the burden on the platform (Earth), potentially even crushing the existing architecture
Earth overcomes Water: Overly heavy platforms and rules will hinder the free flow of data (Water)
Water overcomes Fire: Data bottlenecks will limit the performance of compute
Fire overcomes Metal: Excessive compute demand may break through hardware (Metal) limits
Metal overcomes Wood: Strict hardware and rule limitations will curb the expansion of models (Wood)

Figure 2: Five Elements generation and overcoming relationship diagram. Dashed arrows indicate overcoming relationships, reflecting the system’s internal checks and balances mechanism: any element becoming excessively strong will constrain another element.

Figure 3: Five Elements generation and overcoming relationship diagram. Dashed arrows indicate overcoming relationships, reflecting the system’s internal checks and balances mechanism: any element becoming excessively strong will constrain another element.

Five Elements Balance Diagnosis

Through the Five Elements model, engineering teams can systematically check the role completeness and balance of infrastructure.

Common Imbalance Patterns

Imbalance Pattern	Manifestation	Consequence	Solution
Strong Wood, Weak Water	Focus on model algorithm iteration, neglect data quality	Model performance hits bottlenecks	Strengthen data pipelines and quality control
Strong Metal, Weak Earth	Stack hardware, insufficient platform governance capability	Poor resource utilization, lack of vitality	Improve platform governance and scheduling
Vigorous Fire, Broken Wood	Large compute investment, models can’t keep up	Resource waste	Optimize model architecture, improve compute utilization efficiency

Table 2: Common Imbalance Patterns

Balance Principles

Successful large-scale systems require coordinated cooperation of all five elements

Let each of the five elements fulfill its duties in their respective roles
Maintain generation as primary, overcoming as secondary
Prevent any side from excessive expansion or shrinkage
Regularly check the balance state of Five Elements

Only by letting the five elements fulfill their respective roles and mutually promote each other, while preventing any side from excessive expansion or shrinkage, can the entire system maintain robustness and evolutionary capability.

The Yun Layer: Stages and Cycles of System Evolution

Jimmy Song — Tue, 10 Feb 2026 13:56:38 +0000

Yun (运) here refers to the developmental stages and temporal rhythms experienced by a system, which can be understood as the lifecycle cycles or “fortune” of infrastructure.

Large-scale infrastructure is not static but evolves cyclically through the Exploration Period, Platform Period, Scale Period, and Rebalancing Period, with each stage having its primary contradictions and tasks.

Below are the four evolutionary stages.

Exploration Period (Initial Stage)

Characteristics: High variance, low structure, rapid trial and error

At this stage, new technologies and requirements emerge constantly, system architecture is loose, and diverse experiments coexist.

Primary Tasks:

Explore effective paths
Rapidly validate model and functional directions
Collect data and preliminary stability signals

Five Elements Characteristics: Wood and Fire in Command

Model innovation (Wood) and computing experimentation (Fire) are core drivers
Expansion (Yang) outweighs constraints (Yin)

Architecture Strategy:

✓ Tolerate some chaos
✓ Encourage innovation and iteration
✓ Focus on collecting data and preliminary stability signals
✗ Don’t prematurely introduce heavy processes and restrictions

Platform Period (Growth Stage)

Characteristics: Standardization emerges, interfaces and processes converge

After exploration, the system enters a stage of integration and regulation, beginning to establish unified platforms, standard interfaces, and governance processes, consolidating scattered results into platform capabilities.

Primary Tasks:

Establish unified platforms
Define standard interfaces
Consolidate governance processes

Five Elements Characteristics: Fire Generates Earth

Successful practices in computing and functionality (Fire) give rise to platform support requirements (Earth)
Governance and standards gradually strengthen

Architecture Strategy:

✓ Extract common requirements
✓ Build support platforms (Yin increases)
✓ Lay the foundation for next-stage scaling
✗ Don’t remain in disordered exploration

Scale Period (Mature Stage)

Characteristics: Efficiency, throughput, and cost become the main battlefield

The system is deployed at scale, and focus shifts to optimizing efficiency and costs, improving throughput and reliability.

Primary Tasks:

Optimize efficiency
Improve throughput
Reduce costs
Ensure reliability

Five Elements Characteristics: Heavy Earth Breaks Wood

Platforms (Earth) and hard constraints begin to dominate
Overly idealistic model expansion (Wood) will encounter setbacks from realistic conditions

Architecture Strategy:

✓ Strengthen monitoring and automated operations
✓ Control overly strong “Yang” through governance means
✓ Ensure robust system operation
✗ Don’t continue with startup-era casual practices

Rebalancing/Substitution Period (Renewal Stage)

Characteristics: Old structures are corrected or replaced by new structures

When the previous stage’s patterns reach their limits, the system either enters self-correction by introducing new elements to rebalance, or gets disrupted and replaced by a new paradigm.

Primary Tasks:

Introduce new elements to rebalance
Or accept substitution by a new paradigm

Five Elements Characteristics: Metal and Water Rise Again

Suppressed hardware/rule innovations (Metal) and new data potentials (Water) rise again
Driving system transformation

Architecture Strategy:

✓ Be forward-looking, dare to break through
✓ Transition smoothly, avoid severe volatility
✗ Don’t cling to the status quo

Evolutionary Cycle

The above stages form a cyclical pattern, where the endpoint of each stage is also the starting point of the next ↻.

Figure 1: The “Yun” cycle of AI infrastructure evolution. Systems start from the exploration period, undergo platform period standardization, enter the scale period for efficiency optimization, and ultimately move toward a new cycle of rebalancing or substitution.*

The Art of Following the Momentum

A mature infrastructure organization should be able to determine its current stage based on internal and external signals and adjust its strategy accordingly.

If stage transitions are ignored or excessively rushed, the system will experience disturbances or even crises

Error Examples

Erroneous Behavior	Manifestation	Consequence
Pulling Up Seedlings to Help Them Grow	Managing systems still in exploration period as scaled systems, prematurely suppressing change	Stifling innovation
Going Against the Momentum	Remaining in disordered exploration when it’s time to enter the platform period	Missing the window for structured growth and creating hidden risks
Clinging to the Status Quo	Unwilling to change when rebalancing period is needed	System rigidity and aging

Table 1: Development Stage Characteristics

Stage Assessment Checklist

Through the “Yun” layer perspective, teams can examine the current macro stage:

Are we validating new concepts or expanding our achievements?
What is the system’s primary contradiction?
When might the next stage arrive?
Does our strategy align with the current stage?

Example Questions:

Are we in the exploration period?
- If yes → Focus on rapid trial and error and validation
- If no → Consider whether to enter the platform period
Does our system need standardization?
- If yes → Enter platform period, establish platforms and standards
- If no → Continue exploration

Qi Layer: Effective System Flow and Pressure Fields

Jimmy Song — Tue, 10 Feb 2026 13:56:17 +0000

Qi (气) in Chinese culture refers to the energy and flow field that permeates all things. In AI infrastructure, we borrow the concept of “Qi” to describe the effective flow and pressure distribution within systems.

This includes the circulation of data, tasks, and signals throughout the system, as well as how various explicit or implicit system pressures accumulate, propagate, and release.

The Essence of Qi: Overall State of Affairs

Unlike traditional single-point metric monitoring, the concept of “Qi” reminds us to focus on the overall state of affairs:

Signals are not isolated events, but rather gather and flow like a field

For example:

A sudden spike in GPU utilization may not be abnormal
But if multiple metrics (job queue length, response latency, memory usage, etc.) show a simultaneous trend of increase and persistence → this indicates a change in the “Qi field”
This signals the system entering a high-pressure state

This signal field manifests as the gathering and stretching of Qi, indicating the accumulation of some form of system tension.

Two States of Qi

Qi Flow: System Active

When all elements coordinate well, data and instructions flow smoothly, producing value efficiently:

Processing rates across all stages are basically matched
No long-term backlogs or idle resources
Timely system responses
Balanced resource utilization

Qi Stagnation: System Pathological

If a bottleneck or imbalance occurs somewhere, Qi’s flow is obstructed, causing local pressure to surge:

Jobs queue for long periods
CPU/GPU long-term idle or 100% utilization
Serious message queue backlog
Frequent anomaly alerts

Ultimately, this may trigger failures or performance collapse at weak points.

Qi’s Flow Path

To intuitively understand Qi’s flow path, we can view the system as a closely connected network:

Figure 1: Diagram of system ‘Qi’ flow path. Data (Water) Qi enters Model (Wood), triggering Computing Power (Fire) operation, coordinated via Platform (Earth), executed on Hardware (Metal), producing results that feed back to the data layer, forming a closed loop.

Qi’s Cycle:

Data (Water) Qi enters Model (Wood)
Drives Computing Power (Fire) to operate
Coordinated via Platform (Earth)
Executes computation on Hardware (Metal)
Outputs results, producing new data or signals
Feeds back into the data pool (Water)
Cycle repeats

Two Forms of Qi

Healthy Flow

Qi circulates ceaselessly among the five elements, maintaining system functionality:

If every step flows smoothly → system operates smoothly
If any step is obstructed → Qi flow slows or even reverses, damaging system performance and stability

Pressure Propagation

Qi refers not only to healthy flow, but also to pressure propagation:

Example: Data Inflow Surge

Data inflow surges but model processing capacity cannot keep up
Unprocessed data continuously accumulates
Manifests as excessive pressure in the data layer (Water)
Leading to suppression of computing power performance (Fire weakens)

Example: Hardware Resource Exhaustion

Hardware (Metal) resources exhausted
Computing requests cannot be satisfied
Obstructed Qi transforms into queuing pressure
Feeds back to platform (Earth) scheduling layer and user experience

Application of Qi Layer in Operations

Through the lens of “Qi”, operations and architecture teams can more sensitively detect sub-optimal system states:

Qi State	Manifestation	Warning Significance
Stagnation Emerging	Latency jitter gradually worsening	System entering sub-stable state, needs 疏导
Flow Obstruction	Request failure rate rising, retries increasing	某环节阻塞，needs investigation
Qi Scattering	Metrics fluctuating severely, irregular	System severely imbalanced, needs overall adjustment
Qi Deficiency	Resource utilization long-term low	Configuration unreasonable, needs optimization

Table 1: Qi State and Warning Significance

Qi Disorder Precedes Major Incidents

Latency jitter gradually worsening → signals system entering sub-stable state
If no measures are taken to resolve (scaling resources, optimizing algorithms, or rate limiting) → may evolve to complete failure
Agent task interaction rhythm (Qi) slows or stops → may indicate poor communication between agents or deadlock

Strategies for Guiding Qi Flow

Maintaining smooth Qi flow requires building resilience:

Architecture Level

Peak shaving and valley filling mechanisms: Absorb 突发流量
Message queue backpressure protection: Prevent pressure backflow
Elastic buffer design: Reserve margin to handle impacts

Strategy Level

Slack capacity: Maintain certain redundancy
Elastic scaling strategies: Dynamically adjust resources
Rate limiting and degradation mechanisms: Protect core functionality

Agent System Special Attention

Monitor task queues and communication latency
Ensure information flow (Qi) between agents is unobstructed
Introduce coordinator agents or reduce concurrency when necessary to smooth Qi flow

Qi Layer Monitoring Practices

Establish system-wide observability:

Monitoring Dimension	Focus	Tool Examples
Traffic Distribution	Request flow across stages	Distributed Tracing
Queue Backlog	Queue length trends	Message Queue Monitoring
Resource Utilization	CPU/GPU/Memory/Storage	Prometheus + Grafana
Latency Distribution	P50/P95/P99 latency	APM Tools
Anomaly Trends	Error rate, retry rate changes	Log Aggregation Analysis

Table 2: Qi Layer Monitoring Dimensions

The Qi layer provides an effective liquidity metric, helping us pulse-check whether the system’s “blood and Qi” are abundant and flowing smoothly

Summary

Qi’s operation can be understood as whether the system’s “meridians” are unobstructed:

Qi flow means system active: Data and instructions flow smoothly, producing value efficiently
Qi stagnation means system pathological: Flow obstructed, local pressure surges, ultimately triggering failures

Just as in Traditional Chinese Medicine’s four examination methods, by observing “Qi’s” operation, we can predict the trajectory of system problems and apply targeted remedies.

Dynamic Relationship Modeling: Five Elements Flow Under Yin-Yang Balance

Jimmy Song — Tue, 10 Feb 2026 13:55:47 +0000

Yin-Yang × Five Elements: Intrinsic Tension of Elements

Each Five Elements component contains both Yin and Yang aspects, manifesting with different polarities in different contexts:

Figure 1: Yin-Yang states of Five Elements. Each element includes Yin (potential, static, introverted) and Yang (explicit, dynamic, extroverted) aspects, with transformation possible between them depending on context.

Yin-Yang Attributes of the Five Elements:

Five Elements	Yin State	Yang State
Water (Data)	Potential data reserves, implicit patterns (static storage of historical data)	Instant data flow, real-time feedback
Wood (Model)	Dormant capabilities (unactivated parameters, backup algorithms)	Explicit expansion (model architecture updates, parameter surge)
Fire (Compute)	Stored energy (idle compute, waiting for scheduling)	High-load operation
Earth (Platform)	Static support (stable operation, non-intervention)	Proactive scheduling and expanded governance
Metal (Hardware)	Implicit constraints (unused capacity)	Explicit limits (resource hard caps maxed out)

Table 1: Dynamic Model Overview

Signs of Yin-Yang Imbalance:

Fire Excessively Yin: GPU compute idle for long periods while tasks backlog → Poor scheduling
Fire Excessively Yang: GPUs at 24-hour full load with no elasticity → Hidden crash risk
Earth Excessively Yang: Too many platform rules → Stifling innovation
Earth Excessively Yin: Lack of platform control → Leading to chaos

Five Elements × Qi: Dynamic Network of Flow

The Five Elements framework provides tools to decompose systems, but system components are not static puzzles—rather, they connect into a dynamic network through the flow of Qi.

Generating Relationships: Qi flows smoothly, forming positive feedback loops
Controlling Relationships: Qi stagnates at certain links or reverse effects strengthen

Dynamic Relationship Principles:

Generating primarily, Controlling secondarily—main energy flows transmit successfully through each link, while balancing forces intervene moderately only to prevent extreme situations.

Yun × Yin-Yang Five Elements: Boundary Conditions for Stage Evolution

The stage-based nature of Yun provides a perspective of boundary conditions evolving over time for the aforementioned Yin-Yang Five Elements dynamics.

Each stage strengthens or weakens certain elements and tensions:

Stage	Main Characteristics	Five Elements Characteristics	Yin-Yang Characteristics
Exploration Stage	High variance, low structure, rapid trial and error	Wood and Fire dominant	Expansion (Yang) outweighs Constraints (Yin)
Platform Stage	Standardization emerges, interfaces and processes converge	Fire generates Earth	Governance (Yin increasing) gradually strengthens
Scale Stage	Efficiency, throughput, cost become main battlegrounds	Earth dominates Wood	Stability (Yin) takes precedence
Rebalancing Stage	Old structures corrected or replaced by new structures	Metal and Water resurge	Transformation (Yang) rises again

Table 2: Typical Interaction Scenarios

Dynamic Stage Transitions:

The Yun layer tells us when to shift focus:

As stages change, the system needs to “allocate interests”
Previously dominant elements may become excessive and need convergence
Previously minor elements need strengthening to address shortcomings

Examples:

In Platform Stage/Scale Stage → Must strengthen governance (Earth’s Yang) and hardware optimization (Metal’s Yang)
To curb the 野蛮 growth tendencies left over from early stages (excessive Wood-Fire Qi)
In Rebalancing Stage → May need to reactivate suppressed innovation potential (Water-Wood Qi)

Comprehensive Analysis Case: GPU Scheduling Scenario

Let’s see how to apply the four-layer model to analyze a real GPU scheduling problem.

Problem Scenario: Cluster experiences task queues under high load

Layer	Diagnosis	Findings
Qi Layer	Observe Qi flow state	Compute Fire Qi is obstructed
Five Elements Layer	Locate elements	Data input too intense (Water Yang excessive) but scheduling (Platform Earth) strategy cannot keep up
Yin-Yang Layer	Analyze tensions	Scheduling strategy blindly pursues maximizing utilization (excessively Yang) while lacking elastic buffers (Yin)
Yun Layer	Assess stage	This is an emerging business that just passed exploration stage and hasn’t perfected scheduling—Platform Stage

Table 3: Four-Layer Diagnostic Analysis

Solutions

Based on four-layer collaborative diagnosis, develop comprehensive solutions:

Qi Layer: Unblock Qi flow
- Expand resources or optimize algorithms
Five Elements Layer: Balance elements
- Strengthen platform scheduling capabilities (Earth)
Yin-Yang Layer: Restore balance
- Introduce elastic buffer mechanisms (supplement Yin)
- Avoid blindly pursuing high utilization
Yun Layer: Follow the trend
- Accelerate introduction of standardized scheduling and resource governance (Earth’s Yun is approaching)

Value of Dynamic Modeling

Through the multi-level dynamic modeling above, we can:

Explain complex scenarios more comprehensively: No longer limited to single perspectives
Locate root causes of problems: Find fundamental causes rather than surface phenomena
Point improvement directions: Obtain systematic solutions
Predict system evolution: Prepare in advance for stage transitions

Practical Recommendations

In daily architecture design and operations, you can establish these thinking habits:

When encountering problems: Analyze layer by layer from a four-layer perspective
When making decisions: Consider impacts on all four layers
When conducting post-mortems: Check whether warning signals from the four-layer model were ignored

The value of a system lies not in pursuing the extreme of a single performance indicator without limit, but in balancing all elements to achieve long-term coordinated development

Engineering Practice Guide: Architecture Decisions Guided by Theory

Jimmy Song — Tue, 10 Feb 2026 13:55:59 +0000

The theoretical models mentioned above are not 停留在停留在 the conceptual level, but directly provide guidance for the engineering practice of AI infrastructure. In specific scenarios such as GPU scheduling, Agent runtime, and platform governance, we can follow the principles below to apply the Yin-Yang Five Elements Qi Movement model.

Balance Yin and Yang, Avoid Extremes

Consider both propelling forces and restraining forces when making architecture decisions.

GPU Cluster Scaling:

✓ Satisfy business growth (expanding Yang)
✓ Set quota and priority policies (constraining Yin)
✓ Prevent resource abuse

Agent Runtime Design:

✓ Give agents more autonomy (innovation, Yang)
✓ Introduce monitoring and sandboxing mechanisms (governance, Yin)
✓ Prevent loss of control

Practice Checklist:

After every major adjustment, ask yourself: Have I introduced corresponding counter-forces to stabilize the system?

Complete the Five Elements, Identify and Fill Weaknesses

Regularly review whether the five types of elements in the system are balanced.

GPU Infrastructure Check:

Do data pipelines keep up with computing power improvements? (Water and Fire matching)
Does model optimization fully utilize hardware? (Wood and Metal matching)
Can the scheduling platform handle peak loads? (Earth supporting Fire)
Has hardware resources become a bottleneck? (Metal not holding back)

Agent Platform Check:

Is there high-quality knowledge base or real-time data support? (Water)
Is there strong model capability? (Wood)
Is there sufficient computing resources? (Fire)
Is there a good orchestration framework? (Earth)
Is there a reliable environment and interfaces? (Metal)

Practice Strategy:

Once a bottleneck or overload is discovered in a certain link, decisively invest resources to fill the weakness or reduce the burden on the overloaded part

Problem Discovered	Solution
Insufficient data quality (“Water” weak)	Prioritize data governance
Long-term low hardware utilization (Metal strong, Fire weak)	Optimize algorithms or scheduling to better utilize hardware

Table 1: Problem Discovery and Solutions

Follow the Trend, Align with the Movement

Develop reasonable strategies based on the stage of the system.

Strategies for Different Stages:

Stage	Should Do	Should Not Do
Exploration Phase	Rapid trial and error, validate value	Prematurely introduce heavy processes and constraints
Platform Phase	Standardized management, MLOps tools	Remain in disordered exploration
Scale Phase	Strengthen governance and efficiency optimization	Still use the casual practices of the startup period
Rebalancing Phase	Architecture innovation, introduce new technologies	Refuse to move forward

Table 2: Strategies for Different Stages

Regular Assessment: At each quarter or important milestone, assess:

Which stage are we currently in?
What is the main contradiction in this stage?
When might the next stage arrive?
Prepare in advance for the transition

Practice Cases:

An AI training cluster after validating the concept → Should consider entering standardized management (transitioning from exploration phase to platform phase)
When system scale expansion encounters bottlenecks → Consider whether to enter the rebalancing phase and break through through architecture innovation

Observe Qi Field, Optimize Flow

Establish global observability of the system, focusing on trends and correlations rather than single-point metrics.

Monitoring Methods:

Distributed tracing
Metric correlation analysis
Full-link monitoring

Signals of Qi Disorder:

Signal	Possible Cause
Frequent occurrence of various abnormal logs	Global investigation needed
A metric’s periodic fluctuations becoming increasingly intense	The system may be approaching a limit internally

Table 3: Signals of Qi Disorder

Strategies to Keep Qi Flowing Smoothly:

Architecture Level:

Peak clipping and valley filling mechanisms
Message queue backpressure protection

Strategy Level:

Slack capacity
Elastic scaling strategies

Agent System Special Attention:

Monitor task queues and communication latency
Ensure smooth information flow (Qi) between agents
Introduce coordinator agents or reduce concurrency when necessary

Dynamic Adjustment, Continuous Rebalancing

Integrate the Yin-Yang Five Elements Qi Movement model into the team’s continuous improvement process.

Core Questions in Architecture Reviews or Incident Retrospectives:

Is the current main contradiction more inclined toward expansion or constraint, speed or stability?
Is any Five Elements element overloaded (Yang excess) or missing (Yin deficiency)?
Is System Qi congested somewhere?
Do our strategies align with the current stage?

Continuous Improvement Process:

Problem Discovery → Four-Layer Model Diagnosis → Strategy Formulation → Implementation Adjustment → Effect Evaluation → Continuous Optimization

Practice Case: Large-Scale GPU Training Cluster Optimization

Background: A team encountered stability issues while operating a large-scale GPU training cluster.

Four-Layer Model Diagnosis:

Layer	Diagnosis	Findings
Yin-Yang Layer	Speed vs Stability	Continuously compressing fault tolerance and testing time in pursuit of efficiency (speed Yang), leading to frequent online failures (stability Yin damaged)
Five Elements Layer	Five Elements Check	Data pipeline latency gradually increasing (Water weaker than Fire)
Movement Layer	Stage Judgment	System has moved from barbaric growth period to maturity period
Qi Layer	Qi Flow State	Qi stagnation phenomenon obvious

Table 4: Monitoring Methods

Comprehensive Solution:

Yin-Yang Balance:
- Suspend performance optimization
- Invest time to strengthen fault tolerance mechanisms and testing (supplement stability Yin)
Five Elements Completion:
- Add data preprocessing nodes and caching (strengthen Water)
Movement Adjustment:
- Change mindset, shift focus from feature expansion to optimization and governance
Qi Flow Regulation:
- Build full-link tracing system
- Monitor the time of each link from training job submission to completion
- Identify Qi stagnation points and clear them

Result: While maintaining high utilization, the cluster’s stability was greatly improved, and no serious downtime occurred again.

Scenario Application Quick Reference Table

Scenario	Yin-Yang Focus	Five Elements Check	Movement Judgment	Qi Flow Monitoring
GPU Scheduling	Utilization vs Elasticity	Fire - Earth - Metal Balance	Scale Phase Efficiency Optimization	Task queues, resource utilization curves
Agent Runtime	Autonomy vs Governance	Water - Wood - Fire Coordination	Exploration Phase Rapid Iteration	Communication latency, task interaction rhythm
Platform Governance	Innovation Risk Control vs Process Efficiency	Earth - Metal Constraints	Platform Phase Standardization	Rule execution rate, change frequency
Cost Optimization	Performance vs Cost	Fire - Metal Matching	Scale Phase Refinement	Resource waste, idle time

Table 5: Signals of Qi Disorder

Summary

Through the Yin-Yang Five Elements Qi Movement model, we can in practice:

Avoid Extremes: Not blindly pursuing single metrics
Systematic Thinking: Analyzing problems from multiple dimensions
Follow the Trend: Adjust strategies based on stages
Predict Problems: Early warning of risks through Qi field changes
Continuous Improvement: Establish systematic optimization processes

The value of this system lies in: combining Eastern wisdom with engineering practice to provide a unique and effective thinking framework for complex AI infrastructure

System Diagnosis Principles: Criteria for Health Status

Jimmy Song — Tue, 10 Feb 2026 13:56:28 +0000

To maintain the long-term healthy evolution of AI infrastructure, post-mortem summaries are far from sufficient. We need a set of system diagnosis principles to detect hidden risks early and correct deviations.

Based on the Yin-Yang Five Elements Yun model, diagnosis can be conducted from the following five dimensions:

Five-Dimensional Diagnosis Framework

Figure 1: Five-Dimensional Diagnosis Framework Diagram

Five Elements Balance Check

Assess the current status of five aspects: Data (Water), Models (Wood), Compute (Fire), Platform (Earth), and Hardware (Metal).

Diagnosis Method

Checklist:

Can data pipelines keep up with demands? (Water)
Are model capabilities fully utilized? (Wood)
Are compute resources effectively used? (Fire)
Can the platform support current load? (Earth)
Is hardware becoming a bottleneck? (Metal)

Identify Problems

Problem Type	Manifestation	Solution
Short Board	One element significantly weaker than others	Prioritize strengthening that element
Overload	One element consumes excessive resources or frequently becomes a bottleneck	Introduce limits or expand other elements to share pressure

Table 1: Problem Types and Solutions

Typical Symptoms

Water Level Too Low: Data pipelines always lag behind training needs → Replenish data processing capacity
Metal Overload: Hardware often runs at full capacity or even triggers limit alarms → Expand capacity or impose constraints on upper layers

Most failures do not stem from missing components, but from long-term role imbalance

Qi Flow Smoothness Check

Analyze whether Qi flows smoothly through the system via full-link monitoring.

Diagnosis Method

Key Metrics:

Latency distribution of key processes
Queue backlogs
Resource utilization curves

Qi Smooth vs. Qi Not Smooth

State	Characteristics
Qi Smooth	Processing rates across stages basically match, without long-term backlogs or idle resources
Qi Not Smooth	One stage remains a bottleneck for long periods, or large amounts of resources sit idle

Table 2: Qi Flow: Smooth vs Obstructed

Diagnosis Points

Distinguish temporary fluctuations from persistent trends: brief peaks don’t necessarily indicate Qi blockage, but persistent deviations must be addressed

Tool Support:

Dashboards and automated alerts
Timely capture of “stagnant Qi” locations
Further investigation of causes (which Five Elements imbalance corresponds)

Yin-Yang Dynamics Check

Assess whether current strategy and state are Yang Excess Yin Deficiency or Yin Excess Yang Deficiency.

Diagnosis Method

Qualitative Analysis:

Look at whether recent architecture decisions overly favor one extreme
Have you been continuously expanding and adding new features while ignoring stability?
Or conversely, multiple layers of approval and strict constraints but lack innovation momentum?

Quantitative Metrics:

Metric	Yang Excess	Yin Excess
Change Frequency	Extremely high	Extremely low
Incident Rate	Frequent	Extremely low but no change
Release Rhythm	Continuous	Long-term stagnation

Table 3: Yin-Yang Status

Balance Strategy

State	Symptoms	Solution
Yang Excess Yin Deficiency	Frequent changes with frequent incidents	Pause releases, focus on addressing hazards (replenish Yin)
Yin Excess Yang Deficiency	Long-term no change and stagnation	Introduce challenges and innovation (add Yang)

Table 4: Balance Strategies

Yun Alignment Check

Determine whether the organization’s actions match the system’s current stage, preventing counter-Yun operation.

Diagnosis Method

Combine Business Development and Technical Maturity:

Error Pattern	Manifestation	Consequences
Premature Standardization	Spending 大量精力 on process management and cost optimization for emerging projects	These are typically scale stage concerns, but the project is still in exploration stage
Counter-Yun Exploration	Frequently changing underlying architecture for widely used platforms without rigorous testing	Inconsistent with scaling stage

Table 5: Error Patterns

Stage-Strategy Reference Table

Stage	Should Focus On	Should Not Do
Exploration Stage	Diversity, flexibility, rapid trial and error	Premature pursuit of efficiency
Platform Stage	Standardization, process norms	Frequent arbitrary changes
Scale Stage	Optimization, stability, efficiency	Still growing wildly
Rebalancing Stage	Transformation, breakthrough, innovation	Clinging to the past

Table 6: Stage-Strategy Mapping

Checklist:

Which stage are we currently in?
Do our actions match the stage?
Do we need to adjust strategy?

When discovering actions don’t match the stage, immediately adjust strategy to avoid working at cross-purposes

Yang Runaway Warning

Pay special attention to whether there are signs of Yang state runaway in the system.

What is Yang Runaway?

Exponential explosion or collapse risk caused by unconstrained positive feedback.

Typical Scenarios

Scenario	Mechanism	Risk
Service Call Volume Surge	Bug or abuse → Resource strain → Queuing and retry storms → Further increase in calls	Resource exhaustion
Training Task Self-Replication	Tasks unlimitedly self-replicate to accelerate → Cluster resource exhaustion	System collapse

Table 7: Typical Scenarios

Diagnosis Signals

A metric shows exponential explosive growth
Lack of slowing mechanisms
Formation of vicious cycles

Response Strategy

Strategy	Means	Effect
Establish Hard Limits	Metal’s constraints	Immediate shutdown
Introduce Negative Feedback	Earth’s governance (rate limiting, quotas)	Braking and deceleration
Break Positive Feedback Chain	Activate emergency plan	Pull back to steady state

Table 8: Response Strategies

When discovering a metric showing exponential explosive growth without slowing mechanisms, intervene immediately

Diagnosis Implementation Process

Regular Diagnosis Mechanism

Recommend establishing a periodic diagnosis process:

Figure 2: Regular Diagnosis Mechanism Flowchart

Diagnosis Meeting Agenda

Fixed Session of Weekly Operations Review Meeting:

Check Five Elements scores for each module
Browse global Qi flow diagram
Analyze Yin-Yang dynamics
Discuss current Yun

This systematic examination makes hidden risks 无处遁形，thus achieving prevention before problems occur

Diagnosis Action Matrix

Diagnosis Result	Action Recommendation
Five Elements: One Element Too Weak	Concentrate resources to strengthen the weakness
Five Elements: One Element Overloaded	Expand capacity or introduce constraints
Qi Stagnation at One Stage	Clear bottlenecks, optimize processes
Yang Excess Yin Deficiency	Strengthen governance and stability mechanisms
Yin Excess Yang Deficiency	Activate innovation and boost vitality
Counter-Yun Operation	Adjust strategy and go with the flow
Yang Runaway Warning	Immediate intervention, break positive feedback

Table 9: Diagnosis Action Matrix

Summary

Through the above diagnosis principles, architects and operations teams can periodically take the pulse of infrastructure like TCM pulse diagnosis.

When diagnosis indicates imbalance in some aspect, immediately prescribe remedy based on the theory: replenish what needs replenishing, purge what needs purging.

Long-term adherence will keep the system on a healthy evolutionary trajectory.

Conclusion and Outlook

Jimmy Song — Tue, 10 Feb 2026 13:56:22 +0000

This paper systematically presents the four-layer model of “Yin-Yang - Five Elements - Yun - Qi” for AI infrastructure, providing a comprehensive cognitive map from theory to practice.

Review of Theoretical Model

Through four dimensions, we have constructed a global framework for understanding AI infrastructure:

Layer	Core Value	Key Insights
Yin-Yang	Understanding the tension and balance within systems	Expansion and constraint, innovation and governance, speed and stability—these three are opposites yet unified, all indispensable
Five Elements	Organizing the fundamental role elements of systems	Data, models, computing power, platforms, hardware—these five generate and restrain each other in endless cycles
Yun	Grasping the periodic patterns of system evolution	Exploration phase, platform phase, scale phase, rebalancing phase—act in accordance with the trends
Qi	Insight into the flow state of system operation	When Qi flows, the system is active; when Qi stagnates, the system becomes pathological

Table 1: Layer, Core Value, and Key Insights

More importantly, we have demonstrated how this theory combining Eastern wisdom with engineering practice can provide insights and guidance for real-world problems such as GPU scheduling, Agent Runtime, and platform governance.

Core Value of the Model

Holistic View

Traditional fragmented perspectives often see trees but not the forest, making it difficult to provide timely warnings of systemic risks

The Yin-Yang Five Elements Qi-Yun model, with its holistic view, helps architects:

Break free from the constraints of pure technical metrics
Grasp the principal contradictions and driving forces of system evolution
Extract meaningful patterns from complex signals

Dynamic View

The value of a system lies not in pursuing the extreme of a single performance indicator without limit, but in balancing all elements to achieve long-term coordinated development

The model’s dynamic view reminds us:

Yin-Yang dynamics transform dynamically with environment and stage
The same capability may shift from advantage to risk at different stages
Strategies need timely adjustment as Yun changes

Balance View

The core philosophy of the model is balance rather than extreme:

Not pursuing the limit of a single metric
But pursuing system coordination and sustainability
Finding dynamic balance points within unity of opposites

Practical Application Value

During Architecture Design

Consider the completeness and balance of the Five Elements
Reserve Yin-Yang constraint mechanisms
Design evolution paths that align with Yun trends
Plan channels for Qi flow

During Operations and Governance

Regularly check Five Elements balance
Monitor Qi circulation status
Assess Yin-Yang dynamic changes
Determine Yun phase transitions
Provide early warning of Yang loss-of-control risks

During Decision Review

Analyze root causes from the four-layer model perspective
Check whether basic principles of any layer were violated
Develop systematic solutions
Establish long-term improvement mechanisms

Insights for Architects

In an era of flourishing large models and autonomous agents, infrastructure has become unprecedentedly complex and active.

Cognitive Upgrade

From “managing machines and applications” to “managing intelligence and knowledge”:

Not only focus on application logic itself
But more on how knowledge and intelligence integrate into systems
View models as dynamically evolving components

Mindset Shift

From single-metric optimization to system balance:

Not pursuing the extreme of a single element
But pursuing overall coordination and sustainability
Finding dynamic balance within unity of opposites

Capability Development

From technical expert to systems philosopher:

While mastering technical tools
Cultivate systems thinking and philosophical reflection
Apply holistic frameworks like Yin-Yang and Five Elements

Limitations of the Model

It must be noted that this theory is not a panacea:

Not a Rigid Formula

Its value lies not in providing a rigid formula, but in guiding us to return to reality and think about problems from a more comprehensive perspective

The model provides a thinking framework, not standard answers
Specific applications need to consider actual scenarios
Architects ultimately must make judgments based on specific context

Requires Continuous Validation

Theory needs continuous validation and refinement in practice
Different scenarios may require adjustment and extension
Feedback and improvement in practice are encouraged

Supplement, Not Replace

The model is a tool to assist decision-making
Cannot replace professional judgment and experience
Should be used in combination with other methodologies

Future Outlook

Theory Development

This model has significant room for development:

Quantitative Metrics: Develop more precise quantitative indicators to make the theory more actionable
Tool Support: Develop analysis tools and automated diagnostic systems based on the model
Case Accumulation: Collect more practical cases to validate and enrich the theory
Cross-Domain Application: Explore applications of the model in other complex system domains

Practice Promotion

We hope this framework can help:

CTOs, infrastructure architects, and platform R&D teams
When facing increasingly complex AI infrastructure
Make wiser decisions

Ultimate Vision

Standing with sword in the midst of waves of change, embracing both the Yang of innovation and the Yin of governance, riding the system’s Qi above the currents

Conclusion

AI infrastructure stands at the starting point of a new era. We need not only technological innovation but also conceptual innovation.

The Yin-Yang Five Elements Qi-Yun model offers a unique perspective—combining Eastern philosophical wisdom with modern engineering practice—helping us find simplicity in complexity, stability in change, and unity in opposition.

We hope this model becomes a powerful tool for your thinking about AI infrastructure, helping you find your own “Way” in the balance and evolution of systems.

What Is AI-Native Infrastructure?

Jimmy Song — Sun, 18 Jan 2026 05:43:57 +0000

The essence of AI-native infrastructure is to make model behavior, compute scarcity, and uncertainty governable system boundaries.

AI-native infrastructure is not a simple checklist of technologies, but rather a new operating order designed for a world where “models become actors, compute becomes scarce, and uncertainty is the system default.”

The core of AI-native infrastructure is not faster inference or cheaper GPUs, but providing governable, measurable, and evolvable system boundaries for model behavior, compute scarcity, and uncertainty—making AI systems deliverable, governable, and evolvable in production environments.

Why We Need a More Rigorous Definition

The term “AI-native infrastructure/architecture” is being adopted by an increasing number of vendors, but its meaning is often oversimplified as “data centers better suited for AI” or “more complete AI platform delivery.”

In practice, different vendors emphasize different aspects of AI-native infrastructure:

Cisco emphasizes delivering AI-native infrastructure across edge/cloud/data center domains, highlighting delivery paths where “open & disaggregated” and “fully integrated systems” coexist (e.g., Cisco Validated Designs).
HPE emphasizes an open, full-stack AI-native architecture for the entire AI lifecycle, model development, and deployment.
NVIDIA explicitly proposes an AI-native infrastructure tier to support inference context reuse for long-context and agentic workloads.

For CTOs/CEOs, a definition that can guide strategy and organizational design must meet two criteria:

Clarify how the first-principles constraints of infrastructure have changed in the AI era
Converge “AI-native” from a marketing adjective into verifiable architectural properties and operating mechanisms

Authoritative One-Sentence Definition

AI-native infrastructure is:

An infrastructure system and operating mechanism premised on “models/agents as execution subjects, compute as scarce assets, and uncertainty as the norm,” which closes the loop on “intent (API/Agent) → execution (Runtime) → resource consumption (Accelerator/Network/Storage) → economic and risk outcomes” through compute governance.

This definition contains two layers of meaning:

Infrastructure: Not just a software/hardware stack, but also includes scaled delivery and systemic capabilities (consistent with vendors’ emphasis on “full-stack integration/reference architectures/lifecycle delivery”).
Operating Model: It inevitably rewrites organizational and operational methods, not just a technical upgrade—budget, risk, and release rhythm are strongly bound to the same governance loop.

Three Premises

The core premises of AI-native infrastructure are as follows. The diagram below illustrates the correspondence between these three premises and governance boundaries.

Figure 1: Three constitutional premises of AI-native infrastructure

Model-as-Actor: Models/agents become “execution subjects”
Compute-as-Scarcity: Compute (accelerators, interconnects, power consumption, bandwidth) becomes the core scarce asset
Uncertainty-by-Default: Behavior and resource consumption are highly uncertain (especially in agentic and long-context scenarios)

These three points collectively determine: the core task of AI-native infrastructure is not to “make systems more elegant,” but to make systems controllable, sustainable, and capable of scaled delivery under uncertain behavior.

Boundaries: What AI-Native Infrastructure Manages and What It Doesn’t

In practical engineering, defining boundaries helps focus resources and capability development. The table below summarizes what AI-native infrastructure focuses on versus what it doesn’t:

Not focused on:

Prompt design and business-level agent logic
Individual model capabilities and training secrets
Application-layer product features themselves

Focused on:

Compute Governance: Quotas, budgets, isolation/sharing, topology and interconnects, preemption and priorities, throughput/latency versus cost tradeoffs
Execution Form Engineering: Unified operation, scheduling, and observability for training/fine-tuning/inference/batch processing/agentic workflows
Closed-Loop Mechanisms: How intent is constrained, measured, and mapped to controllable resource consumption and economic/risk outcomes

Verifiable Architectural Properties: Three Planes + One Loop

To facilitate understanding, the following sections introduce the core architectural properties of AI-native infrastructure.

The diagram below shows the visualization of the three planes and the closed loop, facilitating rapid boundary alignment during reviews.

Figure 2: Three Planes and One Loop reference architecture

Three Planes:

Intent Plane: APIs, MCP, Agent workflows, policy expressions
Execution Plane: Training/inference/serving/runtime (including tool calls and state management)
Governance Plane: Accelerator orchestration, isolation/sharing, quotas/budgets, SLO and cost control, risk policies

The Loop:

Only with an “intent → consumption → cost/risk outcome” closed loop can it be called AI-native.

This is also why NVIDIA elevates the sharing and reuse of “new state assets” like inference context to an independent AI-native infrastructure layer: essentially bringing the resource consequences of agentic/long-context into governable system boundaries.

AI-Native vs Cloud Native: Where the Differences Lie

Cloud Native focuses on delivering services in distributed environments with portability, elasticity, observability, and automation. Its governance objects are primarily service/instance/request.

AI-native infrastructure addresses a different set of structural problems:

Execution unit shift: From service request/response to agent action/decision/side effect
Resource constraint shift: From elastic CPU/memory to hard GPU/throughput/token constraints and cost ceilings
Reliability pattern shift: From “reliable delivery of deterministic systems” to “controllable operation of non-deterministic systems”

Therefore, AI-native is not “adding a model layer on top of cloud native,” but rather shifting the governance center from deployment to governance.

Bringing It to Engineering: What Capabilities AI-Native Infrastructure Must Have

To avoid “right concept, misaligned execution,” the following minimum closed-loop capabilities are listed.

Resource Model: Making GPU, Context, and Token First-Class Resources

Cloud native abstracts CPU/memory into schedulable resources; AI-native must further bring the following resources under governance:

GPU/Accelerator Resources: Scheduled and governed by partitioning, sharing, isolation, and preemption
Context Resources: Context windows, retrieval paths, cache hits, KV/inference state asset reuse, etc., which directly affect tokens and costs
Token/Throughput: Become measurable capacity and cost carriers (can enter budgets, SLOs, and product strategies)

When tokens become “capacity units,” the platform is no longer just running services, but operating an “AI factory.”

Budgets and Policies: Binding “Cost/Risk” to Organizational Decisions

AI systems cannot operate with a “ship and done” approach. Budgets and policies must become the control plane:

Trigger rate limiting/degradation when budgets are exceeded
Trigger stricter verification or disable high-risk tools when risk increases
Version releases and experiments are constrained by “budget/risk headroom” (institutionalizing release rhythm)

The key is infrastructure solidifying organizational rules into executable policies.

Observability and Audit: Making Model Behavior Accountable and Observable

Traditional observability focuses on latency/error/traffic; AI-native must add at least three types of signals:

Behavior Signals: Which tools the model called, which systems it read/wrote, what actions it took, what side effects it caused
Cost Signals: Tokens, GPU time, cache hits, queue wait, interconnect bottlenecks
Quality and Safety Signals: Output quality, violation/over-privilege risks, rollback frequency and reasons

Without “behavior observability,” governance cannot be implemented.

Risk Governance: Bringing High-Risk Capabilities Under Continuous Assessment and Control

When model capabilities approach thresholds that can “cause serious harm,” organizations need a systematic risk governance framework, not relying on single-point prompts or manual reviews.

Can be split into two layers:

System-Level Trustworthiness Goals: Organizational-level requirements for security, transparency, explainability, and accountability
Frontier Capability Readiness Assessment: Tiered assessment of high-risk capabilities, launch thresholds, and mitigation measures

The value lies in: transforming “safety/risk” from concepts into executable launch thresholds and operational policies.

Takeaways / Checklist

The following checklist can be used to determine whether an organization has entered the AI-native stage:

Do we treat models as “agents that act,” not as replaceable APIs?
Do we bring compute and budgets into business SLAs and decision processes?
Do we treat uncertainty as the default premise, not as an exception?
Do we have audit, rollback, and accountability for model behavior?
Do we have cross-team AI governance mechanisms, not single-point engineering optimizations?
Can we explain the system’s operating boundaries, cost boundaries, and risk boundaries?

Summary

The essence of AI-native infrastructure lies in: taking models as behavior subjects, compute as scarce assets, and uncertainty as the norm, achieving deliverable, governable, and evolvable AI systems through governance and closed-loop mechanisms. Only by engineering these capabilities can organizations truly step into the AI-native stage.

References

AI-Native Infrastructure One-Page Reference Architecture: Three Planes + One Loop

Jimmy Song — Sun, 18 Jan 2026 00:00:00 +0800

The true value of architecture is enabling organizational consensus on complex systems within five minutes, not creating another new technology stack.

Industry-leading vendors emphasize different aspects: Cisco focuses more on AI-native infrastructure and reference designs such as Cisco Validated Designs; HPE emphasizes an open, full-stack AI-native architecture across the AI full lifecycle; NVIDIA explicitly proposes adding a new AI-native infrastructure tier for inference context reuse in long-context and agentic workloads. This chapter converges these perspectives into a verifiable architecture framework: Three Planes + One Loop.

Note

The “architecture” in this chapter serves as a review framework, not a component checklist. The goal is to unify organizational language and review boundaries, not to reinvent the technology stack.

One-Page Architecture Overview

Below is the detailed reference architecture diagram for the three planes and closed loop of AI-native infrastructure, helping readers quickly establish an overall understanding:

Figure 1: Three planes detailed architecture diagram

This diagram can be understood as: The new control plane (Intent) of AI-native infrastructure must be constrained by the Governance Plane and produce measurable resource consequences in the Execution Plane.

This is also the common ground behind different vendor narratives: Cisco uses reference designs and delivery frameworks to make infrastructure capabilities scalable and replicable; HPE uses open/full-stack to cover lifecycle delivery; NVIDIA elevates the reuse of “context state assets” to an independent infrastructure layer. All three point to the same issue: incorporating AI resource consequences into governable system boundaries.

Core Capabilities of the Three Planes

This section details the core capabilities of each of the three planes to help clarify focus areas during architecture reviews.

Intent Plane

The Intent Plane is responsible for expressing “what I want,” including the following capabilities:

Inference/Training APIs (entry points and contracts)
MCP/Tool calling protocols and tool catalog (standardizing tool access as “declaratable capability boundaries”)
Agent/Workflow (breaking down tasks into executable steps)
Policy as Intent: priorities, budgets, quotas, compliance/security constraints (front-loaded in the form of “intent”)

Key Point: The Intent Plane is not the starting point itself; the real starting point is—whether intent can be translated into executable and governable plans. Otherwise, Agents/MCP will only amplify uncertainty: more tools, longer chains, larger state spaces, and more uncontrollable resource consumption.

During architecture reviews, focus on these questions:

Is intent declarable (contract) and rejectable (admission)?
Does intent carry budget/priority/compliance constraints (policy as intent)?
Is the translation from intent to execution traceable?

Execution Plane

The Execution Plane is responsible for landing intent into “actual execution,” mainly including:

Training, fine-tuning, inference serving, batch processing, agentic runtime
“State and Context” services: cache/KV/vector/context memory, etc., for carrying inference context, retrieval results, and session state
Full-chain observability hooks: token metering, GPU time, video memory, network traffic, storage I/O, queue wait times, etc.

An industry trend worth emphasizing: as long-context and agentic workloads become widespread, “context” itself becomes a critical state asset and may even rise to become an independent infrastructure layer. NVIDIA explicitly proposes inference context memory storage in the Rubin platform, establishing an AI-native infrastructure tier to provide shared, low-latency inference context at the pod level to support reuse (for long-context and agentic workloads).

Review points focus on three things:

Is execution measurable: Can attribution be done across token/GPU/network/storage dimensions?
Is state governable: What are the lifecycle, reuse boundaries, and isolation strategies for context and cache?
Is observability closed-loop oriented: Observability is not for “seeing,” but for “enabling governance to correct deviations.”

Governance Plane

The Governance Plane is the “core differentiator” of AI-native infrastructure, responsible for transforming resource scarcity and uncertainty into a controllable system:

Budget/quotas/billing: governing consumption across teams, tenants, projects, models, and agent tasks
Isolation and sharing strategies: same-card sharing, video memory isolation, preemption, priorities, fairness
Topology-aware scheduling: incorporating GPU, interconnect, network, and storage topology into placement (especially in training and high-throughput inference)
Risk and compliance control: audits, policy enforcement points, sensitive data and access control
Integration with FinOps/SRE/SecOps: incorporating cost, reliability, and risk into a single operational mechanism

From a vendor narrative perspective, this layer typically corresponds to “reference architecture + full-stack delivery”: Cisco emphasizes accelerating and scaling delivery in AI infrastructure through “fully integrated systems + Cisco Validated Designs”; HPE emphasizes end-to-end delivery with “open, full-stack AI-native architecture” to support model development and deployment.

The baseline question for Governance Plane reviews is: Can you make explainable resource allocation and degradation decisions under budget/risk constraints?

Closed-Loop Mechanism Explained

This section introduces the core workflow of the closed-loop mechanism to help understand the essential difference between AI-native and AI-ready.

Significance of the Closed-Loop Mechanism

The closed loop is the most confusing yet critical dividing line between AI-native and “AI-ready.”

The minimal implementation of the closed loop includes four steps:

Admission: Bind intent with policy at the entry point (budget, priority, compliance)
Translation: Translate intent into executable plans (select runtime, resource specifications, topology preferences)
Metering: End-to-end metering and attribution across tokens/GPU/network/storage
Enforcement: Budget triggers degradation/rate limiting/preemption; risk triggers isolation/audits; SLO triggers scaling/routing

In other words: The closed loop is not a “monitoring dashboard,” but a “governance-driven real-time correction mechanism.” If there is no closed loop for “intent → consumption → cost/risk outcomes,” systems can easily spin out of control across cost, risk, quality, and other dimensions.

This is also why “AI-native” is often accompanied by changes in operating model: when system execution speed and resource consumption are amplified by models/agents, organizations must front-load governance mechanisms and institutionalize them. LF Networking also explicitly points out: becoming AI-native is not just a technical migration, but a redefinition of the operating model.

Practical Usage of the One-Page Architecture

In subsequent chapters, this “one-page architecture” can be repeatedly reused as a review template:

Discussing MCP/Agent: Position them in the Intent Plane and constrain with the closed loop (admission/translation) to avoid “intent proliferation”
Discussing runtime and platforms: Place in the Execution Plane, focusing on observable, attributable, governable state assets (context/cache/KV/vector)
Discussing GPUs, scheduling, costs: Ground in the Governance Plane, using budget/isolation/topology/metering as leverage points
Discussing enterprise implementation: Use the closed loop to examine if it’s “truly AI-native” (whether cost/risk outcomes can be written back as executable policies)

If you can only remember one sentence: The determination of AI-native is not in “how many AI components are used,” but in “whether there exists an executable governance closed loop that constrains intent to controllable resource consequences and economic/risk outcomes.”

Summary

The one-page reference architecture provides a unified systems language and review framework for AI-native infrastructure. Through the three planes of Intent, Execution, and Governance, combined with the closed-loop mechanism, organizations can achieve efficient collaboration in architecture design, resource governance, and risk control. Looking ahead, as AI-native capabilities continue to mature, the governance closed loop will become a core competitive advantage for enterprises implementing AI.

References

Why Start with Compute Governance, Not API Design

Jimmy Song — Sun, 18 Jan 2026 05:45:13 +0000

Compute and governance boundaries are the true foundation of AI-native infrastructure architecture.

The previous chapter presented a “Three Planes + One Closed Loop” reference architecture. This chapter focuses on a core CTO/CEO-level question:

How should AI-native infrastructure be layered? What belongs in the “control plane” of APIs/Agents, what belongs in the “execution plane” of runtime, and what must be pushed down to the “governance plane (compute and economic constraints)”?

This question is critical because over the past year, many platform companies “pivoting to AI” have fallen into a common trap: treating AI as an API morphology change rather than a system constraint change. When your system shifts from “serving requests” to “model behavior” (multi-step Agent actions with side effects), what truly determines system boundaries is often not the elegance of API design, but rather: whether compute, context, and economic constraints are institutionalized as enforceable governance boundaries.

The core argument of this chapter can be summarized as:

AI-native infrastructure must be designed starting from “Consequence” rather than stacking capabilities from “Intent”; the control plane is responsible for expressing intent, but the governance plane is responsible for bounding consequences.

The Purpose of Layering: Engineering the Binding Between “Intent” and “Resource Consequences”

In AI-native infrastructure, mechanisms like MCP, Agents, and Tool Calling enhance system capabilities while also introducing higher risks. These risks are not abstract “uncontrollability,” but rather engineering “unbudgetable consequences”:

Path explosion in behavior, long contexts, and multi-round reasoning bring long-tail resource consumption;
The same “intent” can lead to orders-of-magnitude differences in tokens, GPU time, and network/storage pressure;
Without governance closed loops, systems will move toward “cost and risk runaway” while becoming “more capable.”

Therefore, the fundamental purpose of layering is not abstract aesthetics, but achieving a hard constraint goal:

Ensure each layer can translate upper-layer “intent” into executable plans and produce measurable, attributable, and constrainable resource consequences.

In other words, layering is not about making architecture diagrams clearer, but about encoding “who expresses intent, who executes, and who bears consequences” into system structure.

AI-Native Infrastructure Five-Layer Structure and “Three Planes” Mapping

To help understand the layering logic, the diagram below refines the “Three Planes” architecture from the previous chapter, proposing a more actionable “five-layer structure”:

Figure 1: Layered governance relationship from intent to consequence

Top two layers = Intent Plane
Middle two layers = Execution Plane
Bottom layer = Governance Plane

Below is a detailed expansion of the five-layer architecture, showing the primary responsibilities and typical capabilities of each layer:

Figure 2: Five-layer architecture diagram

It is important to note that MCP belongs to Layer 4 (Intent and Orchestration Layer), not Layer 1. The reason is that MCP primarily defines “how capabilities are exposed to models/Agents and how they are invoked,” addressing control plane consistency and composability, but does not directly take responsibility for “how the resource consequences of capability invocations are metered, constrained, and attributed.”

MCP/Agent is the “New Control Plane,” But Must Be Constrained by the Governance Layer

MCP/Agent is called the “new control plane” because it moves system “decisions” from static code to dynamic processes:

“Tool catalogs + schemas + invocations” form a composable capability surface;
Agents complete tasks by selecting tools, invoking tools, and iterating reasoning;
“Policy” is no longer just in code branches but expressed as routing, priorities, budgets, and compliance intent.

However, it is crucial to emphasize an infrastructure stance, which is also the foundation of this chapter:

MCP/Agent can express intent, but the key to AI-native is: intent must be translated into governable execution plans and metered and constrained within economically viable boundaries.

This statement aims to correct two common misconceptions:

Control plane is not the starting point: Treating MCP/Agent as “the entry point for AI platform upgrades” easily leads systems down a “capability-first” path;
Governance plane is the baseline: When compute and tokens become capacity units, any unconstrained “intent expression” will leak as cost, latency, or risk.

Therefore, system layering should be clear: Layer 4 is responsible for “expression,” Layers 1/2/3 are responsible for “fulfillment and bearing consequences,” and the governance loop is responsible for “correction.”

“Context” Is Rising to a New Infrastructure Layer

In traditional cloud-native systems, request states are mostly short-lived, relying more on application-layer state management. Infrastructure typically only handles “computation and networking” without needing to understand the economic value of “request context.”

AI-native infrastructure is different. Long-context, multi-turn dialogue, and multi-agent reasoning mean inference state often survives across requests and directly determines throughput and cost. In particular, KV cache and context reuse are evolving from “performance optimization techniques” to “platform capacity structures.”

This can be summarized as an infrastructure law:

When a state asset (context/state) becomes a determinant variable of system cost and throughput, it rises from application detail to infrastructure layer.

This trend is gradually appearing in the industry: inference context and KV reuse are explicitly elevated to “infrastructure layer” capability development directions. Future expansion will include distributed KV, parameter caching, inference routing state, Agent memory, and a series of “state assets.”

The Foundation of AI-Native Infrastructure: Reference Designs and Delivery Systems

AI-native infrastructure is far more than “buying a few GPUs.” Compared to traditional internet services, AI workloads have three characteristics that make the “foundation” more engineered and productized:

Stronger topology dependencies: Network fabric, interconnects, storage tiers, and GPU affinity determine available throughput;
Harder scarcity constraints: GPU and token throughput boundaries are less “elastic” than CPU/memory;
Higher delivery complexity: Multi-cluster, multi-tenant, multi-model/multi-framework coexistence means only “replicable delivery” can scale.

Therefore, AI Infra is not just a component list, but must include “scalable delivery and repeatable operation” system capabilities:

Reference Designs (validated designs)

Codify “correct topology and ratios” into reusable solutions.

Automated Delivery

Institutionalize deployment, upgrade, scaling, rollback, and capacity planning.

Governance Implementation

Make budgeting, isolation, metering, and auditing default capabilities rather than after-the-fact patches.

From a CTO/CEO perspective, this means: what you purchase is not “hardware” but a “delivery system for predictable capacity.”

“Layered Responsibility Boundaries” from a CTO/CEO Perspective

To facilitate internal alignment on “who is responsible for what and what is the cost of failure,” the table below maps “technical layers” to “organizational responsibilities,” avoiding the scenario where platform teams only build control planes while no one bears consequence boundaries.

Layer	Typical Capabilities	Primary Owner (Recommended)	Cost of Failure
Layer 5 Business Interface	SLA, product experience, business goals	Product / Business	Customer experience and revenue impact
Layer 4 Intent/Orchestration (MCP/Agent)	Capability catalogs, workflow, policy expression	App / Platform / AI Eng	Behavior runaway, tool abuse
Layer 3 Execution (Runtime)	Serving, batching, routing, caching policies	AI Platform / Infra	Insufficient throughput, latency jitter
Layer 2 Context/State	KV/cache/context tier	Infra + AI Platform	Token cost spike, throughput collapse
Layer 1 Compute/Governance	Quotas, isolation, topology scheduling, metering	Infra / FinOps / SRE	Budget explosion, resource contention, incident spillover

Table 1: AI-Native Infrastructure Layer and Organizational Responsibility Mapping

As you can see, the organizational challenge of AI-native is not in “whether we have agents,” but in “whether inter-layer closed loops are established”. When model-driven amplification of consequences occurs, organizations must institutionalize governance mechanisms as platform capabilities: executable budgets, explainable consequences, attributable anomalies, and rewritable policies. This is the true meaning of “starting from compute governance” rather than “starting from API design.”

Conclusion

The layered design of AI-native infrastructure centers on engineering the binding between “intent” and “resource consequences.” The control plane is responsible for expressing intent, while the governance plane is responsible for bounding consequences. Only by institutionalizing governance mechanisms as platform capabilities can we ensure cost, risk, and capacity remain controllable while enhancing capabilities. As context, state assets, and other new variables become infrastructure, AI Infra delivery systems will continue to evolve, becoming the foundation for sustainable enterprise innovation.

References

Operating and Governing AI-Native Infrastructure: Metrics, Budget, Isolation, Sharing, SLO to Cost

Jimmy Song — Sun, 18 Jan 2026 04:17:45 +0000

The key to governing AI-native infrastructure lies in how to institutionalize the closed-loop management of costs and risks arising from uncertainty.

In the cloud-native era, system operations were typically considered “basically deterministic”: request paths were predictable, resource curves were relatively stable, and scaling could respond promptly to load changes. However, entering the AI era, this assumption no longer holds—uncertainty has become the norm.

This chapter aims to provide CTOs/CEOs with key conclusions for architecture reviews:

The starting point of AI-native infrastructure is to treat uncertainty as the default input; The goal is to achieve closed-loop governance of the resource consequences (cost, risk, experience) arising from uncertainty.

This is also why “becoming AI-native” in organizational contexts increasingly points to the reshaping of operational methods and governance models: when system consequences are amplified, governance must be institutionalized.

What is an “Uncertain System”

In this handbook, “uncertainty” does not refer to randomness in the probabilistic sense, but to three types of phenomena in engineering practice:

Unpredictable behavior: execution paths change dynamically with model inference, especially evident in agentic processes (Agent intelligent workflows).
Unpredictable resource consumption: tokens, KV cache, tool calls, I/O, and network overhead exhibit long-tail and burst characteristics.
Non-linear consequences: the same “intent” can produce cost and risk outcomes differing by orders of magnitude.

Therefore, the infrastructure problem of AI-native infrastructure has shifted from “how to make the system more elegant” to:

How to ensure the system maintains economic viability, controllability, and recoverability when worst-case scenarios occur.

During architecture reviews, if you cannot answer “what is the worst case, where are the upper bounds, and how to degrade/rollback when triggered,” you are still reviewing the 惯性 extension of deterministic systems, not true AI-native systems.

Major Sources of Uncertainty

The following table summarizes common sources of uncertainty in AI-native infrastructure and their specific manifestations, facilitating quick reference for CTOs/CEOs.

Type	Manifestations	Impact Areas
Behavior Uncertainty	Agent task decomposition path changes, tool selection and call sequence changes, failure retry and reflection	Cost, Risk, Resilience
Demand Uncertainty	Concurrency and burst, long-tail requests, multi-tenant interference (noisy neighbor)	Resource pools, Experience, Isolation
State Uncertainty	Context reuse across requests, KV cache migration and sharing	Performance, Cost, Governance
Infrastructure Uncertainty	High sensitivity to network/storage/interconnect, congestion and jitter amplified into tail latency	Experience, Cost, Stability

Table 1: Sources and Manifestations of Uncertainty in AI-Native Infrastructure

Behavior Uncertainty

Behavior uncertainty is mainly reflected in changes to agent task decomposition paths, dynamic adjustment of tool selection and call sequences, and path explosion caused by failure retry, reflection, and multi-round planning. Tools and contexts are combined through standard interfaces (such as MCP protocol integration), significantly expanding system capability surfaces while making branch space a governance challenge.

More critically, tool calls are not “free external functions” - they occupy context windows and consume token budgets, amplifying cost and tail latency pressures. Therefore, behavior uncertainty is not merely “feature flexibility” at the product layer, but “cost and risk elasticity” at the platform layer, which must be budgeted, capped, and made auditable.

Demand Uncertainty

Demand uncertainty includes concurrency and burst (peaks), long-tail requests (ultra-long contexts, complex reasoning), and mutual interference under multi-tenancy (noisy neighbor). This drives capacity planning from “average capacity” to “tail capacity + governance strategies.”

In AI-native infrastructure, experience and cost are often determined not by average requests, but by the combination of tail requests: a small number of long-chain, long-context, tool-intensive requests can overwhelm shared resource pools. Therefore, demand uncertainty requires answering: which requests deserve guarantees, which must be throttled, and which should be isolated.

State/Context Uncertainty

State uncertainty is the most underestimated category in the AI era: context is a state asset, and it often exists across requests. When inference state / KV cache is elevated to a reusable, shareable, migratable system capability, it is no longer an application detail but a decisive variable for throughput and unit cost. NVIDIA in public materials identifies Inference Context Memory Storage as a new infrastructure layer, pointing to state reuse and sharing requirements for long-context and agentic workloads.

The conclusion is: “context/state” has changed from optional optimization to a critical infrastructure asset that must be meterable, allocable, and governable.

Infrastructure Uncertainty

AI workloads are far more sensitive to network, interconnect, and storage than traditional microservice workloads. Congestion, packet loss, and I/O jitter are amplified into tail latency and job completion time instability, creating “non-linear consequences” for experience and cost.

This type of uncertainty usually cannot be solved through “component selection” but requires end-to-end path engineering constraints: from topology, bandwidth, and queuing, to transport protocols, isolation strategies, and congestion control—all must be incorporated into the governance plane, not just the operations plane.

How Uncertainty Amplifies Across Layers

The diagram below illustrates the closed-loop relationship between metrics, budgets, and isolation strategies, emphasizing that governance must be rewritable.

Figure 1: SLO to cost feedback loop

The flowchart below demonstrates the cross-layer amplification path of uncertainty in AI-native infrastructure:

Figure 2: Uncertainty amplification across layers

Typical phenomena include:

Agent branch explosion: more tools and composable paths make tail costs increasingly uncontrollable.
Context inflation: long contexts and multi-round reasoning make KV cache a performance bottleneck and cost black hole.
Resource contention distortion: GPU/network contention under multi-tenancy makes “average performance” meaningless—tails must be governed.

Therefore, the core of AI-native is not “making execution stronger,” but enabling you to stably answer three questions:

Where are the upper bounds (budgets, steps, call counts, state occupancy)
What to do when crossing boundaries (degradation, rollback, isolation, blocking)
How results are rewritten (policy iteration and cost correction)

Engineering Response of AI-Native Infrastructure

Enterprises can refer to the following five “hard standards” during reviews—missing any one means inability to achieve closed-loop governance of uncertainty.

Admission: Ingress Admission Control

Implement tiered admission for requests with ultra-long contexts, oversized tool graphs, or ultra-high budgets
Bind “budget, priority, compliance” as part of intent (policy as intent)
Clearly communicate rejection reasons and explain why requests are denied

Key Point

The responsibility of admission is not “to allow features,” but to write consequence constraints into the contract.

Translation: Intent Translation to Governable Execution Plans

Select runtime, routing/batching strategies, and caching strategies for requests
“Cap” agent workflows: maximum steps, maximum tool calls, maximum tokens
Include fallback paths: deterministic alternatives, cached answers, manual/rule-based fallbacks

Key Point

Upgrade from “prompt-driven execution” to “plan-driven execution”—plans must be understandable and constrainable by the governance plane.

Metering: End-to-End Metering and Attribution

Meter tokens, GPU time, KV cache footprint, I/O, and network for each request/agent task
Attribute by tenant, project, model, and tool to form cost and quality metrics
Separately label “tail overhead” so long-tail costs no longer hide in averages

Key Point

No ledger means no budget; no attribution means no governance, let alone ROI.

Enforcement: Budget and Degradation Mechanisms

Budget triggers: rate limiting, degradation, preemption, queuing (by priority and tenant isolation)
Risk triggers: isolation

Conclusion

The core of AI-native infrastructure governance lies in front-loading uncertainty, layered metering, policy feedback, and institutionalized constraints to form a closed loop of cost and risk. Only with engineering mechanisms such as Admission, Translation, Metering, and Enforcement can systems achieve economically viable, controllable, and recoverable operations under normalized uncertainty.

References

Organization and Culture: How the Operating Model Changes

Jimmy Song — Sun, 18 Jan 2026 04:18:02 +0000

The compute governance closed loop is the foundational safeguard for sustainable innovation in AI-native organizations.

Perspective

API/Agent/MCP solve “how intent is expressed,” while compute governance addresses “whether the resource consequences of intent are economically viable and risk-controllable.” In the AI era, the latter becomes a prerequisite for the former. API-first without governance only amplifies costs and uncertainty, pushing organizations into the trap of “functional but unsustainable.”

The FinOps Foundation states directly in “Scaling Kubernetes for AI/ML Workloads with FinOps” that Kubernetes elasticity can easily evolve into a runaway cost problem. Therefore, FinOps should not be just cost reporting, but must become a shared operating model where every scaling decision simultaneously answers two questions: Are performance SLOs met?, and Is it affordable?.

The Challenge of API-first “Implicit Assumptions” in the AI Era

The diagram below shows the boundary relationships and accountability chains between platform, ML, and security teams.

Figure 1: Organizational boundaries and accountability chain

The intuitive path of API-first is: first make the interfaces and workflows work, then gradually optimize performance and cost through engineering. In AI-native infrastructure, this path often fails because it relies on three implicit assumptions that no longer hold in the AI era.

Assumption 1: Resources are not the core scarcity

Traditional software bets scarcity on engineering efficiency, throughput, and stability; whereas AI-native infrastructure scarcity comes primarily from asset boundaries like GPU/interconnect/power consumption. Scarcity is no longer “slow to scale,” but “hard to scale and expensive,” constrained by both supply chain and datacenter conditions.

Assumption 2: Request costs are predictable

Traditional request cost distribution is relatively stable; AI requests are inherently long-tailed: branching in agentic tasks, inflation of long contexts, and chain amplification of tool calls all make tokens and GPU time into random variables that cannot be linearly extrapolated. You think you’re scaling “QPS,” but actually you’re scaling “total cost of tail probability events.”

Assumption 3: State is ephemeral and discardable

The cloud-native era emphasized stateless scaling with externalized state; but on the inference side, inference state/context reuse often determines whether unit costs are controllable. NVIDIA describes this in Rubin’s ICMS (Inference Context Memory Storage) as the “context storage challenge brought by new inference paradigms”: KV cache needs reuse across sessions/services, sequence length growth causes linear KV cache inflation, forcing persistence and shared access, forming a “new context tier,” and proving with TPS and energy efficiency gains that this is not a nice-to-have, but a threshold for scalability.

Conclusion

In AI-native infrastructure, state and compute governance have become prerequisites for “whether it can run,” not post-optimization items.

The Nature of Compute Governance: What is Being Governed

“Compute governance” is often misunderstood as “managing GPUs,” but what truly needs governance is the resource consequences of intent. More precisely, it’s governing the combined effects of four types of objects:

Token Economics

Each request/task’s token consumption, context inflation, implicit token tax from tool definitions and intermediate results, ultimately directly mapping to cost and latency.

Accelerator Time

GPU time, memory footprint, batching strategies, and the impact of routing and cache hits on effective throughput. The key is not “whether there are GPUs,” but “whether output per unit GPU time is controllable.”

Interconnect and Storage (Fabric & Storage)

Network and storage pressures from training all-reduce, inference KV/cache sharing, and cross-service data movement. AI performance and cost are often amplified by fabric, not by APIs.

Organizational Budget and Risk (Budget & Risk)

Multi-tenant isolation, fairness, audit, compliance, and accountability. These determine whether the system can scale to multiple teams/business lines, not just scaling demos to more instances.

The FinOps Foundation also emphasizes: AI/ML cost drivers are not just GPUs; storage (checkpoints/embeddings/artifacts), network (distributed training/cross-AZ), and additional licensing and marketplace fees often “quietly exceed compute.” Therefore, governance objects must cover end-to-end, not just stare at inference bills.

MCP/Agent: Amplification Effects Under Governance Gaps

MCP/Agent expand the “capability surface,” but simultaneously make cost curves steeper, especially showing exponential amplification when governance is missing:

More tools, more branches: Planning space expands, tail probability rises, cost volatility becomes uncontrollable.
Tool definitions and intermediate results consume context: Directly consuming context window and tokens, translating to cost and latency.
Stronger tool usage triggers more external I/O: External system calls, network round trips, and data movement all enter the overall cost function.

Anthropic explicitly states in “Code execution with MCP” that direct tool calls increase cost and latency due to tool definitions and intermediate results consuming context window; when tool numbers rise to hundreds or thousands, this becomes a scalability bottleneck, thus proposing code execution forms to improve efficiency and reduce token consumption.

Conclusion

In the MCP/Agent era, governance is not about suppressing innovation, but making innovation sustainable within budget boundaries. Without governance, agents are not productivity tools, but cost amplifiers.

Minimal Implementation Path for “Compute Governance First”

You don’t have to bind to any vendor, but you must implement a “minimum viable governance stack.” The goal is not perfection, but giving the system controllable boundary conditions from day one.

Admission and Budget (Admission + Budget)

Set budgets and priorities for workload types (training/inference/agent tasks).
Include budget, max steps, max tokens, max tool calls in policy-as-intent, and enforce at the entry point.

Practice Recommendation

FinOps’ core view is: embed FinOps early into architecture, making every scaling decision simultaneously answer “performance” and “affordability,” otherwise bills only get attention when incidents occur.

End-to-End Metering and Attribution (Metering + Attribution)

At minimum achieve one traceable chain: request/agent → tokens → GPU time/memory → network/storage → cost attribution (tenant/project/model/tool).
Without attribution, there is no governance; without governance, enterprise scaling is impossible, because costs and responsibilities cannot align, and organizations will internally 消耗 on “who consumed the budget.”

Isolation and Sharing (Isolation + Sharing)

Sharing for improving utilization; isolation for reducing risk. Both must exist simultaneously, not either/or.
CNCF’s Cloud Native AI report notes: GPU virtualization and sharing (like MIG, MPS, DRA, etc.) can improve utilization and reduce costs, but requires careful orchestration and management, and demands collaboration between AI and cloud-native engineering teams.
The key to governance is not choosing sharing or isolation, but making it an executable policy: who shares under what conditions, who isolates under what conditions.

Topology and Network as First-Class Citizens (Topology + Fabric First)

AI training and high-throughput inference are highly sensitive to network characteristics.
Cisco’s AI-ready infrastructure design guides and related CVD/Design Zone emphasize: building high-performance, lossless Ethernet fabric for AI/ML workloads, and delivering reference architectures and deployment guides through validated designs.
This means topology is not “the datacenter team’s business,” but a core variable determining whether JCT, tail latency, and capacity models hold.

Context/State Becomes a Governance Object (Context as a Governed Asset)

When long-context and agentic become mainstream, KV cache and inference context reuse will directly determine unit costs.
NVIDIA’s ICMS defines this as a “new context tier” for solving KV cache reuse and shared access, emphasizing TPS/energy efficiency gains.
In this era, treating context as a temporary variable is actively relinquishing cost control.

Anti-Pattern Checklist

The following anti-patterns are not “engineering inelegance,” but will trigger organizational 失控，worth vigilance.

API-first, treating governance as post-optimization

Result: System launches first, only to discover unit costs and tail latency are uncontrollable, can only “hard brake” through feature limiting/rate limiting, ultimately locking the product roadmap.
Contrast: FinOps points out elasticity easily becomes runaway costs, must advance cost governance into architecture decisions.

Treating MCP/Agent as capability accelerators, not cost amplifiers

Result: More tools make it “smarter,” but token and external call costs rise exponentially, engineering teams forced to fight systemic amplification with “more complex prompts and rules.”
Contrast: Anthropic notes tool definitions and intermediate results consume context, increase cost and latency, proposing more efficient execution forms as the scalability path.

Only buying GPUs, without sharing/isolation and orchestration

Result: Low utilization, severe contention, budget explosion, organizations internally blame each other “who’s grabbing resources, who’s burning money.”
Contrast: CNCF Cloud Native AI report emphasizes sharing/virtualization improves utilization, but must match orchestration and collaboration mechanisms.

Ignoring network and topology, treating AI as ordinary microservices

Result: Training JCT and inference tail latency amplified by network, capacity planning and cost models fail, more scaling makes it more unstable.
Contrast: Cisco in AI-ready network design and validated designs makes requirements like lossless Ethernet fabric critical foundations for AI/ML.

Summary

The first-principle entry point for AI-native is the compute governance closed loop: budget and admission, metering and attribution, sharing and isolation, topology and network, context assetization. API/Agent/MCP remain important, but must be constrained by this closed loop, otherwise the system can only oscillate between “smarter” and “more bankrupt.”

References

Migration Roadmap: From Cloud Native to AI Native

Jimmy Song — Sun, 18 Jan 2026 04:20:11 +0000

Migration is not “rebuilding the platform,” but using governance loops and organizational contracts to transform uncertainty into controllable engineering capabilities.

The previous five chapters have established: AI-native infrastructure is Uncertainty-by-Default. Therefore, the architectural starting point must be compute governance loops, not “connect a model and call migration complete.” Otherwise, systems easily spiral out of control in three dimensions: cost (runaway cost), risk (unauthorized actions/side effects), and tail performance (P95/P99 and queue tail behavior).

This explains why the FinOps Foundation emphasizes: running AI/ML on Kubernetes, “elasticity” easily evolves into uncontrollable cost overflow. FinOps must be incorporated into architecture and organization upfront as a shared operating model, not as an after-the-fact reconciliation exercise.

This article presents an actionable migration roadmap, covering both technical evolution paths and organizational implementation approaches. You don’t need to “rebuild an AI platform” all at once, but you must establish working governance loops at each stage: budget/admission, metering/attribution, sharing/isolation, topology/networking, and context assetization.

The North Star: From Platform Delivery to Governance Loops

The diagram below shows the migration path from bypass pilot to AI-first refactoring.

Figure 1: AI-native migration roadmap

Cloud-native migration typically centers on “capability delivery”: CI/CD, self-service platforms, service governance, and auto-scaling. Its default assumptions: systems are deterministic, costs grow linearly with requests, and scaling doesn’t significantly alter system boundaries.

AI-native migration must center on “governance loops”, focusing on cost, risk, tail performance, and state assets. Its default assumptions are precisely the opposite: systems are inherently uncertain, and the “actions and consequences” of inference/agents drive costs and risks into nonlinear territory.

North Star Definition

AI-Native Migration = Establish an AI Landing Zone + Compute Governance Loop + Context Tier, and ensure all agents/APIs/runtimes operate within this loop.

Elevating “Landing Zone” to North Star level here isn’t chasing trends—it’s because it naturally serves an organizational-level task: delineating responsibility boundaries between platform teams and workload teams. Major cloud providers universally use Landing Zones to host “shared governance baselines” (networking, identity, policies, auditing, quota/budget), while business teams iteratively build applications within controlled boundaries. For AI, this boundary is the carrier of the governance loop.

Migration Prerequisites: Build Three Foundations First, Then Scale Applications

You can run PoCs and build applications in parallel, but if these three foundations are missing, any “application explosion” can easily transform into platform firefighting and financial disputes.

Foundation A: FinOps / Quotas as Control Plane (Finance and Quotas as Control Plane)

The first migration step is not “launch the first agent,” but incorporating budgets, alerts, showback/chargeback, and quotas into the infrastructure control plane:

Budgets and alerts are not just financial reports, but triggers for runtime policies (rate limiting, degradation, queuing, preemption).
showback/chargeback is not just accounting, but binding “cost consequences” to organizational decisions and product boundaries.
Quotas are not static limits, but evolvable governance instruments (dynamic budgets and priorities by tenant/team/use-case).

Migration Threshold

If you cannot attribute the primary consumption of each agent/job to team/project/model/use-case (at minimum covering tokens, GPU time, KV footprint, key network/storage), you haven’t reached the “scale” starting line. Piloting is acceptable, but expansion is not advisable.

Foundation B: Resource Governance (GPU Sharing/Isolation and Orchestration Capabilities)

The “elasticity” of AI-native infrastructure is constrained by how scarce compute is governed. Treating GPUs as ordinary resources typically results in low utilization and uncontrolled contention. Therefore, you need viable combinations of sharing/isolation and orchestration capabilities:

Sharing/partitioning: MIG/MPS/vGPU paths transform “exclusive” into “pooled.”
Scheduling upgrades: Introduce explicit modeling of topology, queues, fairness, preemption, and cost tiers.
Orchestration loop: Solidify isolation, preemption, and priority policies into executable rules.

The key is not which partitioning technology you choose, but whether you can elevate GPUs from “machine assets” to first-class governance resources and incorporate them into budget and admission systems.

Foundation C: Fabric as a First-Class Constraint (Network/Interconnect as First-Class Constraint)

Training and high-throughput inference are extremely sensitive to congestion, packet loss, and tail latency. Ignoring networking and topology leads to “seemingly sporadic but actually structural” problems:

Training JCT is amplified by tail behavior, invalidating capacity planning;
Inference P99 and queue tails are amplified, making SLOs difficult to honor.

Therefore, you need to build reusable AI-ready network baselines: capacity assumptions, lossless strategies, isolation domain 划分，measurement and acceptance criteria. Networking is not “optimize later,” but baseline engineering that must land in Days 31–60.

Migration Path Selection: Layered by Organizational Risk and Technical Debt

Migration isn’t “pick one path and see it through,” but mapping organizations with different risk appetites and debt structures to different starting approaches and exit criteria. Paths can advance in parallel, but each needs defined applicable conditions and exit criteria.

Path 1: Bypass Pilot / Skunkworks

Applicable when cloud-native platforms are running stably, but AI demand is just emerging, organizational uncertainty is high, and governance mechanisms are not yet mature.

The approach is establishing an “AI minimum closed-loop sandbox” alongside the existing platform. The goal is not “feature completeness,” but “making the loop work”:

Independent GPU pool (or at least independent queue) + basic admission and budget
Minimal token/GPU metering and attribution
Controlled inference/agent entry points (max context / max steps / max tool calls)
“Failure-acceptable” SLOs and cost caps (define boundaries first, then discuss experience)

Exit criteria:

Cost curve is explainable (at minimum attributable to team/use-case)
GPU utilization and isolation strategies form reusable templates
Pilot capabilities can be 下沉 as platform capabilities (enter Path 2)

Path 2: Domain-Isolated Platform

Applicable when AI has entered multi-team, multi-tenant stages, requiring “pilot assets” to be solidified into platform capabilities to prevent cost and risk from spreading across domains.

The approach is building an AI Landing Zone, where the platform team centrally manages shared governance capabilities, and workload teams iteratively build applications within controlled boundaries.

Platform-side essential modules (recommend organizing by “governance loop”):

Identity/Policy: Unified identity, policy distribution, and auditing (policy-as-intent)
Network/Fabric baseline: AI-ready network baseline and automated acceptance
Compute governance: Quotas, budgets, preemption, fairness, isolation/sharing
Observability & Chargeback: End-to-end metering, alerts, showback/chargeback
Runtime catalog: “Golden paths” and templated delivery for inference/training runtimes

Exit criteria: Platform provides “replicable AI workload landing approaches” and can scale use case count under budget constraints, rather than relying on manual firefighting to maintain stability.

Path 3: AI-First Refactor (AI Factory / Replatform)

Applicable when AI is core business, requiring infrastructure to be treated as a “production line” rather than a “cluster,” and optimization objectives to switch from “shipping features” to “throughput/unit cost/energy efficiency.”

The approach centers on “state assets + unit cost” refactoring:

Context/state of inference/agents is explicitly governed and reused (no longer application-level tricks)
Introduce Context Tier architectural assumptions: long context and agentic inference require inference state / KV cache to be reusable across nodes and sessions
Drive platform evolution with “unit token cost, tail latency, throughput/energy efficiency,” not “number of new components”

Exit criteria: Can consistently make engineering decisions using “unit cost and tail performance,” and treat context reuse as a platform capability rather than application team trick caching.

90-Day Actionable Plan: AI Landing Zone + Minimum Governance Loop

The goal is to establish “AI Landing Zone + minimum governance loop” within 90 days, forming a replicable template. The key is not covering all scenarios, but connecting the admission—metering—enforcement—feedback loop.

Day 0–30: Establish the Ledger (Cost & Usage Ledger)

First, define attribution dimensions, establish budgets/alerts and baseline reports, and implement quotas/usage controls.

Attribution dimensions: tenant/team/project/model/use-case/tool
Establish budgets and alerts, baseline reports (cost + business value metrics)
Implement quotas and usage controls (at minimum covering GPU quotas and key service quotas)

Deliverables:

Cost and usage dashboard (weekly-level, traceable)
“Admission Policy v0” (max context / max steps / max budget)

Day 31–60: Establish Resource Governance (GPU Governance + Scheduling)

This phase requires evaluating GPU sharing/isolation strategies, introducing topology/networking constraints, and forming two golden paths for inference and training.

GPU sharing/isolation strategy: MIG/MPS/vGPU/DRA path evaluation and PoC (executable strategy as acceptance criteria)
Introduce topology/networking constraints, form AI-ready network baseline and capacity assumptions (including acceptance criteria)
Form two templated delivery paths for inference/training

Deliverables:

Workload templates (1 each for inference and training)
Scheduling and isolation strategies (whitelisted, auditable)

Day 61–90: Establish the Loop (Enforcement + Feedback)

The final phase requires executing budget policies, migrating pilot use cases to the landing zone, and solidifying organizational interfaces.

Execute budgets: rate limiting/queuing/preemption/degradation strategies, linked to SLOs
Migrate pilot use cases to landing zone (or service landing zone capabilities)
Solidify “organizational interface”: platform team vs workload team responsibility boundaries (forming executable contracts)

Deliverables:

“AI Platform Runbook v1” (including oncall, changes, cost auditing)
Two replicable use case landing paths (new use cases ≤ 30 minutes to golden path)

Operating Model: The “Contract” Between Platform Teams and Workload Teams

Migration success depends on establishing clear, executable “organizational contracts.” The contract essence: who is responsible for “capability provision,” who is responsible for “behavioral consequences.”

Platform teams provide (must be stable)

Landing zone, network baseline, identity and policies, budget/quota systems, metering/attribution, GPU governance capabilities, runtime golden paths

Workload teams own (must be self-service)

Model selection, prompt/agent logic, tool integration, SLO definition, business value measurement, use case risk classification and rollback paths

This is also why the FinOps Framework emphasizes operating model (personas, capabilities, maturity) rather than just tools: without “contracts,” budgets are difficult to execute; if budgets cannot execute, loops cannot form.

Migration Anti-Patterns

Below are common migration anti-patterns and their consequences:

Anti-Pattern	Typical Consequences
Build only API/Agent platform, without ledger and budget	runaway cost (most common, and difficult to remediate afterwards)
Treat GPUs as ordinary resources, without sharing/isolation and scheduling upgrades	Low utilization + uncontrolled contention, platform forced to allocate compute via “administrative means”
Ignore networking and topology	Tail latency and training JCT amplified, capacity planning fails, SLOs difficult to honor
Context not assetized (only “tricky caching” within applications)	Unit cost out of control in long context/agentic era, reuse capabilities difficult to solidify as platform capabilities

Table 1: Common Migration Anti-Patterns and Consequences

Summary

The core of AI-native migration is not a “migration checklist,” but under uncertainty premises, incorporating cost, risk, and tail performance into a unified governance loop, using Landing Zone to carry organizational contracts, and using Context Tier to implement state reuse infrastructure capabilities. Only in this way can platform and business maintain controllability and efficiency during scaled evolution.

References

Glossary

Jimmy Song — Sun, 18 Jan 2026 05:19:24 +0000

Unified terminology is the first step toward organizational consensus.

Conclusion first: in the context of AI-native infrastructure, key terms must remain consistent; otherwise, both governance and communication lose focus.

The following glossary serves to align cross-team terminology.

Core Terms

AI Native Infrastructure

An infrastructure system premised on “models/agents as execution entities, compute as scarce assets, and uncertainty as the norm,” closed-looping “intent → execution → resource consumption → economic and risk outcomes” through compute governance.

Model-as-Actor

Models/agents become “execution entities” with action capabilities, capable of invoking tools, modifying system state, and producing side effects, thus requiring governance and audit.

Compute-as-Scarcity

Compute (GPU, interconnects, power consumption, bandwidth) becomes the core scarce asset, with expansion constrained by supply chain and data center conditions, and costs that cannot be made elastic.

Uncertainty-by-Default

Behavior and resource consumption are highly uncertain (especially in agentic and long-context scenarios), requiring verification and fallback mechanisms.

Intent Plane

The API, Agent, and policy expression layer responsible for expressing “what I want,” including priorities, budgets, compliance, and other policies.

Execution Plane

The training/inference/serving/runtime layer responsible for translating intent into actual execution, including state management, tool invocation, model routing, and so on.

Governance Plane

The quota/budget, isolation/sharing, and cost control layer responsible for bounding resource consequences, including topology-aware scheduling, SLO and risk policies.

The Loop

Possessing a closed loop of “intent → consumption → cost/risk outcomes,” comprising four steps: Admission, Translation, Metering, and Enforcement.

Compute Governance

Governing the resource consequences of intent, including four categories of objects: token economics, accelerator time, interconnect and storage, and organizational budgets and risks.

FinOps / Financial Operations

Embedding cost governance early into architecture so that every scaling decision simultaneously answers “whether performance meets requirements” and “whether it is affordable.”

Agent

An execution entity that completes tasks by selecting tools, invoking tools, and iterating reasoning, with uncertain behavioral paths and resource consumption.

MCP / Model Context Protocol

A protocol that standardizes tool access as “declaratable capability boundaries,” defining how capabilities are exposed to models/agents and how they are invoked.

Operating Model

Institutional design for organization and operational methods, including responsibility boundaries, collaboration mechanisms, and decision-making processes, answering “who is responsible for what and what are the costs of failure.”

References

Executive Checklist (10 Questions)

Jimmy Song — Sun, 18 Jan 2026 05:21:59 +0000

The following 10 questions assess whether an organization possesses the strategic and execution readiness for AI-native infrastructure. The diagram below categorizes these questions into three domains: strategy, governance, and execution, facilitating executive discussions.

Figure 1: Executive checklist structure diagram

Can you clearly define the unit cost for each major AI workload (e.g., per 1M tokens, per agent task, per batch job)?
Do you have budget/quota mechanisms that can constrain team/project/tenant compute consumption within controllable bounds?
Can you make explicit policy trade-offs between “performance (throughput/latency)—cost—risk” (rather than relying on verbal constraints)?
Can your platform handle uncertainty: spikes, long-tail effects, and resource fluctuations caused by agent path explosions?
Are agent/MCP “intents” mapped to actionable and billable/auditable resource consequences?
Do you have clear resource isolation and sharing strategies (same-card sharing, memory isolation, preemption, prioritization) to improve utilization?
Can you achieve cross-layer observability: end-to-end tracing from request/agent → runtime → GPU/network/storage → cost?
Does your infrastructure support rapid adoption of new hardware/interconnect/topology changes (heterogeneity and evolution are the norm)?
Has the organization established “AI SRE/ModelOps + FinOps” collaboration mechanisms and accountability boundaries (who owns cost and reliability)?
When you say “we are AI-native,” can you provide three planes + one closed loop architecture diagram and governance strategy on a single page?

References

When GPUs Move Toward Open Scheduling: Structural Shifts in AI Native Infrastructure

Jimmy Song — Fri, 13 Feb 2026 14:32:46 +0000

The future of GPU scheduling isn’t about whose implementation is more “black-box”—it’s about who can standardize device resource contracts into something governable.

Figure 1: GPU Open Scheduling

Introduction

Have you ever wondered: why are GPUs so expensive, yet overall utilization often hovers around 10–20%?

Figure 2: GPU Utilization Problem: Expensive GPUs with only 10-20% utilization

This isn’t a problem you solve with “better scheduling algorithms.” It’s a structural problem - GPU scheduling is undergoing a shift from “proprietary implementation” to “open scheduling,” similar to how networking converged on CNI and storage converged on CSI.

In the HAMi 2025 Annual Review, we noted: “HAMi 2025 is no longer just about GPU sharing tools—it’s a more structural signal: GPUs are moving toward open scheduling.”

By 2025, the signals of this shift became visible: Kubernetes Dynamic Resource Allocation (DRA) graduated to GA and became enabled by default, NVIDIA GPU Operator started defaulting to CDI (Container Device Interface), and HAMi’s production-grade case studies under CNCF are moving “GPU sharing” from experimental capability to operational excellence.

This post analyzes this structural shift from an AI Native Infrastructure perspective, and what it means for Dynamia and the industry.

Why “Open Scheduling” Matters

In multi-cloud and hybrid cloud environments, GPU model diversity significantly amplifies operational costs. One large internet company’s platform spans H200/H100/A100/V100/4090 GPUs across five clusters. If you can only allocate “whole GPUs,” resource misalignment becomes inevitable.

“Open scheduling” isn’t a slogan—it’s a set of engineering contracts being solidified into the mainstream stack.

Standardized Resource Expression

Before: GPUs were extended resources. The scheduler didn’t understand if they represented memory, compute, or device types.

Figure 3: Open Scheduling Standardization Evolution

Now: Kubernetes DRA provides objects like DeviceClass, ResourceClaim, and ResourceSlice. This lets drivers and cluster administrators define device categories and selection logic (including CEL-based selectors), while Kubernetes handles the full loop: match devices → bind claims → place Pods onto nodes with access to allocated devices.

Even more importantly, Kubernetes 1.34 stated that core APIs in the resource.k8s.io group graduated to GA, DRA became stable and enabled by default, and the community committed to avoiding breaking changes going forward. This means the ecosystem can invest with confidence in a stable, standard API.

Standardized Device Injection

Before: Device injection relied on vendor-specific hooks and runtime class patterns.

Now: The Container Device Interface (CDI) abstracts device injection into an open specification. NVIDIA’s Container Toolkit explicitly describes CDI as an open specification for container runtimes, and NVIDIA GPU Operator 25.10.0 defaults to enabling CDI on install/upgrade—directly leveraging runtime-native CDI support (containerd, CRI-O, etc.) for GPU injection.

This means “devices into containers” is also moving toward replaceable, standardized interfaces.

On this standardization path, HAMi’s role needs redefinition: it’s not about replacing Kubernetes—it’s about turning GPU virtualization and slicing into a declarative, schedulable, governable data plane.

Data Plane Perspective

HAMi’s core contribution expands the allocatable unit from “whole GPU integers” to finer-grained shares (memory and compute), forming a complete allocation chain:

Device discovery: Identify available GPU devices and models
Scheduling placement: Use Scheduler Extender to make native schedulers “understand” vGPU resource models (Filter/Score/Bind phases)
In-container enforcement: Inject share constraints into container runtime
Metric export: Provide observable metrics for utilization, isolation, and more

This transforms “sharing” from ad-hoc “it runs” experimentation into engineering capability that can be declared in YAML, scheduled by policy, and validated by metrics.

Scheduling Mechanism: Enhancement, Not Replacement

HAMi’s scheduling doesn’t replace Kubernetes—it uses a Scheduler Extender pattern to let the native scheduler understand vGPU resource models:

Filter: Filter nodes based on memory, compute, device type, topology, and other constraints
Score: Apply configurable policies like binpack, spread, topology-aware scoring
Bind: Complete final device-to-Pod binding

This architecture positions HAMi naturally as an execution layer under higher-level “AI control planes” (queuing, quotas, priorities)—working alongside Volcano, Kueue, Koordinator, and others.

Figure 4: HAMi Scheduling Architecture (Filter → Score → Bind)

CNCF public case studies provide concrete answers: in a hybrid, multi-cloud platform built on Kubernetes and HAMi, 10,000+ Pods run concurrently, and GPU utilization improves from 13% to 37% (nearly 3×).

Figure 5: CNCF Production Case Studies: Ke Holdings 13%→37%, DaoCloud 80%+ utilization, SF Technology 57% savings

Here are highlights from several cases:

Case Study 1: Ke Holdings (February 5, 2026)

Environment: 5 clusters spanning public and private clouds
GPU models: H200/H100/A100/V100/4090 and more
Architecture: Separate “GPU clusters” for large training tasks (dedicated allocation) vs “vGPU clusters” with HAMi fine-grained memory slicing for high-density inference
Concurrent scale: 10,000+ Pods
Outcome: Overall GPU utilization improved from 13% to 37% (nearly 3×)

Case Study 2: DaoCloud (December 2, 2025)

Hard constraints: Must remain cloud-native, vendor-agnostic, and compatible with CNCF toolchain
Adoption outcomes:
- Average GPU utilization: 80%+
- GPU-related operating cost reduction: 20–30%
- Coverage: 10+ data centers, 10,000+ GPUs
Explicit benefit: Unified abstraction layer across NVIDIA and domestic GPUs, reducing vendor dependency

Case Study 3: Prep EDU (August 20, 2025)

Negative experience: Isolation failures in other GPU-sharing approaches caused memory conflicts and instability
Positive outcome: HAMi’s vGPU scheduling, GPU type/UUID targeting, and compatibility with NVIDIA GPU Operator and RKE2 became decisive factors for production adoption
Environment: Heterogeneous RTX 4070/4090 cluster

Case Study 4: SF Technology (September 18, 2025)

Project: EffectiveGPU (built on HAMi)
Use cases: Large model inference, test services, speech recognition, domestic AI hardware (Huawei Ascend, Baidu Kunlun, etc.)
Outcomes:
- GPU savings: Large model inference runs 65 services on 28 GPUs (37 saved); test cluster runs 19 services on 6 GPUs (13 saved)
- Overall savings: Up to 57% GPU savings for production and test clusters
- Utilization improvement: Up to 100% GPU utilization improvement with GPU virtualization
Highlights: Cross-node collaborative scheduling, priority-based preemption, memory over-subscription

These cases demonstrate a consistent pattern: GPU virtualization becomes economically meaningful only when it participates in a governable contract—where utilization, isolation, and policy can be expressed, measured, and improved over time.

Strategic Implications for Dynamia

From Dynamia’s perspective (and as VP of Open Source Ecosystem), the strategic value of HAMi becomes clear:

Two-Layer Architecture: Open Source vs Commercial

HAMi (CNCF open source project): Responsible for “adoption and trust,” focused on GPU virtualization and compute efficiency
Dynamia enterprise products and services: Responsible for “production and scale,” providing commercial distributions and enterprise services built on HAMi

Figure 6: Dynamia Dual Mechanism: Open Source vs Commercial

This boundary is the foundation for long-term trust—project and company offerings remain separate, with commercial distributions and services built on the open source project.

Global Narrative Strategy

The internal alignment memo recommends a bilingual approach:

First layer: Lead globally with “GPU virtualization / sharing / utilization” (Chinese can directly use “GPU virtualization and heterogeneous scheduling,” but English first layer should avoid “heterogeneous” as a headline)

Second layer: When users discuss mixed GPUs or workload diversity, introduce “heterogeneous” to confirm capability boundaries—never as the opening hook

Core anchor: Maintain “HAMi (project and community) ≠ company products” as the non-negotiable baseline for long-term positioning

The Right Commercialization Landing

DaoCloud’s case study already set vendor-agnostic and CNCF toolchain compatibility as hard constraints, framing vendor dependency reduction as a business and operational benefit—not just a technical detail. Project-HAMi’s official documentation lists “avoid vendor lock” as a core value proposition.

In this context, the right commercialization landing isn’t “closed-source scheduling”—it’s productizing capabilities around real enterprise complexity:

Systematic compatibility matrix
SLO and tail-latency governance
Metering for billing
RBAC, quotas, multi-cluster governance
Upgrade and rollback safety
Faster path-to-production for DRA/CDI and other standardization efforts

Forward View: The Next 2–3 Years

My strong judgment: over the next 2–3 years, GPU scheduling competition will shift from “whose implementation is more black-box” to “whose contract is more open.”

The reasons are practical:

Hardware Form Factors and Supply Chains Are Diversifying

OpenAI’s February 12, 2026 “GPT‑5.3‑Codex‑Spark” release emphasizes ultra-low latency serving, including persistent WebSockets and a dedicated serving tier on Cerebras hardware
Large-scale GPU-backed financing announcements (for pan-European deployments) illustrate the infrastructure scale and financial engineering surrounding accelerator fleets

These signals suggest that heterogeneity will grow: mixed accelerators, mixed clouds, mixed workload types.

Low-Latency Inference Tiers Will Force Systematic Scheduling

Low-latency inference tiers (beyond just GPUs) will force resource scheduling toward “multi-accelerator, multi-layer cache, multi-class node” architectural design—scheduling must inherently be heterogeneous.

Open Scheduling Is Risk Management, Not Idealism

In this world, “open scheduling” isn’t idealism—it’s risk management. Building schedulable governable “control plane + data plane” combinations around DRA/CDI and other solidifying open interfaces, ones that are pluggable, multi-tenant governable, and co-evolvable with the ecosystem—this looks like the truly sustainable path for AI Native Infrastructure.

The next battleground isn’t “whose scheduling is smarter”—it’s “who can standardize device resource contracts into something governable.”

Conclusion

When you place HAMi 2025 back in the broader AI Native Infrastructure context, it’s no longer just the year of “GPU sharing tools”—it’s a more structural signal: GPUs are moving toward open scheduling.

Figure 7: Open Scheduling Future Vision

The driving forces come from both ends:

Upstream: Standards like DRA/CDI continue to solidify
Downstream: Scale and diversity (multi-cloud, multi-model, even accelerators beyond GPUs)

For Dynamia, HAMi’s significance has transcended “GPU sharing tool”: it turns GPU virtualization and slicing into declarative, schedulable, measurable data planes—letting queues, quotas, priorities, and multi-tenancy actually close the governance loop.

AI Learning Resources: 44 Curated Collections from Our Cleanup

Jimmy Song — Sun, 08 Feb 2026 12:20:05 +0000

“The best way to learn AI is to start building. These resources will guide your journey.”

Figure 1: AI Learning Resources Collection

In my ongoing effort to keep the AI Resources list focused on production-ready tools and frameworks, I’ve removed 44 collection-type projects—courses, tutorials, awesome lists, and cookbooks.

These resources aren’t gone—they’ve been moved here. This post is a curated collection of those educational materials, organized by type and topic. Whether you’re a complete beginner or an experienced practitioner, you’ll find something valuable here.

Why Remove Collections from AI Resources?

My AI Resources list now focuses on concrete tools and frameworks—projects you can directly use in production. Collections, while valuable, serve a different purpose: education and discovery.

By separating them, I:

Keep the resources list actionable and focused
Create a dedicated space for learning materials
Make it easier to find what you need

📚 Awesome Lists (14 Collections)

Awesome lists are community-curated collections of the best resources. They’re perfect for discovering new tools and staying updated.

Must-Explore Awesome Lists

Awesome Generative AI

Models, tools, tutorials, and research papers
Great for: Comprehensive overview of generative AI landscape

Awesome LLM

LLM resources: papers, tools, datasets, applications
Great for: Deep dive into large language models

Awesome AI Apps

Practical LLM applications, RAG examples, agent implementations
Great for: Real-world implementation examples

Awesome Claude Code

Claude Code commands, files, and workflows
Great for: Maximizing Claude Code productivity

Awesome MCP Servers

MCP servers for modular AI backend systems
Great for: Building with Model Context Protocol

Specialized Awesome Lists

Awesome ChatGPT Prompts - Prompt examples for various scenarios
Awesome LLM Apps - LLM applications with code examples
Awesome Multimodal LLM - Multimodal model resources
Awesome MCP Clients - MCP client tools and SDKs
Awesome Claude Skills - Claude Skills and workflows
Awesome GitHub Copilot - Copilot customizations
Awesome Nano Banana Pro - Image model prompts and examples
Awesome SaaS - AI platform templates
Awesome Claude Code Subagents - Claude Code subagents

🎓 Courses & Tutorials (9 Curricula)

Structured learning paths from universities and tech companies.

Microsoft’s AI Curriculum

AI for Beginners

12 weeks, 24 lessons covering neural networks, deep learning, CV, NLP
Great for: Complete AI foundation
Format: Lessons, quizzes, projects

Machine Learning for Beginners

12-week, 26-lesson curriculum on classic ML
Great for: ML fundamentals without deep math
Format: Project-based exercises

Generative AI for Beginners

18 lessons on building GenAI applications
Great for: Practical GenAI development
Format: Hands-on projects

AI Agents for Beginners

11 lessons on agent systems
Great for: Understanding autonomous agents
Format: Project-driven learning

EdgeAI for Beginners

Optimization, deployment, and real-world Edge AI
Great for: On-device AI applications
Format: Practical tutorials

MCP for Beginners

Model Context Protocol curriculum
Great for: Building with MCP
Format: Cross-language examples and labs

Official Platform Courses

Hugging Face Learn Center

Free courses on LLMs, deep RL, CV, audio
Great for: Hands-on Hugging Face ecosystem
Format: Interactive notebooks

OpenAI Cookbook

Runnable examples using OpenAI API
Great for: OpenAI API best practices
Format: Code examples and guides

PyTorch Tutorials

Basics to advanced deep learning
Great for: PyTorch mastery
Format: Comprehensive tutorials

🍳 Cookbooks & Example Collections (5 Collections)

Practical code examples and recipes.

Claude Cookbooks

Notebooks and examples for building with Claude
Great for: Anthropic Claude integration
Format: Jupyter notebooks

Hugging Face Cookbook

Practical AI cookbook with Jupyter notebooks
Great for: Open models and tools
Format: Hands-on examples

Tinker Cookbook

Training and fine-tuning examples
Great for: Fine-tuning workflows
Format: Platform-specific recipes

E2B Cookbook

Examples for building LLM apps
Great for: LLM application development
Format: Recipes and tutorials

arXiv Paper Curator

6-week course on RAG systems
Great for: Production-ready RAG
Format: Project-based learning

📖 Guides & Handbooks (5 Resources)

In-depth guides on specific topics.

Prompt Engineering Guide

Comprehensive prompt engineering resources
Great for: Mastering prompt design
Format: Guides, papers, lectures, notebooks

Evaluation Guidebook

LLM evaluation best practices from Hugging Face
Great for: Assessing LLM performance
Format: Practical guide

Context Engineering

Design and optimize context beyond prompt engineering
Great for: Advanced context management
Format: Practical handbook

Context Engineering Intro

Template and guide for context engineering
Great for: Providing project context to AI assistants
Format: Template + guide

Vibe-Coding Workflow

5-step prompt template for building MVPs with LLMs
Great for: Rapid prototyping with AI
Format: Workflow template

🗂️ Template & Workflow Collections

Reusable templates and workflows.

Claude Code Templates

Code templates for various programming scenarios
Great for: Claude AI development
Format: Template collection

n8n Workflows

2,000+ professionally organized n8n workflows
Great for: Workflow automation
Format: Searchable catalog

N8N Workflows Catalog

Community-driven reusable workflow templates
Great for: Workflow import and versioning
Format: Template catalog

📊 Research & Evaluation

Academic and evaluation resources.

LLMSys PaperList

Curated list of LLM systems papers
Great for: Research on training, inference, serving
Format: Paper collection

Free LLM API Resources

LLM providers with free/trial API access
Great for: Experimentation without cost
Format: Provider list

🎨 Other Notable Resources

System Prompts and Models of AI Tools

Community-curated collection of system prompts and AI tool examples
Great for: Prompt and agent engineering
Format: Resource collection

ML Course CS-433

EPFL Machine Learning Course
Great for: Academic ML foundation
Format: Lectures, labs, projects

Machine Learning Engineering

ML engineering open-book: compute, storage, networking
Great for: Production ML systems
Format: Comprehensive guide

Realtime Phone Agents Course

Build low-latency voice agents
Great for: Voice AI applications
Format: Hands-on course

LLMs from Scratch

Build a working LLM from first principles
Great for: Understanding LLM internals
Format: Repository + book materials

💡 How to Use This Collection

For Complete Beginners

Start with: Microsoft’s AI for Beginners
Practice with: PyTorch Tutorials
Explore: Awesome AI Apps for inspiration

For Developers

Build skills: OpenAI Cookbook + Claude Cookbooks
Find tools: Awesome Generative AI + Awesome LLM
Learn workflows: n8n Workflows Catalog

For Researchers

Stay updated: Awesome Generative AI + LLMSys PaperList
Deep dive: Awesome LLM
Implement: Hugging Face Cookbook

For Product Builders

Find examples: Awesome AI Apps
Learn workflows: n8n Workflows Catalog
Study patterns: Awesome LLM Apps

🔄 What Was NOT Removed

Agent frameworks and production tools remain in the AI Resources list, including:

AutoGen - Microsoft’s multi-agent framework
CrewAI - High-performance multi-agent orchestration
LangGraph - Stateful multi-agent applications
Flowise - Visual agent platform
Langflow - Visual workflow builder
And 80+ more agent frameworks

These are functional tools you can use to build applications, not educational collections. They belong in the AI Resources list.

📝 Summary

I removed 44 collection-type projects from the AI Resources list to keep it focused on production tools:

14 Awesome Lists - Discover new tools and stay updated
9 Courses & Tutorials - Structured learning paths
5 Cookbooks - Practical code examples
5 Guides & Handbooks - In-depth resources
4 Template Collections - Reusable workflows
7 Other Resources - Research and evaluation

These resources remain incredibly valuable for learning and discovery. They just serve a different purpose than the production-focused tools in my AI Resources list.

Next Steps:

Bookmark this post for future reference
Explore the AI Resources list for production tools (agent frameworks, databases, etc.)
Check out my blog for more AI engineering insights

Acknowledgments: This collection was compiled during my AI Resources cleanup initiative. Special thanks to all the maintainers of these awesome lists, courses, and collections for their invaluable contributions to the AI community.

Standing on Giants' Shoulders: The Traditional Infrastructure Powering Modern AI

Jimmy Song — Sun, 08 Feb 2026 08:00:00 +0000

“If I have seen further, it is by standing on the shoulders of giants.” — Isaac Newton

Figure 1: Standing on Giants’ Shoulders: The Traditional Infrastructure Powering Modern AI

In the excitement surrounding LLMs, vector databases, and AI agents, it’s easy to forget that modern AI didn’t emerge from a vacuum. Today’s AI revolution stands upon decades of infrastructure work—distributed systems, data pipelines, search engines, and orchestration platforms that were built long before “AI Native” became a buzzword.

This post is a tribute to those traditional open source projects that became the invisible foundation of AI infrastructure. They’re not “AI projects” per se, but without them, the AI revolution as we know it wouldn’t exist.

The Evolution: From Big Data to AI

Era	Focus	Core Technologies	AI Connection
2000s	Web Search & Indexing	Lucene, Elasticsearch	Semantic search foundations
2010s	Big Data & Distributed Computing	Hadoop, Spark, Kafka	Data processing at scale
2010s	Cloud Native	Docker, Kubernetes	Model deployment platforms
2010s	Stream Processing	Flink, Storm, Pulsar	Real-time ML inference
2020s	AI Native	Transformers, Vector DBs	Built on everything above

Table 1: Evolution of Data Infrastructure

Big Data Frameworks: The Data Engines

Before we could train models on petabytes of data, we needed ways to store, process, and move that data.

Apache Hadoop (2006)

GitHub: https://github.com/apache/hadoop

Hadoop democratized big data by making distributed computing accessible. Its HDFS filesystem and MapReduce paradigm proved that commodity hardware could process web-scale datasets.

Why it matters for AI:

Modern ML training datasets live in HDFS-compatible storage
Data lakes built on Hadoop became training data reservoirs
Proved that distributed computing could scale horizontally

Apache Kafka (2011)

GitHub: https://github.com/apache/kafka

Kafka redefined data streaming with its log-based architecture. It became the nervous system for real-time data flows in enterprises worldwide.

Why it matters for AI:

Real-time feature pipelines for ML models
Event-driven architectures for AI agent systems
Streaming inference pipelines
Model telemetry and monitoring backbones

Apache Spark (2014)

GitHub: https://github.com/apache/spark

Spark brought in-memory computing to big data, making iterative algorithms (like ML training) practical at scale.

Why it matters for AI:

MLlib made ML accessible to data engineers
Distributed data processing for model training
Spark ML became the de facto standard for big data ML
Proved that in-memory computing could accelerate ML workloads

Search Engines: The Retrieval Foundation

Before RAG (Retrieval-Augmented Generation) became a buzzword, search engines were solving retrieval at scale.

Elasticsearch (2010)

GitHub: https://github.com/elastic/elasticsearch

Elasticsearch made full-text search accessible and scalable. Its distributed architecture and RESTful API became the standard for search.

Why it matters for AI:

pioneered distributed inverted index structures
Proved that horizontal scaling was possible for search workloads
Many “AI search” systems actually use Elasticsearch under the hood
Query DSL influenced modern vector database query languages

OpenSearch (2021)

GitHub: https://github.com/opensearch-project/opensearch

When AWS forked Elasticsearch, it ensured search infrastructure remained truly open. OpenSearch continues the mission of accessible, scalable search.

Why it matters for AI:

Maintains open source innovation in search
Vector search capabilities added in 2023
Demonstrates community fork resilience

Databases: From SQL to Vectors

The evolution from relational databases to vector databases represents a paradigm shift—but both have AI relevance.

Traditional Databases That Paved the Way

Dgraph (2015) - Graph database proving that specialized data structures enable new use cases
TDengine (2019) - Time-series database for IoT ML workloads
OceanBase (2021) - Distributed database showing ACID transactions could scale

Why they matter for AI:

Proved that specialized database engines could outperform general-purpose ones
Database internals (indexing, sharding, replication) are now applied to vector databases
Multi-model databases (graph + vector + relational) are becoming the norm for AI apps

Cloud Native: The Runtime Foundation

When Docker and Kubernetes emerged, they weren’t built for AI—but AI couldn’t scale without them.

Docker (2013) & Kubernetes (2014)

GitHub: https://github.com/kubernetes/kubernetes

Kubernetes became the operating system for cloud-native applications. Its declarative API and controller pattern made it perfect for AI workloads.

Why it matters for AI:

Model deployment platforms (KServe, Seldon Core) run on K8s
GPU orchestration (NVIDIA GPU Operator, Volcano, HAMi) extends K8s
Kubeflow made K8s the standard for ML pipelines
Microservice patterns enable modular AI agent architectures

Service Mesh & Serverless

Istio (2016), Knative (2018) - Service mesh and serverless platforms that proved:

Network-level observability applies to AI model calls
Scale-to-zero is essential for cost-effective inference
Traffic splitting enables A/B testing of ML models

Why they matter for AI:

AI Gateway patterns evolved from API gateways + service mesh
Serverless inference platforms use Knative-style autoscaling
Observability patterns (tracing, metrics) are now standard for ML systems

API Gateways: From REST to LLM

API gateways weren’t designed for AI, but they became the foundation of AI Gateway patterns.

Kong, APISIX, KGateway

These API gateways solved rate limiting, auth, and routing at scale. When LLMs emerged, the same patterns applied:

AI Gateway Evolution:

Traditional API Gateway (2010s)
 ↓
Rate Limiting → Token Bucket Rate Limiting
Auth → API Key + Organization Management
Routing → Model Routing (GPT-4 → Claude → Local Models)
Observability → LLM-specific Telemetry (token usage, cost)
 ↓
AI Gateway (2024)

Why they matter for AI:

Proved that centralized API management scales
Plugin architectures enable LLM-specific features
Traffic management patterns apply to prompt routing
Security patterns (mTLS, JWT) now protect AI endpoints

Workflow Orchestration: The Pipeline Backbone

Data engineering needs pipelines. ML engineering needs pipelines. AI agents need workflows.

Apache Airflow (2015)

GitHub: https://github.com/apache/airflow

Airflow made pipeline orchestration accessible with its DAG-based approach. It became the standard for ETL and data engineering.

Why it matters for AI:

ML pipeline orchestration (feature engineering, training, evaluation)
Proved that DAG-based workflow definition works at scale
Prompt engineering pipelines use Airflow-style orchestration
Scheduler patterns are now applied to AI agent workflows

n8n, Prefect, Flyte

Modern workflow platforms that evolved from Airflow’s foundations:

n8n (2019) - Visual workflow automation with AI capabilities
Prefect (2018) - Python-native workflow orchestration for ML
Flyte (2019) - Kubernetes-native workflow orchestration for ML/data

Why they matter for AI:

Multi-modal agents need workflow orchestration
RAG pipelines are essentially ETL pipelines for embeddings
Prompt chaining is DAG-based orchestration

Data Formats: The Lakehouse Foundation

Before we could train on massive datasets, we needed formats that supported ACID transactions and schema evolution.

Delta Lake, Apache Iceberg, Apache Hudi

These table formats brought reliability to data lakes:

Why they matter for AI:

Training datasets need versioning and reproducibility
Feature stores use Delta/Iceberg as storage formats
Proved that “big data” could have transactional semantics
Schema evolution handles ML feature drift

The Invisible Thread: Why These Projects Matter

What do all these projects have in common?

They solved scaling first - AI training/inference needs horizontal scaling
They proved distributed systems work - Modern AI is fundamentally distributed
They created ecosystem patterns - Plugin systems, extension points, APIs
They established best practices - Observability, security, CI/CD
They built developer habits - YAML configs, declarative APIs, CLI tools

The AI Native Continuum

Modern “AI Native” infrastructure didn’t replace these projects—it builds on them:

Traditional Project	AI Native Evolution	Example
Hadoop HDFS	Distributed model storage	HDFS for datasets, S3 for checkpoints
Kafka	Real-time feature pipelines	Kafka → Feature Store → Model Serving
Spark ML	Distributed ML training	MLlib → PyTorch Distributed
Elasticsearch	Vector search	ES → Weaviate/Qdrant/Milvus
Kubernetes	ML orchestration	K8s → Kubeflow/KServe
Istio	AI Gateway service mesh	Istio → LLM Gateway with mTLS
Airflow	ML pipeline orchestration	Airflow → Prefect/Flyte for ML

Table 2: From Traditional to AI Native

Why We’re Removing Them from AI Resources List

This post honors these projects, but we’re also removing them from our AI Resources list. Here’s why:

They’re not “AI Projects”—they’re foundational infrastructure.

Hadoop, Kafka, Spark are data engineering tools, not ML frameworks
Elasticsearch is search, not semantic search
Kubernetes is general-purpose orchestration
API gateways serve REST/GraphQL, not just LLMs

But their absence doesn’t diminish their importance.

By removing them, we acknowledge that:

AI has its own ecosystem - Transformers, vector DBs, LLM ops
Traditional infra has its own domain - Data engineering, cloud native
The intersection is where innovation happens - AI-native data platforms, LLM ops on K8s

The Giants We Stand On

The next time you:

Deploy a model on Kubernetes
Stream features through Kafka
Search embeddings with a vector database
Orchestrate a RAG pipeline with Prefect

Remember: You’re standing on the shoulders of Hadoop, Kafka, Elasticsearch, Kubernetes, and countless others. They built the roads we now drive on.

The Future: Building New Giants

Just as Hadoop and Kafka enabled modern AI, today’s AI infrastructure will become tomorrow’s foundation:

Vector databases may become the new standard for all search
LLM observability may evolve into general distributed tracing
AI agent orchestration may reinvent workflow automation
GPU scheduling may influence general-purpose resource management

The cycle continues. The giants of today will be the foundations of tomorrow.

Conclusion: Gratitude and Continuity

As we clean up our AI Resources list to focus on AI-native projects, we don’t forget where we came from. Traditional big data and cloud native infrastructure made the AI revolution possible.

To the Hadoop committers, Kafka maintainers, Kubernetes contributors, and all who built the foundation: Thank you.

Your work enabled ChatGPT, enabled Transformers, enabled everything we now call “AI.”

Standing on your shoulders, we see further.

Acknowledgments: This post was inspired by the need to refactor our AI Resources list. The 27 projects mentioned here are being removed—not because they’re unimportant, but because they deserve their own category: The Foundation.

My First Month at Dynamia: Why AI Native Infra Is Worth the Investment

Jimmy Song — Fri, 06 Feb 2026 12:56:35 +0000

Time flies—it’s already been a month since I joined Dynamia. In this article, I want to share my observations from this past month: why AI Native Infra is a direction worth investing in, and some considerations for those thinking about their own career or technical direction.

Introduction

After nearly five years of remote work, I officially joined Dynamia last month as VP of Open Source Ecosystem. This decision was not sudden, but a natural extension of my journey from cloud native to AI Native Infra.

But this article is not just about my personal choice. I want to answer a more universal question: In the wave of AI infrastructure startups, why is compute governance a direction worth investing in?

For the past decade, I have worked continuously in the infrastructure space: from Kubernetes to Service Mesh, and now to AI Infra. I am increasingly convinced that the core challenge in the AI era is not “can the model run,” but “can compute resources be run efficiently, reliably, and in a controlled manner.” This conviction has only grown stronger through my observations and reflections during this first month at Dynamia.

This article answers three questions: What is AI Native Infra? Why is GPU virtualization a necessity? Why did I choose Dynamia and HAMi?

What Is AI Native Infra

The core of AI Native Infrastructure is not about adding another platform layer, but about redefining the governance target: expanding from “services and containers” to “model behaviors and compute assets.”

I summarize it as three key shifts:

Models as execution entities: Governance now includes not just processes, but also model behaviors.
Compute as a scarce asset: GPU, memory, and bandwidth must be scheduled and metered precisely.
Uncertainty as the default: Systems must remain observable and recoverable amid fluctuations.

In essence, AI Native Infra is about upgrading compute governance from “resource allocation” to “sustainable business capability.”

Why GPU Virtualization Is Essential

Many teams focus on model inference optimization, but in production, enterprises first encounter the problem of “underutilized GPUs.” This is where GPU virtualization delivers value.

Structural idleness: Small tasks monopolize large GPUs, leaving them idle for long periods.
Pseudo-isolation risks: Native sharing lacks hard boundaries, so a single task OOM can cause cascading failures.
Scheduling failures: Some users queue for GPUs while others occupy but do not use them, leading to both shortages and idleness.
Fragmentation waste: There may be enough total GPU, but not enough full cards, making efficient packing impossible.
Vendor lock-in anxiety: Proprietary, tightly coupled solutions make migration costs uncontrollable.

In short: GPUs must not only be allocatable, but also splittable, isolatable, schedulable, and governable.

The Relationship Between HAMi and Dynamia

This is the most frequently asked question. Here is the shortest answer:

HAMi: A CNCF-hosted open source project and community focused on GPU virtualization and heterogeneous compute scheduling.
Dynamia: The founding and leading company behind HAMi, providing enterprise-grade products and services based on HAMi.

Open source projects are not the same as company products, but the two evolve together. HAMi drives industry adoption and technical trust, while Dynamia brings these capabilities into enterprise production environments at scale. This “dual engine” approach is what makes Dynamia unique.

What HAMi Provides

HAMi (Heterogeneous AI Computing Virtualization Middleware) delivers three key capabilities on Kubernetes:

Virtualization and partitioning: Split physical GPUs into logical resources on demand to improve utilization.
Scheduling and topology awareness: Place workloads optimally based on topology to reduce communication bottlenecks.
Isolation and observability: Support quotas, policies, and monitoring to reduce production risks.

Currently, HAMi has attracted over 360 contributors from 16 countries, with more than 200 enterprise end users, and its international influence continues to grow.

Market Trends: The AI Infrastructure Startup Wave

AI infrastructure is experiencing a new wave of startups. The vLLM team’s company raised $150 million, SGLang’s commercial spin-off RadixArk is valued at $4 billion, and Databricks acquired MosaicML for $1.3 billion—all pointing to a consensus: Whoever helps enterprises run large models more efficiently and cost-effectively will hold the keys to next-generation AI infrastructure.

Against this backdrop, the positioning of Dynamia and HAMi is even clearer. Many teams focus on “model performance acceleration” and “inference optimization” (like vLLM, SGLang), while we focus on “resource scheduling and virtualization”—enabling better orchestration of existing accelerated hardware resources.

The two are complementary: the former makes individual models run faster and cheaper, while the latter ensures that compute allocation at the cluster level is efficient, fair, and controllable. This is similar to extending Kubernetes’ CPU/memory scheduling philosophy to GPU and heterogeneous compute management in the AI era.

Why AI Native Infra Is Worth the Investment

My observations this month have convinced me that compute governance is the most undervalued yet most promising area in AI infrastructure. If you are considering a career or technical investment, here is my assessment:

First, this is a real and urgent pain point

Model training and inference optimization attract a lot of attention, but in production, enterprises first encounter the problem of “underutilized GPUs”—structural idleness, scheduling failures, fragmentation waste, and vendor lock-in anxiety. Without solving these problems, even the fastest models cannot scale in production. GPU virtualization and heterogeneous compute scheduling are the “infrastructure below infrastructure” for enterprise AI transformation.

Second, this is a clear long-term track

Frameworks like vLLM and SGLang emerge constantly, making individual models run faster. But who ensures that compute allocation at the cluster level is efficient, fair, and controllable? This is similar to extending Kubernetes’ success in CPU/memory scheduling to GPU and heterogeneous compute management in the AI era. This is not something that can be finished in a year or two, but a direction for continuous construction over the next five to ten years.

Third, this is an open and verifiable path

Dynamia chose to build on HAMi as an open source foundation, first solving general capabilities, then supporting enterprise adoption. This means the technical direction is transparent and verifiable in the community. You can form your own judgment by participating in open source, observing adoption, and evaluating the ecosystem—rather than relying on the black-box promises of proprietary solutions.

Fourth, this is a window of opportunity that is opening now

AI infrastructure is being redefined. Investing in its construction today will continue to yield value in the coming years. The vLLM team’s company raised $150 million, SGLang’s commercial spin-off RadixArk is valued at $4 billion, Databricks acquired MosaicML for $1.3 billion—all validating the same trend: Whoever helps enterprises run large models more efficiently will hold the keys to next-generation AI infrastructure.

I hope to bring my experience in cloud native and open source communities to the next stage of HAMi and Dynamia: turning GPU resources from a “cost center” into an “operational asset.” This is not just my career choice, but my judgment and investment in the direction of next-generation infrastructure.

Join the HAMi Community

Add me on WeChat (jimmysong) to join the HAMi community focused on GPU virtualization and heterogeneous compute scheduling.

If you are also interested in HAMi, GPU virtualization, AI Native Infra, or Dynamia, feel free to reach out.

Summary

From cloud native to AI Native Infra, my observations this month have only strengthened my conviction: The true upper limit of AI applications is determined by the infrastructure’s ability to govern compute resources.

HAMi addresses the fundamental issues of GPU virtualization and heterogeneous compute scheduling, while Dynamia is driving these capabilities into large-scale production. If you are also looking for a technical direction worth long-term investment, AI Native Infra—especially compute governance and scheduling—is a track with real pain points, a clear path, an open ecosystem, and an opening window of opportunity.

Joining Dynamia is not just a career choice, but a commitment to building the next generation of infrastructure. I hope the observations and reflections in this article can provide some reference for you as you evaluate technical directions and career opportunities.

If you are also interested in HAMi, GPU virtualization, AI Native Infra, or Dynamia, feel free to reach out.

The True Inflection Point of ADD: When Spec Becomes the Core Asset of AI-Era Software

Jimmy Song — Tue, 20 Jan 2026 07:51:36 +0000

The role of Spec is undergoing a fundamental transformation, becoming the governance anchor of engineering systems in the AI era.

The Essence of Software Engineering and the Cost Structure Shift Brought by AI

From first principles, software engineering has always been about one thing: stably, controllably, and reproducibly transforming human intent into executable systems.

Artificial Intelligence (AI) does not change this engineering essence, but it dramatically alters the cost structure:

Implementation costs plummet: Code, tests, and boilerplate logic are rapidly commoditized.
Consistency costs rise sharply: Intent drift, hidden conflicts, and cross-module inconsistencies become more frequent.
Governance costs are amplified: As agents can act directly, auditability, accountability, and explainability become hard constraints.

Therefore, in the era of Agent-Driven Development (ADD), the core issue is not “can agents do the work,” but how to maintain controllability and intent preservation in engineering systems under highly autonomous agents.

The ADD Era Inflection Point: Three Structural Preconditions

Many attribute the “explosion” of ADD to more mature multi-agent systems, stronger models, or more automated tools. In reality, the true structural inflection point arises only when these three conditions are met:

Agents have acquired multi-step execution capabilities

With frameworks like LangChain, LangGraph, and CrewAI, agents are no longer just prompt invocations, but long-lived entities capable of planning, decomposition, execution, and rollback.

Agents are entering real enterprise delivery pipelines

Once in enterprise R&D, the question shifts from “can it generate” to “who approved it, is it compliant, can it be rolled back.”

Traditional engineering tools lack a control plane for the agent era

Tools like Git, CI, and Issue Trackers were designed for “human developer collaboration,” not for “agent execution.”

When these three factors converge, ADD inevitably shifts from an “efficiency tool” to a “governance system.”

The Changing Role of Spec: From Documentation to System Constraint

In the context of ADD, Spec is undergoing a fundamental shift:

Spec is no longer “documentation for humans,” but “the source of constraints and facts for systems and agents to execute.”

Spec now serves at least three roles:

Verifiable expression of intent and boundaries

Requirements, acceptance criteria, and design principles are no longer just text, but objects that can be checked, aligned, and traced.

Stable contracts for organizational collaboration

When agents participate in delivery, verbal consensus and tacit knowledge quickly fail. Versioned, auditable artifacts become the foundation of collaboration.

Policy surface for agent execution

Agents can write code, modify configurations, and trigger pipelines. Spec must become the constraint on “what can and cannot be done.”

From this perspective, the status of Spec is approaching that of the Control Plane in AI-native infrastructure.

The Reality of Multi-Agent Workflows: Orchestration and Governance First

In recent systems (such as APOX and other enterprise products), an industry consensus is emerging:

Multi-agent collaboration no longer pursues “full automation,” but is staged and gated.
Frameworks like LangGraph are used to build persistent, debuggable agent workflows.
RAG (e.g., based on Milvus) is used to accumulate historical Specs, decisions, and context as long-term memory.
The IDE mainly focuses on execution efficiency, not engineering governance.

Figure 1: APOX user interface

APOX (AI Product Orchestration eXtended) is a multi-agent collaboration workflow platform for enterprise software delivery. Its core goals are:

To connect the entire process from product requirements to executable code with a governable Agentflow and explicit engineering artifact chain.
To assign dedicated AI agents to each delivery stage (such as PRD, PO, Architecture, Developer, Implementation, Coding, etc.).
To embed manual approval gates and full audit trails at every step, solving the “intent drift and consistency” governance problem that traditional AI coding tools cannot address.
The platform provides a VS Code plugin for real-time sync between local IDE and web artifacts, allowing Specs, code, tasks, and approval statuses to coexist in the repository.
Supports assigning different base models to different agents according to enterprise needs.

APOX is not about simply speeding up code generation, but about elevating “Spec” from auxiliary documentation to a verifiable, constrainable, and traceable core asset in engineering—building a control plane and workflow governance system suitable for Agent-Driven Development.

Such systems emphasize:

An explicit artifact chain from PRD → Spec → Task → Implementation.
Manual confirmation and audit points at every stage.
Bidirectional sync between Spec, code, repository, and IDE.

This is not about “smarter AI,” but about engineering systems adapting to the agent era.

The Long-Term Value of Spec: The Core Anchor of Engineering Assets

This is not to devalue code, but to acknowledge reality:

There will always be long-term differentiation in algorithms and model capabilities.
General engineering implementation is rapidly homogenizing.
What is hard to replicate is: how to define problems, constrain systems, and govern change.

In the ADD era, the value of Spec is reflected in:

Determining what agents can and cannot do.
Carrying the organization’s long-term understanding of the system.
Serving as the anchor for audit, compliance, and accountability.

Code will be rewritten again and again; Spec is the long-term asset.

Risks and Challenges of ADD: Living Spec and Governance Constraints

ADD also faces significant risks:

Can Spec become a Living Spec

That is, when key implementation changes occur, can the system detect “intent changes” and prompt Spec updates, rather than allowing silent drift?

Can governance achieve low friction but strong constraints

If gates are too strict, teams will bypass them; if too loose, the system loses control.

These two factors determine whether ADD is “the next engineering paradigm” or “just another tool bubble.”

The Trend Toward Control Planes in Engineering Systems

From a broader perspective, ADD is the inevitable result of engineering systems becoming “control planes”:

Engineering systems are evolving from “human collaboration tools” to “control systems for agent execution.”

In this structure:

Agent / IDE is the execution plane.
RAG / Memory is the state and memory plane.
Spec is the intent and policy plane.
Gates, audit, and traceability form the governance loop.

This closely aligns with the evolution path of AI-native infrastructure.

Summary

The winners of the ADD era will not be the systems with “the most agents or the fastest generation,” but those that first upgrade Spec from documentation to a governable, auditable, and executable asset. As automation advances, the true scarcity is the long-term control of intent.

AI Voice Dictation Input Methods Are Becoming the New Shortcut Key for the Programming Era

Jimmy Song — Sun, 18 Jan 2026 06:53:08 +0000

Voice input methods are not just about being “fast”—they are becoming a brand new gateway for developers to collaborate with AI.

Warning

On January 12, 2026, due to financial difficulties encountered during operations, the Miaoyan project announced the cessation of operations and the team was disbanded. The application will no longer be updated or maintained, but existing versions can continue to be used on the current device and system, and do not store any audio or transcription content.

Figure 1: Can voice input become the new shortcut for developers? My in-depth comparison experience.

AI Voice Input Methods Are Becoming the “New Shortcut Key” in the Programming Era

I am increasingly convinced of one thing: PC-based AI voice input methods are evolving from mere “input tools” into the foundational interaction layer for the era of programming and AI collaboration.

It’s not just about typing faster—it determines how you deliver your intent to the system, whether you’re writing documentation, code, or collaborating with AI in IDEs, terminals, or chat windows.

Because of this, the differences in voice input method experiences are far more significant than they appear on the surface.

My Six Evaluation Criteria for AI Voice Input Methods

After long-term, high-frequency use, I have developed a set of criteria to assess the real-world performance of AI voice input methods:

Response speed: Does text appear quickly enough after pressing the shortcut to keep up with your thoughts?
Continuous input stability: Does it remain reliable during extended use, or does it suddenly fail or miss recognition?
Mixed Chinese-English and technical terms: Can it reliably handle code, paths, abbreviations, and product names?
Developer friendliness: Is it truly designed for command line, IDE, and automation scenarios?
Interaction restraint: Does it avoid introducing distracting features that interfere with input itself?
Subscription and cost structure: Is it a standalone paid product, or can it be bundled with existing tool subscriptions?

Based on these criteria, I focused on comparing Miaoyan, Shandianshuo, and Zhipu AI Voice Input Method.

Miaoyan: Currently the Most “Developer-Oriented” Domestic Product

Miaoyan was the first domestic AI voice input method I used extensively, and it remains the one I am most willing to use continuously.

Figure 2: Miaoyan is currently my most-used Mac voice input method.

Command Mode: The Key Differentiator for Developer Productivity

It’s important to clarify that Miaoyan’s command mode is not about editing text via voice. Instead:

You describe your need in natural language, and the system directly generates an executable command-line command.

This is crucial for developers:

It’s not just about input
It’s about turning voice into an automation entry point
Essentially, it connects voice to the CLI or toolchain

This design is clearly focused on engineering efficiency, not office document polishing.

Usage Experience Summary

Fast response, nearly instant
Output is relatively clean, with minimal guessing
Interaction design is restrained, with no unnecessary concepts
Developer-friendly mindset

But there are some practical limitations:

It is a completely standalone product
Requires a separate subscription
Still in relatively small-scale use

From a product strategy perspective, it feels more like a “pure tool” than part of an ecosystem.

Note

Shandianshuo: Local-First Approach, Developer Experience Depends on Your Setup

Shandianshuo takes a different approach: it treats voice input as a “local-first foundational capability,” emphasizing low latency and privacy (at least in its product narrative). The natural advantages of this approach are speed and controllable marginal costs, making it suitable as a “system capability” that’s always available, rather than a cloud service.

Figure 3: Shandianshuo settings page

However, from a developer’s perspective, its upper limit often depends on “how you implement enhanced capabilities”:

If you only use it for basic transcription, the experience is more like a high-quality local input tool. But if you want better mixed Chinese-English input, technical term correction, symbol and formatting handling, the common approach is to add optional AI correction/enhancement capabilities, which usually requires extra configuration (such as providing your own API key or subscribing to enhanced features). The key trade-off here is not “can it be used,” but “how much configuration cost are you willing to pay for enhanced capabilities.”

If you want voice input to be a “lightweight, stable, non-intrusive” foundation, Shandianshuo is worth considering. But if your goal is to make voice input part of your developer workflow (such as command generation or executable actions), it needs to offer stronger productized design at the “command layer” and in terms of controllability.

Zhipu AI Voice Input Method: Stable but with Friction

I also thoroughly tested the Zhipu AI Voice Input Method.

Figure 4: Zhipu Voice Input Method settings interface

Its strengths include:

More stable for long-term continuous input
Rarely becomes completely unresponsive
Good tolerance for longer Chinese input

But with frequent use, some issues stand out:

Idle misrecognition: If you press the shortcut but don’t speak, it may output random characters, disrupting your input flow
Occasionally messy output: Sometimes adds irrelevant words, making it less controllable than Miaoyan
Basic recognition errors: For example, “Zhipu” being recognized as “Zhipu” (with a different character), which is a trust issue for professional users
Feature-heavy design: Various tone and style features increase cognitive load

Subscription Bundling: Zhipu’s Practical Advantage

Although I prefer Miaoyan in terms of experience, Zhipu has a very practical advantage:

If you already subscribe to Zhipu’s programming package, the voice input method is included for free.

This means:

No need to pay separately for the input method
Lower psychological and decision-making cost
More likely to become the “default tool” that stays

From a business perspective, this is a very smart strategy.

Main Comparison Table

The following table compares the three products across key dimensions for quick reference.

Dimension	Miaoyan	Shandianshuo	Zhipu AI Voice Input Method
Response Speed	Fast, nearly instant	Usually fast (local-first)	Slightly slower than Miaoyan
Continuous Stability	Stable	Depends on setup and environment	Very stable
Idle Misrecognition	Rare	Generally restrained (varies by version)	Obvious: outputs characters even if silent
Output Cleanliness/Control	High	More like an “input tool”	Occasionally messy
Developer Differentiator	Natural language → executable command	Local-first / optional enhancements	Ecosystem-attached capabilities
Subscription & Cost	Standalone, separate purchase	Basic usable; enhancements often require setup/subscription	Bundled free with programming package
My Current Preference	Best experience	More like a “foundation approach”	Easy to keep but not clean enough

Table 1: Core Comparison of Miaoyan, Shandianshuo, and Zhipu AI Voice Input Methods

User Loyalty to AI Voice Input Methods

The switching cost for voice input methods is actually low: just a shortcut key and a habit of output.

What really determines whether users stick around is:

Whether the output is controllable
Whether it keeps causing annoying minor issues
Whether it integrates into your existing workflow and payment structure

For me personally:

The best and smoothest experience is still Miaoyan
The one most likely to stick around is probably Zhipu
Shandianshuo is more of a “foundation approach” and worth watching for how its enhancements evolve

These points are not contradictory.

Summary

Miaoyan is more mature in engineering orientation, command capabilities, and input control
Zhipu has practical advantages in stability and subscription bundling
Shandianshuo takes a local-first + optional enhancement approach, with the key being how it balances “basic capability” and “enhancement cost”
Who truly becomes the “default gateway” depends on reducing distractions, fixing frequent minor issues, and treating voice input as true “infrastructure” rather than an add-on feature

The competition among AI voice input methods is no longer about recognition accuracy, but about who can own the shortcut key you press every day.

From Spatial Data to AI Open Source: Technical Standards, Data Sovereignty, and the Global Divide

Jimmy Song — Sun, 11 Jan 2026 03:29:28 +0000

The divide in technical standards and data sovereignty determines the global competitive landscape of infrastructure open source in the AI era.

In this article, I will use the differences in air quality data presentation in Apple Maps and Weather as a starting point to explore how technical standards and data sovereignty influence the open source paths of AI in different countries. I will further analyze why, in the AI era, infrastructure-level open source has become the key battleground for ecosystem dominance.

Author’s Note

This article originates from a very everyday observation: Why is air quality data in China shown as “points” in Apple Maps and Weather, while in other countries it is often displayed as “areas”?

Figure 1: Air quality map in Apple Weather, showing point-based data in China and area-based data in other countries

At first glance, it seems like a product experience difference. But when I reconsidered this issue in the context of engineering, standards, and system design, I realized it actually points to a much bigger question: how different countries understand the relationship between technology, standards, openness, and sovereignty.

As an engineer who has long worked in cloud native, AI infrastructure, and open source ecosystems, I gradually realized that this difference is not limited to air quality or map data. In the AI era, it is further amplified, directly affecting how we open source models, build infrastructure, and whether we can participate in the formulation of global rules.

Writing this article is not about judging right or wrong, but about using a concrete example to explain a structural difference and discuss the long-term impact and real opportunities this difference may bring in the AI era.

What is especially important: at the level of AI infrastructure and infra-level open source, the competition has just begun. China is not without opportunities, but the choice of path will become more critical than ever.

Differences in Air Quality Data Presentation: A Microcosm of Technical Standards and Sovereignty

The following image illustrates the divide between spatial data, AI open source, and technical standards. By comparing how air quality data is presented in Apple Maps and Weather in different countries, you can intuitively feel the differences in technical standards and sovereignty strategies behind the scenes.

Figure 2: The divide between spatial data, AI open source, and technical standards

If you regularly use global products such as maps, weather, traffic, or various data services, you may notice a recurring phenomenon that is rarely discussed seriously: the way data is presented in China often differs significantly from global mainstream standards.

A very intuitive example comes from the air quality display in Apple Maps or Weather. In China, air quality is usually shown as discrete points; in the US, Europe, Japan, and other countries, it is often rendered as continuous coverage areas.

At first glance, this seems like a product experience difference, and may even lead people to mistakenly believe that “China’s data is incomplete.” But if you treat it as an engineering or system design issue, you will find: this is not a matter of data capability, but a different choice in technical standards, data sovereignty, and openness strategies.

And this choice is not limited to air quality.

Air Quality Is Just a Slice: Greater Differences in Spatial Public Data

Air quality is just a highly visible and relatively low-risk example. Similar differences have long existed in broader spatial and public data domains.

Maps and coordinate systems
Surveying and high-precision spatial data
Real-time traffic and population movement
Remote sensing, environmental, and urban operation data

In global mainstream systems, such data is usually regarded as public information infrastructure. It is standardized, gridded, API-ified, allows interpolation, modeling, and redistribution, and is widely used in research, business, and product innovation.

In China, this data often takes another form: hierarchical, discrete, strictly defined, and with centralized interpretation authority.

This is not a technical preference in a single field, but a systemic logic of technology and governance.

Three Global Paths

Placing China in a global context, we can see that there are roughly three different paths worldwide regarding “how public data and technical standards are opened.”

Engineering-Open Type: Standards and Ecosystem First

Represented by the US and some European countries, the core features of this system are:

Public data prioritized as infrastructure
Standards and interfaces come first
Encourages engineering autonomy and ecosystem evolution
Tolerates model inference and uncertainty

This path directly shaped the global landscape of foundational software and infrastructure-level open source. Linux, Kubernetes, and the cloud native system are essentially products of openness at the rules layer.

Governance-Sovereignty Type: Control and Auditability First

Represented by China, this path emphasizes:

Sensitivity of spatial and public data
Data as part of governance capability
Standards, definitions, and release methods are highly bound
Emphasizes traceability, accountability, and controllability

In this system, “point data” is not a sign of technological backwardness, but a governable technical form. When a technical system is designed as a governance system, its primary goal is not reusability, but controllability.

Compromise-Coordinated Type: Cautious Openness, Engineering Internationalization

Some countries try to find a balance between the two, maintaining caution in spatial data while being highly internationalized in engineering and industry. This shows that the difference is not about being advanced or backward, but about different objective functions.

The following diagram compares the core characteristics, typical cases, and advantages/challenges of these three paths from a global perspective. The “Engineering-Open Type” on the left shapes the global infrastructure software landscape through standards and ecosystems; the “Governance-Sovereignty Type” in the middle emphasizes data sovereignty and security controllability but has limitations in influence at the rules layer; the “Compromise-Coordinated Type” on the right attempts to find a balance between security and openness. The divide between these three paths directly affects the infrastructure competition landscape of various countries in the AI era.

Figure 3: Global Perspective: Three Paths for Public Data and Technical Standards

The Essence of “Point” vs. “Area” in Air Quality

Among all spatial public data, air quality is an ideal observation window:

Does not directly involve military or core economic security
Highly visible, updated daily, and perceptible to everyone

China does not lack air quality data; on the contrary, the density of monitoring stations is among the highest in the world. The real difference lies in:

Whether interpolation is allowed
Whether model inference is allowed
Whether platforms are allowed to reinterpret the data

“Point” means authenticity and traceability; “area” means models, inference, and redistribution of interpretive authority. This is precisely the watershed between technical standards and data sovereignty.

The following diagram compares two different technical paths. The left side, “Governance-Sovereignty Type,” emphasizes data traceability and controllability, using discrete point-based data presentation. The right side, “Engineering-Open Type,” allows model interpolation and inference, providing more user-friendly experience through continuous area-based coverage. The essence of this difference lies not in the level of technical capability, but in the different choices made between data sovereignty, governance capability, and open ecosystems.

Figure 4: Technical Standards and Sovereignty Divide in Spatial Data Presentation

The Amplification Effect in the AI Era

With the above logic in mind, many phenomena in the AI era become less confusing.

For example:

Why are Chinese AI companies more willing to open source large language model (LLM) weights, while American companies have clearly shifted toward closed source in recent years?
Why is foundational software and infrastructure-level open source still mainly led by the US?

The key is not “whether to open source,” but “which layer is open sourced.”

Model weights are static, declarable assets
Infrastructure, runtimes, protocols, and standards are dynamic, evolving system rules

Open sourcing weights is essentially openness at the asset layer; infrastructure-level open source means relinquishing control over operating rules and interpretive authority.

The following diagram compares two different layers of AI open source. The left side shows “Model Weight Layer Open Source,” which is a typical feature of Chinese path—opening static digital assets with low cost and controllable risk, but not involving rule-making. The right side shows “Infrastructure Layer Open Source,” which is a core strategy of US path—by open sourcing development tools, protocol standards, runtimes, and compute scheduling and other infrastructure, defining how AI is used, thereby mastering ecosystem rules and interpretive authority. Key insight: Open sourcing model weights does not equal mastering AI ecosystem, and the real competitive focus is shifting to the infrastructure layer of “how AI runs.”

Figure 5: Two Layers of AI Era Open Source: Model Weights vs Infrastructure

The US Approach: Focusing on Rules and Runtime Layers

In the past year or two, US-led AI open source and ecosystem initiatives have shown a highly consistent direction: not rushing to open source the strongest models, but focusing on defining “how AI is used.”

The Linux Foundation established AAIF (Agentic AI Foundation), focusing on AI infrastructure, standards, and toolchain collaboration
Protocols like MCP (Model Context Protocol) aim to define common interaction methods between agents and tools/systems
Major tech companies are generally focusing on APIs, platforms, runtimes, and ecosystem binding

The commonality of these actions: competing in model capability, but controlling the usage rules.

China’s Shift: From Model-Oriented to Infrastructure-Oriented

It is important to emphasize that this difference does not mean China is unaware of the issue.

Whether in policy discussions or within industry and research institutions, the risk of “only open sourcing models without controlling infrastructure and standard dominance” has been repeatedly discussed.

The real challenge lies in how to achieve a directional shift within the existing governance logic and risk framework. This shift has already appeared in some concrete practices.

Exploration and Practice at the Infrastructure Layer

In the AI era, infrastructure often starts with the most engineering-driven problems.

HAMi Project

Projects like HAMi do not focus on model capability, but on:

Abstraction, allocation, and isolation of GPU resources
How multi-tenant AI workloads are run
How computing power transitions from hardware assets to governable system resources

The significance of such projects is not about being “SOTA,” but about entering the domain of “how AI runs.”

AI Runtime Reconstruction from a System Software Review

Exploration at the research institution level is also noteworthy. The FlagOS initiative by the Beijing Academy of Artificial Intelligence is a clear signal: AI is being redefined as a system software issue, not just a model or algorithm problem.

Long-Term Tech Stack Investment by Industry Players

In the industry, Huawei’s strategy reflects a similar direction: not simply open sourcing models, but attempting to build a complete, controllable AI tech stack, from computing power to frameworks, platforms, and ecosystems. This is a slower, heavier, but more infrastructure-competitive path.

Realistic Assessment: The Starting Point of AI Infrastructure Competition

Taking a longer view, we find an easily overlooked fact:

At the level of AI infrastructure and infra-level open source, there is no settled pattern between China and the US.

The US advantage lies in:

Mature engineering culture
Standard organizations and foundation mechanisms
High proficiency in openness at the rules layer

China’s variables include:

Huge AI application scenarios
Extreme demand for computing power and system efficiency
Ongoing directional adjustments

The real uncertainty is not “whether we can catch up,” but whether it is possible to gradually open up space for engineering autonomy and standard co-construction while maintaining governance bottom lines.

Summary

The “points” and “areas” of air quality, model weights and the world of operations—behind these appearances lies not a simple technical route dispute, but how a country finds its own balance between openness, standards, and sovereignty.

In the AI era, this issue will not disappear, but will become more concrete and more engineering-driven. And this is precisely where there are still opportunities for China’s AI infrastructure open source.

Joining Dynamia: Embarking on a New Journey in AI Native Infrastructure

Jimmy Song — Wed, 07 Jan 2026 07:49:21 +0000

Compute governance is the critical bottleneck for AI scaling. From hardware consumption to core asset, this long-undervalued path needs to be redefined.

Figure 1: Dynamia.ai

A New Beginning

I have officially joined Dynamia as Open Source Ecosystem VP, responsible for the long-term development of the company in open source, technical narrative, and AI Native Infrastructure ecosystem directions.

Why I Chose Dynamia

I chose to join Dynamia not because it’s a company trying to “solve all AI problems,” but precisely the opposite—it’s because Dynamia focuses intensely on one unavoidable, yet long-undervalued core issue in AI Native Infrastructure: compute, especially Graphics Processing Units (GPU), are evolving from “technical resources” into infrastructure elements that require refined governance and economic management.

Through years of practice in cloud native, distributed systems, and AI infrastructure (AI Infra), I’ve formed a clear judgment: as Large Language Models (LLM) and AI Agents enter the stage of large-scale deployment, the real bottleneck limiting system scalability and sustainability is no longer just model capability itself, but how compute is measured, allocated, isolated, and scheduled, and how a governable, accountable, and optimizable operational mechanism is formed at the system level. From this perspective, the core challenge of AI infrastructure is essentially evolving into a “resource governance and Token economy” problem.

About Dynamia and HAMi

Dynamia is an AI-native infrastructure technology company rooted in open source DNA, driving efficiency leaps in heterogeneous compute through technological innovation. Its leading open source project, HAMi (Heterogeneous AI Computing Virtualization Middleware), is a Cloud Native Computing Foundation (CNCF) sandbox project providing GPU, NPU and other heterogeneous device virtualization, sharing, isolation, and topology-aware scheduling capabilities, widely adopted by 50+ enterprises and institutions.

Dynamia’s Technical Approach

In this context, Dynamia’s technical approach—starting from the GPU layer, which is the most expensive, scarcest, and least unified abstraction layer in AI systems, treating compute as a foundational resource that can be measured, partitioned, scheduled, governed, and even “tokenized” for refined accounting and optimization—aligns highly with my long-term judgment on AI-native infrastructure.

This path doesn’t use “model capabilities” or “application innovation” as selling points in the short term, nor is it easily packaged into simple stories. However, with rising compute costs, heterogeneous accelerators becoming the norm, and AI systems moving toward multi-tenant and large-scale operations, these infrastructure-level capabilities are gradually becoming prerequisites for the establishment and expansion of AI systems.

Future Focus

As Dynamia’s Open Source Ecosystem VP, I will focus on technical narrative of AI-native infrastructure, open source ecosystem building, and global developer collaboration, promoting compute from “hardware resource being consumed” to governable, measurable, and optimizable AI infrastructure core asset, laying the foundation for the scaling and sustainable evolution of AI systems in the next stage.

Summary

Joining Dynamia is an important milestone in my career and a concrete action demonstrating my long-term optimism about AI-native infrastructure. Compute governance is not a short-term trend that yields quick results, but an infrastructure proposition that cannot be bypassed for AI large-scale deployment. I look forward to exploring, building, and landing solutions on this long-undervalued path with global developers.

References

Running Parallel AI Agents on My Mac: Hands-On with Verdent's Standalone App

Jimmy Song — Sun, 04 Jan 2026 02:25:48 +0000

I’ve been spending more time recently experimenting with vibe coding tools on real projects, not demos. One of those projects is my own website, where I constantly tweak content structure, navigation, and layout.

During this process, I started using Verdent’s standalone Mac app more seriously. What stood out was not any single feature, but how different the experience felt compared to traditional AI coding tools.

Figure 1: Verdent Standalone App UI

Verdent doesn’t behave like an assistant waiting for instructions. It behaves more like an environment where work happens in parallel.

A Different Starting Point: Tasks, Not Chats

Most AI coding tools begin with a conversation. Verdent begins with tasks.

When I opened my website repository in the Verdent app, I didn’t start with a long prompt. I created multiple tasks directly: one to rethink navigation and SEO structure, another to explore homepage layout improvements, and a third to review existing content organization.

Each task immediately spun up its own agent and workspace. From the beginning, the app encouraged me to think in parallel, the same way I normally would when sketching ideas on paper or jumping between files.

This framing alone changes how you work.

Built for Multitasking, Without Losing Context

Switching contexts is unavoidable in real development work. What usually breaks is continuity.

Verdent handles this well. Each task preserves its full context independently. I could stop one task mid-way, switch to another, and come back later without re-explaining the problem or reloading files.

For example, while one agent was analyzing my site’s navigation structure, another was exploring layout options. I moved between them freely. Nothing was lost. Each agent remembered exactly what it was doing.

This feels closer to how developers think than how chat-based tools operate.

Safe Parallel Coding with Workspaces

Parallel work only becomes truly safe when code changes are isolated. When parallelism moves from discussion to actual code modification, risk management becomes essential.

Verdent solves this with Workspaces. Each workspace is an isolated, independent code environment with its own change history, commit log, and branches. This isn’t just about separation—it’s about making concurrent code changes manageable.

What this means in practice:

Multiple tasks can write code simultaneously
Changes remain isolated from each other
If conflicts arise, they’re visible and cleanly resolvable

I intentionally let different agents operate on overlapping parts of my project: one modifying Markdown content and links, another adjusting CSS and layout logic. Both ran in parallel. No conflicts emerged. Later, I reviewed the diffs from each workspace and merged only what made sense.

This kind of isolation removes significant anxiety from AI-assisted coding. You stop worrying about breaking things and start experimenting more freely, knowing that each change exists in its own contained environment.

Parallel Agent Execution Feels Like Delegation

Parallelism doesn’t mean that all agents complete the same phase of work at the same time—instead, by isolating and overlapping phases, what was once a strictly sequential process is compressed into a more efficient, collaborative mode.

In Verdent, each agent runs in its own workspace, essentially an automatically managed branch or worktree. In practice, I often create multiple tasks with different responsibilities for the same requirement, such as planning, implementation, and review. But this doesn’t mean they all complete the same phase simultaneously.

These tasks are triggered as needed, each running for a period and producing clear artifacts as boundaries for collaboration. The planning task generates planning documents or constraint specifications; the implementation task advances code changes based on those documents and produces diffs; the review task, according to the established planning goals and audit criteria, performs staged reviews of the generated changes. By overlapping phases around artifacts, the originally strict sequential process is compressed into a workflow that more closely resembles team collaboration.

The value of splitting into multiple tasks is not parallel execution, but parallel cognition and clear collaboration boundaries.

While it’s technically possible to put multiple roles into a single task, this causes planning, implementation, and review to share the same context, which weakens role isolation and the auditability of results.

Configurability and Design Trade-offs

Beyond the workflow model itself, Verdent exposes a surprisingly rich set of configurable capabilities.

It allows users to customize MCP settings, define subagents with configurable prompts, and create reusable commands via slash (/) shortcuts. Personal rules can be written to influence agent behavior and response style, and command-level permissions can be configured to enforce basic security boundaries. Verdent also supports multiple mainstream foundation models, including GPT, Claude, Gemini, and K2. For users who prefer a lightweight coding experience without a full IDE, Verdent offers DiffLens as an alternative review-oriented interface. Both subscription-based and credit-based pricing models are supported.

Figure 2: Verdent Settings

That said, Verdent makes a clear set of trade-offs. It is not built around tab-based code completion, nor does it offer a plugin system. If it did, it would start to resemble a traditional IDE - which does not seem to be its goal. Verdent is not designed for direct, fine-grained code manipulation; most changes are mediated through conversational tasks and agent-driven edits. This makes the experience clean and focused, but it also means that for large, highly complex codebases, Verdent may function better as a complementary orchestration layer rather than a full-time development environment.

Where Verdent Fits Today

There are many AI-assisted coding tools emerging right now. Some focus on smarter editors, others on faster generation.

Verdent feels different because it focuses on orchestration, not just assistance.

It doesn’t try to replace your editor. It sits one level above, coordinating planning, execution, and review across multiple agents.

That makes it particularly suitable for exploratory work, refactoring, and early-stage design - exactly the kind of work I was doing on my website.

Final Thoughts

Using Verdent’s standalone app didn’t just speed things up. It changed how I structured work.

Instead of doing everything sequentially, I started thinking in parallel again - and letting the system support that way of thinking.

Verdent feels less like an AI feature and more like an environment that assumes AI is already part of how development happens.

For developers experimenting with AI-native workflows, that shift is worth paying attention to.

2025 Annual Review: The Transformation Journey from Cloud Native to AI Native

Jimmy Song — Wed, 31 Dec 2025 10:02:01 +0000

The waves of technology keep evolving; only by actively embracing change can we continue to create value. In 2025, I chose to move from Cloud Native to AI Native—this year marked a key turning point for personal growth and system reinvention.

2025 was a turning point for me. This year, I not only changed my technical direction but also the way I approach problems. Moving from Cloud Native infrastructure to AI Native Infrastructure was not just a migration of content, but an upgrade in mindset.

Figure 1: Farewell 2025!

This year, I conducted a large-scale refactoring of the website and systematically organized the content. Beyond the technical improvements, I want to share my thoughts and changes throughout the year.

A Bold Shift: Embracing the AI Native Era

At the beginning of 2025, I made an important decision: to reposition myself from a Cloud Native Evangelist to an AI Infrastructure Architect. This was not just a change in title, but a strategic transformation after careful consideration.

As I witnessed the surge of AI technologies and the rise of Agent-based applications reshaping software, I realized that clinging to the boundaries of Cloud Native might mean missing an era. So, I systematically adjusted the website’s content structure, shifting the focus toward AI Native Infrastructure.

This transformation was not about abandoning the past, but extending forward from the foundation of Cloud Native. Classic content like Kubernetes and Istio remains and is continuously updated, but new topics such as AI Agent and the AI OSS landscape have been added, forming a more complete knowledge map.

Content Creation: From Technical Details to Ecosystem Perspective

AI Agent: Building Systematic Knowledge

Agents represent a major evolution in software for the AI era. When I tried to understand Agent design principles, I found fragmented information everywhere but lacked a systematic knowledge base.

So I created content that analyzes the Agent context lifecycle and control loop mechanisms, summarizing several proven architectural patterns. To make complex knowledge easier to digest, I organized it into logical sections so readers can learn step by step.

AI Tool Ecosystem: Mapping the Open Source Landscape

AI tools and frameworks are emerging rapidly, with new projects appearing daily. To help readers quickly grasp the ecosystem, I built a comprehensive AI OSS database.

This database covers everything from Agent frameworks to development tools and deployment services. I not only included active projects but also established an archive mechanism, preserving detailed information on over 150 historical projects. More importantly, I developed a scoring system to objectively evaluate projects across dimensions like quality and sustainability, helping readers decide which tools are worth investing time in.

Blogging: Capturing Technology Trends Faster

In 2025, I wrote over 120 blog posts. Compared to previous years, these articles focused more on observing and reflecting on technology trends, rather than just technical tutorials.

I started paying attention to deeper questions: How will AI infrastructure evolve? What does Beijing’s open source initiative mean for the AI industry? What ripple effects might a tech acquisition trigger? These articles allowed me and my readers to not only see “what” technology is, but also “why” and “what’s next.”

User Experience: Making Knowledge Easier to Discover and Consume

No matter how good the content is, if it can’t be easily found and read, its value is greatly diminished. In 2025, I invested significant effort into website functionality, with one goal: to provide readers with a smoother reading experience.

Comprehensive Search Upgrade

As the volume of content grew, the original search function could no longer meet demand. I redesigned the search system to support fuzzy search and result scoring, and optimized index loading performance. More importantly, the new search interface is more user-friendly, supporting keyboard navigation and category filtering so users can find what they want faster.

Multi-Device Experience Optimization

Mobile reading experience has improved significantly. I refactored the mobile navigation and table of contents, making reading on phones much smoother. Dark mode is now more refined, fixing several display issues and ensuring images and diagrams look good on dark backgrounds.

Efficiency Revolution in Content Distribution

A major change was optimizing the WeChat Official Account publishing workflow. Previously, publishing website content to WeChat required manual handling of many details; now, it’s almost one-click export. This workflow automatically processes images, metadata, styles, and all details, reducing a half-hour task to just a few minutes.

Additionally, I added a glossary feature for technical term highlighting and tooltips; improved SEO and social sharing metadata; and cleaned up outdated content. These seemingly minor improvements quietly enhance the user experience.

Content Evolution: More Dimensional Knowledge Expression

Looking back at content creation in 2025, I found clear changes in several dimensions.

From Tutorials to Observations

Early content leaned toward technical tutorials and practical guides, showing “how to do.” This year, I focused more on “why” and “what are the trends.” I wrote more technology trend analyses, ecosystem maps, and in-depth case studies. These may not directly teach you how to use an API, but they help you understand the direction of technological evolution.

From Chinese to Bilingual

AI is a global wave and cannot be limited to the Chinese-speaking world. In 2025, I wrote bilingual documentation for almost all new AI tools, and important blog posts also have English versions. This increased the workload, but allowed the content to reach a broader audience.

From Text to Multimedia

Text is efficient, but not all knowledge is best expressed in words. This year, I used many architecture and schematic diagrams to explain complex concepts, adding 59 new charts. These visual elements lower the barrier to understanding, making abstract concepts more intuitive. I also optimized image display in dark mode to ensure consistent visual experience.

Development Approach: Embracing AI-Assisted Programming

2025 was not only a year of shifting content themes toward AI, but also a year of deep practice in AI-assisted programming.

I developed a VS Code plugin and created many prompts to automate repetitive tasks. I experimented with various AI programming tools and settled on a toolchain that suits me. I even migrated the website to Cloudflare Pages and used its edge computing services to develop a chatbot. These practices greatly improved development efficiency, giving me more time to focus on thinking and creating rather than mechanical coding.

This made me realize: AI will not replace developers, but developers who use AI well will replace those who do not. I also shared more insights to help others master AI-assisted programming.

Looking Ahead to 2026: Keep Moving Forward

Looking back at 2025, the site underwent a profound transformation—from a Cloud Native tech blog to an AI infrastructure knowledge base. But this is just the beginning, not the end.

Looking forward to 2026, I plan to continue deepening in several areas:

Enhancing the knowledge system: Continue to supplement GPU infrastructure and AI Agent content, especially practical cases and performance tuning knowledge.
Tracking ecosystem evolution: AI tools and frameworks iterate rapidly; I need to keep up with this fast-changing ecosystem and update content in a timely manner.
Deepening engineering practice: Share more practical AI engineering experience to help readers turn theory into practice.
Exploring knowledge connections: Consider building a knowledge graph to connect different content sections, providing smarter navigation and recommendations.

Summary

2025 was a year of change and growth. From Cloud Native to AI Native, from technical practice to ecosystem observation, both the content and functionality of the site have made qualitative leaps.

What makes me happiest is that this transformation allowed me and my readers to stand at the forefront of the technology wave. We are not just learning new technologies, but thinking about how technology changes the world and the way we write software.

The waves of technology keep evolving; only by actively embracing change can we continue to create value. Thank you to every reader for your companionship and support. I look forward to sharing more insights and practices in 2026.

Further Reading:

The Butterfly Effect After Manus Was Acquired by Meta

Jimmy Song — Tue, 30 Dec 2025 03:30:51 +0000

The success or failure of AI applications often lies not in the technology itself, but in the ability to scale delivery and create a closed loop.

Figure 1: The Butterfly Effect After Manus Was Acquired by Meta

When “Those Who Discuss It” Are Not “Those Who Pay for It”

On December 30, 2025, a piece of news went viral: Manus was acquired by Meta for billions of dollars (Manus Joins Meta for Next Era of Innovation). This startup, founded in China and under pressure from tech giants since its inception, completed a whirlwind journey in less than a year—from explosive growth, relocating to Singapore, to being acquired by a global giant.

According to Manus’s official statement, its products and subscriptions will continue to be available via the app and website, and the company will remain operational in Singapore. The team will join Meta to provide general Agent capabilities for Meta’s consumer and enterprise products (including Meta AI).

Rather than focusing on “who won,” I’m more interested in the chain reaction this event triggered: it activated completely opposite judgment systems among different groups, and this split is reshaping the growth paths and strategies for AI applications and startups.

Two Public Opinion Arenas: Blessings and Doubts Coexist

After Manus was acquired, the mainstream sentiment in social circles was one of congratulations and excitement. Many saw it as a stellar example of a Chinese team going global—achieving remarkable results in the most competitive field in a very short time.

Meanwhile, the comment sections of public accounts became “venting valves for counter-narratives,” with skepticism centering on three main points:

Whether the technology has real barriers (e.g., “there are countless similar products,” “it’s not hard for big companies to build their own”).
Valuation and bubble concerns (e.g., “another case of the AI bubble”).
Distrust in the buyer’s judgment (e.g., “giants making desperate bets,” “history repeating itself”).

This divergence isn’t about who understands AI better, but about different evaluation frameworks: social circles focus on “trajectory and outcome,” while comment sections focus on “legitimacy and worthiness.”

Many people are impressed by Manus’s marketing buzz and controversies, which can lead to skepticism. But if it achieved a “strict $100M ARR” in 10 months, one fact is clear: its revenue doesn’t depend on broad consensus, but comes from a highly concentrated group of global users with strong willingness to pay.

Manus’s core user profile is closer to “individuals as production units,” including freelancers, indie developers, independent researchers, and key deliverers in small and medium businesses. They don’t care about debates over “wrapping” or not; they care about “can I deliver end-to-end tasks,” and “can this help me hire one less person, work fewer late nights, or avoid juggling ten tools.”

This leads to a counterintuitive phenomenon: those who discuss the most may not pay, while those who pay steadily are often silent.

For these users, tools are not identity badges—they are profit levers.

Three Lessons for Entrepreneurs: The Growth Paradigm in the AI Application Era Has Changed

Based on the above, the Manus case offers three lessons for entrepreneurs:

Growth No Longer Equals Positive Reviews

AI applications can commercialize first and build consensus later. Public opinion can remain divided for a long time, but cash flow doesn’t wait for unified recognition.

“Heavy Marketing” Is Becoming a Capability, Not a Stigma

As foundational models and capabilities spread rapidly, differentiation is quickly erased. Being seen, understood, and paid for is itself part of the moat. Not all marketing deserves respect, but “distribution and mindshare” have become unavoidable battlegrounds for AI applications.

Globalization Is No Longer a Bonus, but May Be a Survival Strategy

From payment willingness, compliance boundaries, talent density to valuation systems, market structure means many teams “can only complete the loop overseas.” It’s not romantic, but it’s reality.

A Personal Reflection

As someone long engaged in cloud native and AI infrastructure, I’m used to evaluating products by their “technical barriers.” But cases like Manus remind me: at the AI application layer, barriers may not first appear in models or code, but often in organizational speed, productization capability, delivery loop, and distribution efficiency.

When a system can reliably turn “capability” into “results,” it has built a commercial moat—even if its tech stack doesn’t meet outsiders’ ideals of “purity.”

The biggest butterfly effect of Manus being acquired by Meta may not be the deal itself, but making more entrepreneurs realize: in the AI era, the winning move is shifting from “what model you use” to “whether you can deliver results at scale.”

Summary

The acquisition of Manus by Meta is not just a convergence of capital and technology, but also a microcosm of the changing growth paradigm in the AI application era. For entrepreneurs, understanding and mastering “user structure,” “distribution capability,” and “global closed loops” will be key to future competition.

AI Infra Open Source in China: Analysis of Beijing and Shanghai's Plans

Jimmy Song — Thu, 25 Dec 2025 10:01:13 +0000

Institutionalized open source marks a new starting point for China’s AI Infra, but true breakthroughs and risks lie in the engineering and governance details.

Perspective on Beijing and Shanghai’s Open Source Plans

Using the simultaneous release of open source ecosystem plans by Beijing and Shanghai as a lens, and drawing on China’s past foundation practices and international open source governance experience, this article explores the real opportunities, structural constraints, and potential risks as AI Infrastructure (AI Infra, Artificial Intelligence Infrastructure) enters a new phase of institutionalized open source.

Figure 1: Beijing and Shanghai successively launch open source ecosystem construction plans

Why Compare Beijing and Shanghai Together

It is rare for me to write an article solely because of a local policy document. However, during Christmas, both Beijing and Shanghai’s Bureaus of Economy and Information Technology released their respective open source ecosystem construction plans:

This time, the fact that both cities released their plans on the same day sends a signal worth serious attention: China is attempting to advance open source in a more systematic and institutionalized way, especially regarding open source capabilities related to AI Infra.

If you only look at Beijing’s plan, it is easy to interpret it as a local industrial policy upgrade. But when you consider both Beijing and Shanghai’s plans together, it looks more like a clearly defined “dual-center structure.”

The question is no longer whether to develop open source, but:

In the AI era, what institutional forms, engineering paths, and governance models will open source take?

Open Source as “Industrial Infrastructure Engineering”

Both Beijing and Shanghai’s plans reflect a highly consistent judgment:

Open source is no longer seen as a spontaneous community activity, but as an industrial infrastructure capability that requires systematic construction.

This is especially evident in the field of AI Infra.

Issues such as computing power scheduling, model evaluation, toolchains, data elements, license compliance, and supply chain security—previously hidden in “engineering details”—are now systematically incorporated into policy language for the first time. This at least shows that decision-makers have realized:

AI competition is not only about model parameter scale
It is even more about toolchains, infrastructure, evaluation systems, and engineering capabilities
These capabilities are naturally more suitable for building public foundations through open source

In this respect, Beijing and Shanghai are highly aligned.

Two Open Source Paths: Infra vs. Platform

When we zoom in, the differences between the two plans become clear.

Beijing: “Foundation-Oriented” Open Source Path for AI Infra

Beijing’s plan focuses on:

Heterogeneous computing power scheduling
Model evaluation toolchains
Data elements and data governance
RISC-V software-hardware collaboration
SBOM, license compatibility, open source compliance
Supply chain security and industrial resilience

This is a typical perspective of “treating AI as an infrastructure problem.”

It is less concerned with the number of projects or community size, and more with:

Whether reusable engineering capabilities can be formed
Whether these can be trusted by industry and government over the long term
Whether they can stand up to scrutiny in terms of security, compliance, and governance

To some extent, Beijing is answering the question:

How can open source become a “governable, auditable, and scalable public capability”?

Shanghai: “Scale and Internationalization” Path for AI Platform

In contrast, Shanghai’s plan has a different focus:

Building an international open source community for artificial intelligence
Covering the entire platform chain from development, training, testing, hosting, to operation
Overseas sites, multilingual support, international activities
Resource linkage through computing vouchers and model vouchers
“Open source platform first release / global simultaneous release” dual-release mechanism
Clear targets for community, enterprise, and developer scale

Shanghai cares more about:

How open source can achieve scale effects
How it can support the growth of commercial enterprises
How it can be seen and adopted globally

This is a path of “treating open source as a global digital product and platform capability.”

Together: A Complete but Tension-Filled Structure

When viewed together, Beijing and Shanghai’s plans form a more complete picture:

Beijing is responsible for “making open source solid,” while Shanghai is responsible for “taking open source global.”

Structurally, this is a clear division of labor:

Beijing focuses on institutions, governance, and foundational capabilities
Shanghai focuses on community, commercialization, and international communication

These two paths are not in conflict; in theory, they are even complementary. The real question is whether they can form positive feedback in practice, rather than operating in silos.

Cautious Attitude Toward “Institutionalized, Platformized Open Source”

Precisely because both plans are so “systematic,” I am even more cautious.

The reason is simple: this is not China’s first attempt to promote open source through foundations, associations, or platforms.

Over the past decade, we have seen similar paths repeatedly, and recurring structural problems:

The difficulty of establishing neutrality and multi-party trust is extremely high
There is a huge gap between showcase metrics (quantity, activities, certifications) and ecosystem strength
Commercialization and long-term maintenance mechanisms are hard to sustain

These problems will not disappear just because the plans are more comprehensive.

Four Risks to Watch Under the Dual Plans

If we are to “listen to their words and watch their actions,” I would focus on the following four risks:

Will Metrics Hijack Engineering Reality

When “internationally influential projects,” “star projects,” and “first-release projects” become hard metrics, will this induce packaging, migration, and short-term hype, rather than truly solving engineering problems?

Will It Slide Toward Platform Centralism

The long-term pattern of AI Infra is closer to a model that prioritizes protocols, standards, and interoperability. If it eventually evolves into “a few platforms concentrating resources and discourse power,” it may be efficient in the short term but will suppress external participation and international collaboration in the long run.

Is Internationalization Underestimated as an “Operational Issue”

True international collaboration is never just about language, sites, or events; it also involves governance structures, compliance boundaries, and supply chain trust.

Will Application Demonstrations Become One-Off Projects

If “first plans” and “computing vouchers” are just procurement tactics without continuous iteration and community feedback mechanisms, the long-term benefit to the ecosystem will be very limited.

What Are the “Hard Results” of AI Infra Open Source After Three Years

If we review the success of this round of institutionalized open source after three years, I would look for three types of results:

Whether de facto standards and interoperable ecosystems have emerged, including scheduling interfaces, evaluation benchmarks, Agent tool invocation protocols, and observability semantics.
Whether compliance and supply chain security have become public capabilities—SBOM, license compatibility, vulnerability monitoring—truly productized and service-oriented.
Whether a sustainable maintenance business mechanism has been established, allowing core maintainers to stay long-term, rather than relying on passion and subsidies.

If I were to use a North Star metric to measure the success of these plans, it would be the emergence of several outstanding open source commercial companies rooted in China and serving the world.

Summary

The open source ecosystem plans of Beijing and Shanghai mark a new phase of institutionalization and engineering for AI Infra open source in China. Over the next three years, the real achievements will not be about meeting targets, but about forming sustainable engineering capabilities, de facto standards, and maintenance mechanisms. Only through continuous participation and practice can open source become the public foundation of AI infrastructure.

References

From 2025 Onwards, Software Engineering Shifts from Code-Centric to Runtime and Cost-Centric

Jimmy Song — Wed, 24 Dec 2025 14:59:11 +0000

In 2025, the core of software engineering is no longer just about code itself, but about runtime controllability and cost governance. This shift is fundamentally reshaping the industry’s underlying logic.

Looking back at 2025, I became increasingly aware that this year was not about “code becoming unimportant,” but rather that the value coordinates of engineering have shifted as a whole. For more than a decade, software engineering has focused on code quality, architectural evolution, and delivery efficiency. But starting in 2025, the key to system success is shifting—towards whether the runtime is controllable and whether costs are governable.

This is not just a slogan, but a conclusion repeatedly validated by my real-world experiences throughout the year.

My 2025: From “Platform Engineering” to “Runtime Challenges”

In my annual review, I noted a clear change: I spent less time on “how to write a good system,” and more time on “how to keep the system running stably, reliably, and affordably.”

This shift in focus is a natural extension of a decade of cloud native evolution.

The following timeline diagram illustrates how my focus has changed over recent years:

Figure 1: My Focus Shift Timeline

My focus shifted from cloud native platform engineering to LLM application engineering, then to AI infrastructure, and finally to Agentic Runtime with governance and cost control.

When AI workloads truly enter business scenarios, the core challenges engineers face also change:

Are inference, training, and evaluation competing for the same compute pool?
Is GPU utilization consistently below expectations?
Does cost scale linearly and uncontrollably with concurrency?
Does the system have failure isolation and replay capabilities?

These issues go far beyond the code level.

Industry Consensus: AI Is Shifting the Focus of Engineering

By 2025, an industry consensus is emerging: AI is rewriting software engineering. But the real change is not happening in the IDE or code completion speed—it is reflected in the migration of engineering complexity.

Previously, complexity was concentrated in code and interfaces, and problems were solved through abstraction, refactoring, and testing.

Now, complexity has shifted to the runtime, resource, and cost layers, and must be addressed through scheduling, isolation, observability, and governance.

This is why the same AI tools:

Serve as “accelerators” for junior engineers
But act as “magnifiers” for senior engineers

AI tools amplify whether you truly understand how systems run in production.

Why “Cost” Becomes a First Principle

In traditional cloud native systems, low CPU utilization is often just an efficiency issue; but in AI systems, low GPU utilization is often a cash flow problem.

In 2025, I repeatedly encountered scenarios like:

Resources “seem insufficient,” but utilization is not actually high
Scaling up to solve queuing issues ends up increasing unit costs
The system lacks clear budget and quota boundaries, so throttling becomes the only way to stop the bleeding

The root cause of these phenomena is not model selection, but the lack of a runtime and cost control plane tailored for AI workloads.

The following flowchart visually illustrates the cyclical relationship between GPU resources and cost pressures:

Figure 2: GPU Resource and Cost Cycle in AI Systems

In AI systems, limited GPU supply leads to queuing and waiting, which causes throughput to drop. Attempts to solve this through blind scaling only increase unit costs and create budget pressure, ultimately forcing the adoption of finer scheduling and governance strategies.

Engineering problems ultimately manifest as cost issues.

The Rise of Agents: The Real Challenge Is at Runtime

In 2025, Agent (Intelligent Agent, Agent, Intelligent Agent) became a hot topic; by 2026, it will enter the “can it actually run” stage.

The challenge for Agents has never been about “how smart they are,” but rather:

Whether there are clear permission and data boundaries
Whether they run in an isolated execution environment
Whether they can be observed, evaluated, and replayed
Whether they are subject to explicit cost and budget constraints

These capabilities form the outline of Agentic Runtime (Agentic Runtime, Intelligent Agent Runtime) that I have been trying to clarify throughout the year.

The following flowchart shows the core capability layers of Agentic Runtime:

Figure 3: Agentic Runtime Capability Layers

Agentic Runtime builds from the foundation of Agents and workflows, connecting through orchestration and tool protocols, with the runtime managing state, memory, and evaluation. It provides secure execution environments (Sandbox and Policy), and ultimately implements a resource and cost control plane that unifies GPU, quota, and billing management.

Without a runtime, an Agent is just a demo; without cost constraints, an Agent is just a risk amplifier.

Outlook for 2026: The “Foundation” of Engineering Matters Again

Looking ahead to 2026, I remain cautiously optimistic.

I do not believe the future belongs to “those who write the best prompts,” but more likely to:

Those who understand runtime boundaries
Those who can govern compute as a constrained resource
Those who design AI systems as long-running systems

From 2025 onwards, software engineering is no longer code-centric, but runtime and cost-centric. This is not a regression, but a return: a return to being responsible for the whole system and for real-world constraints.

For me personally, this is both a year-end summary and the direction I will continue to invest in for the coming years.

Summary

In 2025, the focus of software engineering has shifted from code itself to runtime and cost governance. The rise of AI and Agents has not diminished the value of engineering, but has pushed complexity to a higher level. In the future, understanding runtime, managing compute and cost will become the new core competencies for engineers. I hope this year-end review provides some inspiration and reflection for fellow professionals.

From Cloud Native to AI Native: Why Kubernetes Is the Foundation for Next-Gen AI Agents

Jimmy Song — Wed, 24 Dec 2025 12:25:52 +0000

As a long-time practitioner in the cloud native field, I am increasingly convinced of one thing: AI Agents are not just a change in application form, but a migration of infrastructure paradigms.

As artificial intelligence evolves from demos and copilots to systems that truly take on tasks and responsibilities, AI Agents are becoming the new execution units in enterprise IT architectures. They not only “think,” but also act: they can invoke tools, access systems, and collaborate to achieve goals.

This raises an important question:

What kind of infrastructure should such systems run on?

In my view, Kubernetes remains a solid choice for large-scale scenarios—but only if we reimagine Kubernetes in an AI-native way.

Cloud Native Challenges for Production-Grade AI Agents

In real production environments, AI Agents expose infrastructure needs that are fundamentally different from traditional microservices. Agents are not “just another HTTP service”; they have three distinct characteristics:

Behavior is non-deterministic (driven by model inference)
Execution paths are dynamic (tool invocation cannot be fully enumerated in advance)
Decisions must be auditable, constrained, and reviewable

If we simply apply existing cloud native infrastructure, we quickly hit bottlenecks.

The following table summarizes the main challenges and risks AI Agents face in cloud native environments:

Challenge Category	Real Needs of Agents	What Happens If Missing
Policy & Security	Dynamic control of tool and data access based on context, identity, and task	Agents have “superuser” privileges, risks are uncontrollable
Observability	Not just “did it succeed,” but also why was this decision made	Hard to debug, hard to review, hard to hold accountable
Governance & Consistency	Platform-level guardrails enforce organizational policies	Each Agent could become a “shadow AI”

Table 1: Challenges and Risks for AI Agents in Cloud Native Environments

All these issues point to one conclusion:

AI Agents must be treated as first-class citizens in Kubernetes, not just ordinary workloads.

Core Architecture: Making Agents Native Kubernetes Objects

Looking back at the evolution of cloud native technologies, we’ve gone through similar stages:

Physical machines → Virtual machines
Virtual machines → Containers
Containers → Microservices
Microservices → Declarative, governable platforms

AI Agents are simply the next step.

A production-ready AI Agent architecture requires at least three layers:

Agent Orchestration Layer: Declaratively define Agents
Tool Service-ization Layer (MCP Services): Turn capabilities into governable services
AI Native Data Plane / Gateway: Unify policy, security, and protocols

Agent Orchestration Layer: Declarative Agent Management

Agents should no longer be “runtime objects” inside an SDK—they should be managed like Pods or Deployments.

Key concepts:

Agents as Kubernetes Resources

Agents are defined using CRD (CustomResourceDefinition)
Lifecycle managed via kubectl or GitOps
Agent models, tools, and policies are all explicitly declared

A typical Agent definition includes:

Agent logic (inference loop)
Model configuration (specifying which large language model to use)
Callable toolset

This closely mirrors how we once decomposed “applications” into Deployments, Services, and ConfigMaps.

Tool Service-ization Layer: MCP Services Are Essential

In Agent architectures, tools are where real “actions” happen.

Early MCP tools were often:

Local processes
Tightly coupled to a single Agent
Lacking versioning, permissions, and auditing

This is unsustainable in enterprise environments.

The Essence of MCP Service-ization

Tools → Remote services
Services → Kubernetes native workloads
Capabilities → Reusable, governable, auditable

This step is fundamentally similar to how we once turned scripts into microservices.

AI Native Gateway: The “Control Plane Entry” for the Agent World

As the number of Agents grows and tools/models diversify, connectivity itself becomes a system risk.

Traditional API Gateways do not understand scenarios like:

MCP
Agent-to-Agent (A2A) communication
Model invocation context

Thus, we need an AI native gateway dedicated to mediation and governance.

It must understand at least three types of traffic:

A2T: Agent → Tool
A2L: Agent → LLM
A2A: Agent ↔ Agent

And enforce, across these paths:

Identity and authorization
Policy and guardrails
Auditing and rate limiting

Architecture Overview

The diagram below illustrates the core layers and traffic paths of an AI-native system on Kubernetes:

Figure 1: AI Native Architecture Layers and Traffic Paths

Summary

AI Agents do not negate cloud native; on the contrary:

AI Agents are the natural extension of cloud native in the era of intelligence.

Declarative → Agent definitions
Service → MCP Services
Service Mesh → AI Native Gateway

If Kubernetes is the “automated factory,” then AI Agents are the intelligent workers who actually get things done.

And the AI native gateway is the security and governance system tailored for these intelligent workers.

This is not an optional architecture—it is the only path for AI to reach production.

AI Open Source Landscape: A One-Stop Guide to AI Project Navigation and Scoring System

Jimmy Song — Tue, 23 Dec 2025 08:34:05 +0000

The AI Open Source Landscape is not just a project directory, but an innovative attempt to bring transparency and quantifiability to the AI open source ecosystem.

Note: This article is intended for general readers and focuses on platform features and usage scenarios. If you want to see the technical details and formulas behind the scoring, please refer to: AI Project Scoring and Inclusion Criteria.

Project Background and Positioning

The AI Open Source Landscape aims to provide developers, researchers, and enterprise users with a one-stop navigation and evaluation platform for AI open source projects. With the rapid development of large language models (LLM, Large Language Model), multimodal models (Multimodal Model), and other AI technologies, the open source community has seen a surge of innovative projects. However, information is scattered and quality varies, making it difficult for users to filter and make decisions.

Figure 1: AI Open Source Landscape

The AI Open Source Landscape systematically collects mainstream AI open source projects. As of the time of writing, it has included 851 open source projects. This landscape combines a multi-dimensional scoring system to help users efficiently discover, compare, and select the most suitable AI tools and frameworks for their needs. The platform not only focuses on models themselves, but also covers datasets, inference engines, evaluation tools, application frameworks, and the entire ecosystem chain, striving to promote transparency, quantifiability, and sustainable development in the AI open source ecosystem.

Main Interface and Feature Highlights

The platform homepage presents project distribution in both landscape and list views, supporting category filtering, keyword search, and tag navigation to help users quickly locate target projects.

Figure 2: Open Source Project Detail Page

For general readers, the main experience points include:

Card view: One-sentence overview, star rating, and overall score for quick browsing and comparison.
Health card: Displays overall health and key dimensions (activity, community, influence, sustainability) on the project page or sidebar, with the latest update marked for easy assessment of maintenance status.
Detail page: Provides more background information, project links, and application scenarios to help you evaluate suitability for your needs.
Smart badges: Visually display labels such as “Active”, “New Project”, “Popular”, “Archived” on cards, helping you quickly capture key project features.

If you are interested in the specific rules for badge determination or scoring, detailed explanations are available on the Scoring Rules Page.

Scoring and Ranking Mechanism

The platform uses multi-dimensional scores to reflect the overall health and popularity of projects. The main dimensions include: Activity, Community, Quality, Sustainability, and the comprehensive Health score. These scores help you quickly judge whether a project is suitable for production or experimentation.

Data Sources and Update Mechanism

The platform’s data mainly comes from GitHub, project lists, official documentation, and community recommendations. We regularly and automatically synchronize and update metrics to ensure that the “last updated” and scores displayed on the interface reflect the current maintenance status of projects. Projects that have not been updated for a long time or are determined to be “inactive” are moved to the Archived Page. Archived projects remain searchable and retain historical scores, but will not appear in the default view of active rankings, making it easier for readers to focus on projects that are still maintained and active.

For general readers, the key points are:

The page displays key metrics and “last updated” time, helping you quickly judge whether a project is still maintained.
The AI Open Source Landscape continuously iterates on the scoring model to improve fairness and differentiation.

How to Contribute and Correct Data

If you want a project to be included or its data updated, you can:

Submit an AI Open Source Project Inclusion Request
Keep the project’s README, License, documentation, and other information complete in the repository to facilitate our data collection and assessment.
For faster synchronization or if you encounter data issues, contact the maintainers via project issues or raise a request in the site discussion area.

Typical Use Cases or User Feedback

The AI Open Source Landscape has been widely used in various scenarios such as AI developer selection, enterprise technology research, and academic studies. For example:

Developers can quickly filter models or tools that meet their needs through the platform, saving significant research time.
Enterprise technical teams use the ranking lists for competitor analysis and technology planning.
Educational and research institutions refer to the landscape to understand trends in the AI open source ecosystem, supporting course design and topic selection.

Some users have commented that the platform is “comprehensive, well-structured, and fair in scoring,” greatly improving the efficiency of AI project selection and learning. Community suggestions continue to drive ongoing improvements in platform features and content.

Summary

The AI Open Source Landscape systematically and quantitatively organizes the AI open source ecosystem: the backend worker is responsible for reliable data collection and scoring calculations (supporting backfill and migration), while frontend components handle fast rendering and visualization (including smart badges, health cards, and metric explanations).

If you want to learn more about the scoring details or participate in improvements:

The community is welcome to join in evaluation, backfilling historical data, and refining scoring rules, working together to make the AI open source ecosystem more transparent and sustainable.

AI 2026: Infrastructure, Agents, and the Next Cloud-Native Shift

Jimmy Song — Fri, 19 Dec 2025 03:54:31 +0000

The real turning point for AI in 2026 is not autonomy, but the maturity of infrastructure - where agentic runtimes, GPU efficiency, and organizational design will decide who wins.

Introduction: 2026 Is Not an AI Moment, It Is an Infrastructure Moment

Over the past fifteen years, every major shift in software has followed a familiar arc. Microservices were adopted not out of love for distributed systems, but because monoliths reached organizational limits. Kubernetes succeeded not because containers were novel, but because infrastructure finally matched how teams operated. Cloud native was never about YAML—it was about operability at scale.

AI now stands at a similar inflection point.

The central question for 2026 is not whether models will become more autonomous. That debate overlooks the core issue. Instead, the real question is whether AI can become operable, governable, and economically sustainable within real systems.

Most organizations today are limited not by intelligence, but by infrastructure: inefficient GPU utilization, escalating inference costs, fragile agent demos, and a tendency to treat AI as a feature rather than a runtime. The next phase of AI will be shaped not by model breakthroughs, but by the maturity of AI infrastructure and its ability to absorb responsibility.

From Automation to Capability Multiplication — A Familiar Cloud-Native Pattern

Reflecting on early cloud adoption, the dominant narrative was cost reduction: fewer servers, lower CapEx, elastic scaling. Yet, the true payoff emerged later, when teams realized cloud enabled entirely new operating models.

AI is repeating this pattern.

The following diagram illustrates the shift from automation to capability multiplication.

Figure 1: From Automation to Capability Multiplication

The first wave of AI focused on labor replacement. The second wave reframes AI as capability multiplication: the same team, observing more signals, covering broader areas, and acting sooner.

This mirrors the evolution of monitoring, tracing, and SRE practices. Rather than reducing engineers, these systems enabled continuous observation instead of occasional sampling.

Preemptive AI systems—monitoring every interaction, log, and signal—are only viable if the underlying infrastructure can support them. This exposes a critical constraint: AI capability scales faster than AI infrastructure.

Without efficient scheduling, isolation, and utilization, multiplying capability simply multiplies cost.

Agents Are Becoming Distributed Systems, Whether We Admit It or Not

The industry often discusses agents as products. In reality, agents are evolving into distributed systems.

The diagram below highlights this architectural shift.

Figure 2: Agents Are Becoming Distributed Systems

Single-agent designs resemble early monoliths: impressive demos, fragile behavior, and opaque failure modes. As tasks grow in complexity, systems must decompose work into planning, execution, verification, and review—making coordination inevitable.

This is not merely a philosophical change, but an architectural one.

Multi-agent systems introduce challenges familiar from the microservices era:

Coordination and orchestration
Resource contention
Fault isolation
Observability and rollback
Deterministic artifacts between stages

Labeling this as “multi-agent collaboration” can be misleading. What is actually occurring is workload decomposition and control-plane emergence. Agents are transitioning from tools to workloads competing for limited resources.

Recognizing this clarifies why agent progress is inseparable from infrastructure maturity.

AI Infra Is the Missing Layer Between Models and Organizations

Cloud native taught us that abstractions only scale when a control plane exists.

Currently, AI lacks a mature control plane.

The following image demonstrates the gap between models and organizations.

Figure 3: AI Infra Is the Missing Layer Between Models and Organizations

Models are powerful, but the surrounding infrastructure—scheduling, isolation, quota enforcement, cost attribution, observability—remains primitive, especially at the GPU layer.

GPUs are expensive, scarce, and often underutilized. In many environments, utilization remains below 30–40%, while inference costs continue to rise. Training pipelines monopolize resources, inference workloads spike unpredictably, and organizations must choose between waste and throttling innovation.

This is not a model problem. It is fundamentally an AI infrastructure problem.

The next phase of AI will depend on treating GPUs as we learned to treat CPUs:

Fine-grained allocation
Fair sharing
Preemption and prioritization
Clear ownership and accounting

Until GPU utilization becomes a primary design goal, AI systems will remain economically fragile.

Domain Expertise Matters Because Infrastructure Finally Exposes It

As models plateau in general reasoning, differentiation shifts elsewhere.

The diagram below illustrates how infrastructure exposes domain expertise.

Figure 4: Domain Expertise Matters Because Infrastructure Finally Exposes It

In cloud-native systems, competitive advantage eventually moved from frameworks to operational excellence: superior runbooks, incident response, and cost control. AI is following a similar trajectory.

High-value AI systems must operate within dense, rule-heavy domains such as finance, healthcare, manufacturing, and infrastructure operations. What matters is not abstract intelligence, but the ability to encode domain constraints, exceptions, and failure patterns.

Here, domain experts become central—not as prompt engineers, but as system shapers. Their decisions define agent permissions, human intervention points, and error containment strategies.

Infrastructure determines whether this expertise can be safely operationalized.

Simulation Is Becoming the New Staging Environment for AI

One of the most important lessons from cloud-native operations: distributed systems are not tested in production.

AI systems that act, plan, and modify state are no exception.

The following image shows simulation as the new staging environment.

Figure 5: Simulation Is Becoming the New Staging Environment for AI

Training and validating agents directly in live environments is unsustainable. The future lies in simulation-first AI development—sandboxed environments that mirror real systems, workloads, and constraints.

This approach is analogous to staging clusters, chaos engineering, and load testing, but elevated for decision-making systems. Evaluation shifts from static benchmarks to behavioral metrics: intervention rates, rollback frequency, and cost impact.

Organizations that build these environments will advance faster and safer. Those that do not may remain limited by conservative deployments and restricted autonomy.

Summary

Technological revolutions succeed not on novelty alone, but when infrastructure, tooling, and organizational models align.

AI is nearing that pivotal moment.

The leaders in 2026 will be those who:

Treat AI as a runtime, not just a feature
Optimize for resource efficiency, especially GPUs
Recognize agents as distributed systems
Redesign organizations around continuous learning systems
Invest in infrastructure ahead of autonomy

AI is no longer just a model problem. It is an infrastructure challenge—and the next phase will be decided not in labs, but in production systems.

What I Saw at COSCon'25: The Real State of Open Source in China

Jimmy Song — Thu, 18 Dec 2025 06:14:51 +0000

Attending COSCon'25 in Beijing, I observed firsthand how open source in China is shifting: AI is now the default context, discussions are grounded in real engineering, and the community is embracing long-term thinking. These are not just trends—they are the new reality.

In early December this year, I attended COSCon'25, the China Open Source Annual Conference, in Beijing. Although I have worked in open source for many years, this was my first time participating in an event organized by the Open Source Society—and I joined as a sub-forum producer. Previously, I thought such conferences were too high-level or disconnected from reality, but after actually taking part, I found there was much to gain.

A quick note: this article is not an official conference summary or review. The organizers have already published detailed information about the event’s scale, attendee numbers, and forum sessions. If you’re interested in those details, please refer to the official article: COSCon'25: The 10th China Open Source Annual Conference Successfully Concludes in Beijing—A Comprehensive Recap!

Figure 1: 10th COSCon Venue

What I want to share is this: Standing on site, on the engineering front lines, and as an organizer rather than an audience member, I saw real changes happening in Chinese open source.

This COSCon: No More Trying to “Prove Open Source Matters”

One clear impression: Almost no one spent time arguing “why do open source” anymore.

In earlier years, common narratives included:

Is open source safe?
Can open source be commercialized?
Can China create its own open source projects?

But at COSCon'25, these questions were basically assumed as “background conditions.” The focus shifted to those already doing open source, and what comes next.

This doesn’t mean the issues have disappeared, but it does mean:

In China’s engineering circles, open source is no longer a “philosophical choice”—it’s a practical way of working.

AI as Background Noise, Not the Main Character

The theme of this year’s conference was Open Source × Open Intelligence, but interestingly, AI did not take center stage.

Instead, it was more like background noise— Almost every topic touched on AI, but no one was giving talks solely “about AI.”

You would see it repeatedly in areas like:

Cloud native scheduling, focusing on GPU / NPU / heterogeneous resources
Storage and data, focusing on data paths for training and inference
Serverless, focusing on LLM cold starts and elasticity
Observability, focusing on what to do when system complexity gets out of hand

AI was not treated as a “hot trend,” but as a new workload reality. This is a significant change, though not one easily captured in press releases.

Real Impressions as a Cloud Native Sub-forum Producer

I helped organize the cloud native open source sub-forum at this year’s conference. This role gave me a perspective very different from that of a typical attendee.

First, Topics Clearly Converged on “Engineering Problems”

There were almost no talks about Kubernetes concepts; Very few about “architectural philosophies.”

Instead, the focus was on:

What pitfalls did you encounter at what scale?
Why did you choose this solution over another?
Which problems remain unsolved?

Many presentations weren’t “pleasant to hear,” but they were very real.

Second, The Boundary Between Academia and Industry Is Thinning

This was especially evident this year.

Some talks from universities and research institutes were no longer just “from a paper’s perspective,” but directly addressed core issues in industrial systems, such as:

Cold start of serverless LLMs
The real value of RDMA in inference paths
Whether prefill/decode separation is truly feasible in engineering

These topics may not be immediately applicable, but they are now colliding head-on with engineering problems, rather than talking past each other.

Third, Open Source Is No Longer Just About Code

In many discussions, “governance,” “maintenance cost,” and “community collaboration” came up frequently.

This is a signal: When a project is truly being used, code is no longer the hardest part.

Main Forum: More Questions, Not Answers

If I had to sum up the main forum in one sentence: It kept raising questions, but wasn’t in a hurry to provide answers.

For example:

Has the boundary of open source changed in the AI era?
Should models, data, and chips become part of the open source core?
Are developers’ roles being redefined?

There are no standard answers to these questions, but the fact that they are being raised repeatedly shows they have become common concerns, not just the thoughts of a few.

Exhibition Area and Sub-forums: Closer to the Real Ecosystem

Compared to the main forum, I personally paid more attention to the sub-forums and exhibition area.

There, you would see:

Many projects no longer emphasize “who they want to replace”
More discussions about “who they can work with”
Several communities are seriously discussing long-term maintenance, not just releasing versions

This may not be glamorous, but it’s important.

A Personal Judgment

If I had to make a judgment about COSCon'25, I would say:

Chinese open source is shifting from “can we do it” to “can we sustain it for the long term.”

This is a more difficult, but also more realistic, stage.

This COSCon did not try to create a grand narrative. Instead, it felt like a “status exposure” at a particular stage: There are more questions, participants are more diverse, but the discussions are also closer to the real world.

Open source doesn’t depend on a single conference to move forward, but being on site helps you see more clearly: Where exactly are we standing right now?

Decoding Goose: Why It Joined AAIF and What This Means for Agentic Runtime

Jimmy Song — Fri, 12 Dec 2025 08:16:48 +0000

Goose is not a project that excites you at first glance in this wave of Agent innovation, but its entry into AAIF signals a deeper shift in how we think about Agentic Runtime and AI-Native infrastructure.

At first glance, Goose is not a project that immediately excites people.

Figure 1: Goose App UI

It doesn’t have flashy demos, nor does it showcase overwhelming multimodal capabilities, and it certainly doesn’t look like an AI product aimed at consumers. Yet, this seemingly “plain” project became one of the first donations to the Agentic AI Foundation (AAIF), standing alongside Anthropic’s MCP and OpenAI’s AGENTS.md.

This fact alone is worth a closer look.

This article does not aim to prove how powerful Goose is, but rather to answer three more practical questions:

What overlooked but long-term critical problems does Goose actually solve?
Why was it Goose, and not another Agent framework, that entered AAIF?
What does this mean for Agentic Runtime and AI-Native infrastructure, which I care about?

Goose’s True Positioning: It’s Not an IDE or a Chatbot

If you only look at its surface features, Goose is easily mistaken for one of two things:

A “multi-model AI desktop client”
Or an “intelligent programming assistant that can run commands”

But inside Block, it was never designed as a “tool” from the start.

Goose’s origin is closely tied to Block’s engineering environment.

Block (formerly Square) is a classic engineering-driven company: complex systems, high automation needs, many internal tools, and very high execution costs in real production environments. In its recent AI transformation, Block did not focus on “which model to choose” or “which AI tool to introduce,” but directly targeted the engineering execution layer itself.

Goose was born in this context.

Its goal is not to “help people code faster,” but to enable models to stably and controllably take action: run tests, modify code, drive UIs, call internal systems, and operate reliably in real engineering environments.

In short:

Goose is more like an executable Agent Runtime than a conversation-centric product.

Block’s AI Transformation Started with Organization, Not Tools

To understand Goose, you can’t ignore a key organizational shift at Block.

In an interview with Block’s CTO, one signal was very clear: the starting point for AI transformation was not buying tools or stacking models, but the organizational structure itself.

Block shifted from a business-line GM model to a more functionally oriented structure, making engineering and design the company’s core scheduling units again. This is essentially a proactive response to Conway’s Law.

If the organizational structure doesn’t allow technical capabilities to be orchestrated centrally, Agents will ultimately remain “personal assistants” or “engineering toys.”

From this perspective, Goose is not just a tool, but a cultural signal:

Every employee can use AI to build and execute real system behaviors.

This also explains a fact many overlook: Goose was not packaged as SaaS, nor was it rushed to commercialization, but was open-sourced and rapidly standardized.

Because its role inside Block is closer to an “operating system for execution models” than a product that can be sold separately.

Why Did Goose Enter AAIF? Not Because It’s “Technically Strongest”

This is what confuses outsiders the most.

If you only look at flashy features, model support, or community popularity, Goose doesn’t stand out. But AAIF’s choice was not about “maximum capability,” but about whether the position is right.

Looking at the first batch of AAIF projects, a clear chain emerges:

MCP (Anthropic): Defines how models safely and standardly call tools
AGENTS.md (OpenAI): Defines behavioral conventions for Agents in code repositories
Goose (Block): A real, runnable, open-source Agent execution framework

Goose’s role is not to set new protocols, but to serve as the practical carrier and reference implementation for these protocols.

It proves one thing:

MCP is not just a paper standard
Agents are not just research concepts
In real enterprise environments, they can actually run

From this angle, Goose’s “ordinariness” is actually an advantage.

It is not tied to Block’s business moat, nor does it have irreplaceable private APIs; it can be forked, replaced, audited—“boring” enough, and neutral enough.

And that is the most important trait of public infrastructure.

Goose’s Value Lies Not in Today, But in 2–3 Years

From a longer-term perspective, Goose’s value becomes clearer.

What we’re experiencing now is much like the early days of containers: Most Agent projects today are demos, IDE plugins, or workflow wrappers, but what’s really missing is a sustainable, schedulable, observable execution layer.

Goose is already moving in this direction.

Block’s metrics for Goose’s success are straightforward:

How many human hours are saved each week
How much non-technical teams reduce their dependence on engineering teams

Behind this is a judgment I’m increasingly convinced of:

What enterprises truly need is not “smarter models,” but “cheaper execution.”

The long-term value of Agents is not in generation quality, but in execution substitution rate.

AAIF Is an Attempt at Infrastructure-Level Consensus

Just as CNCF did for cloud native, AAIF is not guaranteed to succeed.

But it at least marks a shift: Agents are no longer just application-layer innovations, but are beginning to enter the stage of infrastructure-layer collaboration.

As a reference implementation, Goose is likely to remain in this ecosystem for a long time—even if it is replaced, rewritten, or evolved in the future.

Summary

If you see Goose as a “product,” it is indeed not dazzling.

But if you place it in the long-term evolution path of Agentic AI, its significance becomes clear:

It is not the end, but a necessary intermediate state.

For me, the emergence of Goose further confirms one thing:

Agentic Runtime is not a conceptual problem, but an engineering and organizational one.

And that is one of the most worthwhile directions to invest energy in over the next few years.

ARK: Multi-Agent Systems Are Finally Entering the Engineer's World

Jimmy Song — Thu, 11 Dec 2025 13:19:42 +0000

The deep integration of cloud native and AI, with the ARK platform, provides a new paradigm for engineering multi-agent systems.

Introduction

AI Agents are moving from the “single agent demo” stage to “large-scale operation.” The real challenge does not lie in the model itself, but in engineering issues at runtime: model management, tool invocation, state maintenance, elastic scaling, team collaboration, observability, deployment, and upgrades. These are problems that traditional agent libraries struggle to solve.

ARK (Agentic Runtime for Kubernetes) provides a fully operational, observable, governable, and continuously deliverable multi-agent operating system. It is not a Python library, but a complete runtime platform.

Figure 1: ARK Dashboard

Note: In this article, ARK refers to McKinsey’s open-source ARK Agent Runtime for Kubernetes.

This article, from an engineer’s perspective, will reorganize ARK’s core capabilities and answer the following questions:

What engineering challenges does ARK actually solve?
Why is it worth special attention in the cloud native field?
How is it fundamentally different from frameworks like LangChain and CrewAI?
What insights does it offer for the Agentic Runtime ecosystem?

ARK Architecture: Treating Agents as Kubernetes-Native Workloads

The core idea of ARK is: An agent is not a script, but a schedulable, governable, and observable Kubernetes workload.

The following architecture diagram illustrates ARK’s underlying structure.

Figure 2: ARK Overall Architecture

This diagram highlights ARK’s key design points:

CRDs declare requirements (Agent, Model, Team, Tool, Memory, etc.)
The Controller translates declarations into actual Pods/Services
The API provides a unified communication entry point and team orchestration
Memory supports long-term state management for agents
MCP Server enables external systems to become tools
Dashboard provides visual management and observability

ARK adopts the typical cloud-native Operator pattern and applies it to multi-agent systems.

CRD: ARK’s “Abstraction Layer”

Unlike traditional agent frameworks where “code is logic,” ARK uses CRDs (Custom Resource Definitions) to abstract the components of agent applications.

The main CRD types in ARK include:

Model
Agent
Team
Tool
Memory
Evaluation

These CRDs correspond to all the key components of an agent system.

The following diagram shows the structure of the CRDs:

Figure 3: CRD Structure (Simplified)

Through CRDs, ARK achieves the following engineering features:

All resources are GitOps-ready, supporting declarative management
Changes are auditable, reversible, and continuously deliverable
The evolution of models, tools, and agents does not require business code changes

This is the key gene of ARK’s engineering-oriented system.

Agent Execution Flow: From Query to Tool Invocation

The following image shows how to view query details in the ARK Dashboard.

Figure 4: Viewing Query Details in ARK Dashboard

In ARK, the complete execution flow for an agent receiving a query is as follows:

Figure 5: Agent Execution Flow

This flow has the following characteristics:

Memory modules are naturally involved in the execution flow, without code specialization
Large language model (LLM, Large Language Model) and tool invocation are governed by the runtime
Agents can reside in Pods long-term, not just as one-off processes

This makes ARK more like an “agent microservice platform.”

Below is an example of a request and response:

The True Value of Multi-Agent: Team Orchestration

ARK’s Team CRD allows multiple agents to be woven into a higher-level “system,” enabling multi-agent collaboration.

The following diagram shows the collaboration model of a multi-agent team:

Figure 6: Multi-Agent Team Collaboration

The engineering value of Team is reflected in:

Making “expert collaboration” declarative and configurable
Flexible strategies (such as polling, role assignment, routing, etc.)
A2A Gateway handles message passing
The Team itself is observable (every round of collaboration is logged)

For enterprises, this means the “agent organizational structure” can be standardized, replayed, and tuned.

Fundamental Differences Between ARK and Other Frameworks

Many engineers, upon first seeing ARK, may wonder:

“Is it just LangChain or CrewAI wrapped in Kubernetes?”

In fact, there are fundamental differences. The following diagram compares the structural differences between ARK and mainstream agent frameworks:

Figure 7: ARK vs LangChain / AutoGPT / CrewAI

The table below further summarizes the key differences:

Dimension	Traditional Agent Libraries	ARK
Core Pattern	Write Python code	Write CRDs (declarative)
Deployment	Local/Container	Kubernetes-native scheduling
State	Managed inside code	Memory CR + Service
Tools	Integrated at code level	Tool CR + MCP
Multi-Agent	Dialog managed in code	Team CR + A2A protocol
Observability	Almost none	OTel / Langfuse / Dashboard
Use Cases	Demo / Prototype / Single Agent	Enterprise production / Multi-Agent Systems

Table 1: ARK vs Traditional Agent Libraries

In short:

LangChain is a “library for building agents,” while ARK is a “platform for running agents.”

The two are not in conflict and are, in fact, highly complementary.

The Engineering Value of ARK

To summarize ARK’s engineering value in simple terms:

Turns agents into governable workloads
Unifies models, tools, and memory as reusable resources
Makes multi-agent collaboration structured, observable, and tunable
Brings agent upgrades and iteration into CI/CD + GitOps mode
Enables enterprises to manage agents like microservices

This is a clear evolution path:

Agent → Service → Platform → Runtime → Operating System

ARK is currently positioned at the fourth stage: Runtime.

Insights for Agentic Runtime

ARK provides three direct insights for building Agentic Runtimes:

Unified Scheduling System

The agent runtime must run on a unified scheduling system (Kubernetes, MicroVM, Wasmtime, etc.)

Declarative Capability Boundaries

Must use declarative abstractions to split capability boundaries, including:
- Model Layer
- Tool Layer
- Memory Layer
- Workflow Layer
- Team Layer
- State Layer

Observability

Observability is essential; otherwise, multi-agent systems cannot be engineered
- Langfuse
- OTel
- Logs / Events
- Structured JSON

ARK demonstrates a direction:

Multi-agent systems are an engineering problem, not a prompt engineering problem.

Summary

If you only need to build a simple agent, frameworks like LangChain, CrewAI, and AutoGPT are sufficient.

But if you want to operate a system composed of dozens or hundreds of agents that need to collaborate, run long-term, and support continuous delivery and governance, runtimes like ARK are the inevitable trend.

It provides Agentic AI with:

A cloud-native runtime model
Observable execution paths
Governable abstraction layers
Extensible, componentized architecture

Therefore, ARK deserves to be regarded as an early model for engineering multi-agent systems.

Can Open Source Suddenly Disappear? An AI Chat Dev Tool Went 404 Overnight

Jimmy Song — Thu, 11 Dec 2025 05:20:12 +0000

“Open source” in the AI era is no longer a trustworthy promise. Commercial projects can withdraw their code at any time, and developers must be wary of the gap between appearances and reality.

The Disappearance of Lunary’s Repository: A Real Case of Open Source “Vanishing”

While updating the AI open source project library on my website, I encountered a situation that left me stunned for the first time: An “open-source AI tool” that still promotes itself, with an active website and commercial services, suddenly vanished from GitHub—its repository went straight to 404.

The project is called Lunary.

Original repository address: https://github.com/lunary-ai/lunary It now returns a 404 Not Found.

Notably, the official site lunary.ai remains online, but the core promise of an “open-source codebase” has disappeared.

Lunary’s Positioning and Features

Here is an overview of Lunary’s main features and positioning to help understand its role in the AI tool ecosystem.

Lunary claims to be an Observability and Evaluations platform for large language model (LLM, Large Language Model) applications, focusing on:

LLM conversation and feedback logs
Cost, latency, and metrics analysis
Prompt version management
Distributed tracing
Evaluations
Supports both self-hosted and managed modes
Provides JS / Python SDKs, integrates with LangChain

Its overall positioning is clear: “Development and debugging tools for AI applications.”

In fact, products like this have emerged rapidly over the past year, forming a new AI DevTool track.

The Reality and Risks Behind the “Open Source” Label

The core issue is not the tool itself, but its claim to be “open source.”

Lunary has consistently emphasized:

“Lunary is an open-source platform for developers.”

This statement is great for attracting users, as open source implies transparency, trustworthiness, self-hosting, and community participation.

But now the repository is gone, with only the website continuing its promotion—raising many questions.

Lunary is not a niche hobby project, but a commercial company-led initiative. If an individual suddenly deletes a repo, it’s not surprising, but for a company operating publicly, this move is extremely rare.

This is the first time I’ve truly seen a reality in the AI DevTools space: “Open source” is being used as a branding term, not a commitment.

Possible Industry Reasons for Repo Deletion

Let’s analyze some common industry reasons for deleting a repository to help developers understand the motivations behind such actions.

Increased commercial pressure: These tools often struggle with sustainable business models, prompting teams to shift to closed-source SaaS.
Pivoting: The company finds the original direction unprofitable and prepares to change course.
Team changes: Acquisition, key member departures, or funding issues can all lead to repo shutdowns.
Compliance or legal risks: Observability products involve user data, which may require public code to be taken down.

Regardless of the reason, the impact on users is the same: it is no longer an “open-source product.”

The “Pseudo Open Source” Phenomenon in AI Tools

The most noteworthy aspect is not Lunary itself, but the rapid spread of this phenomenon in the AI tool space.

Many projects use “open source” as a user acquisition strategy but lack open governance and long-term commitment.

High substitutability, homogeneity, and commercial pressure mean these DevTools have low survival rates.

When commercial teams lead open source, a single decision can make the repository disappear instantly.

In the cloud native era, we’ve already seen a wave of “pseudo open source.” In the AI era, this trend is accelerating.

Three Practical Lessons for Developers

Based on this case, here are three practical lessons for developers:

The “open source label” does not guarantee trustworthiness: Open source projects led by commercial companies without community or foundation backing can be withdrawn at any time.
AI DevTools are far less stable than infrastructure: These tools are not essential, highly replaceable, and have short lifecycles.
Tool usability should take precedence over “open source status”: Because it may stop being open source at any moment.

My First Experience Maintaining an AI Project List and Facing Repo Deletion

After collecting hundreds of projects over the past two years, this is the first time I’ve encountered a “commercial open source project disappearing, official repo 404” case.

To me, this is an industry signal: the AI open source world is entering a period of drift, and commercial projects’ open source commitments are increasingly unstable.

It also reminds everyone making technical choices: in the AI era, open source is no longer a label you can automatically trust.

Summary

The disappearance of the Lunary repository is not an isolated incident, but a reflection of the “pseudo open source” phenomenon in the AI tool space. Developers should be cautious about the actual commitments behind the “open source” label, paying attention to project governance and sustainability. In the future, the boundary between open source and commercial will become even more blurred, and rational judgment and risk awareness will be essential for technical decision-making.

Lunary’s sudden disappearance highlights the instability of open source projects in the AI DevTools space. For developers, technical choices should focus more on project usability and community governance, rather than relying solely on the “open source” label. As the industry evolves, similar incidents may become more frequent. Only rational judgment and risk awareness can help you stand firm in the fast-changing tech landscape.

CNCF in the AI Native Era? The Agentic AI Foundation Is Officially Established

Jimmy Song — Wed, 10 Dec 2025 03:25:38 +0000

The standardization and open collaboration of the agent ecosystem is no longer a luxury, but the critical watershed for whether AI Native can be engineered and implemented.

The establishment of AAIF (Agentic AI Foundation) is the result of leading vendors staking out the “agent protocol layer” in advance.
The real challenge is not technical, but how organizations transition from “human execution + AI assistance” to “agent execution + human supervision”.
Successful agent adoption requires a phased adoption path, not just a bunch of protocols and demos.
CNCF and AAIF are complementary: CNCF manages “what infrastructure agents run on”, AAIF manages “how agents collaborate”. This matches the system I am building in ArkSphere.

Cloud Native Problems Are Solved, AI Native Problems Are Just Beginning

Over the past decade, Cloud Native technologies like Kubernetes, Service Mesh, and microservices have standardized “how applications run in the cloud”. But AI Native faces a completely different challenge: It’s not about “how to deploy a service”, but “how many behaviors in the system can be handed over to agents to execute themselves”.

CNCF’s Cloud Native AI (CNAI) addresses infrastructure-level issues: “How can model training/inference/RAG run at scale and securely on Kubernetes?”

But what AI Native truly lacks is another layer: How do agents collaborate, access tools, get governed, and audited?

This is exactly the gap AAIF aims to fill.

AAIF’s Three Weapons: Protocol + Runtime + Development Standard

AAIF hosts three core technologies contributed by its founding members:

① Anthropic’s Model Context Protocol (MCP)

https://github.com/modelcontextprotocol

A “system call interface for agents”:

Unified definition for how agents access databases, APIs, files, and external tools.
Designed to be more like an AI version of gRPC + OAuth.
Already integrated by Claude, Cursor, ChatGPT, VS Code, Microsoft Copilot, Gemini, and others.

It may not be the flashiest technology, but it could become the plumbing for the entire Agentic ecosystem.

② Block’s Goose Framework

https://github.com/block/goose

Reference runtime for MCP:

Local-first, composable agent workflow engine.
Enables enterprises to pilot agents in small scopes without betting on a specific vendor.
Serves as an engineering template for protocol implementation.

③ OpenAI’s AGENTS.md

https://agents.md

A simple but effective standard:

Place an AGENTS.md file in the project repository.
Clearly document build steps, testing, constraints, and context rules.
Any agent that understands AGENTS.md can operate the codebase using the same instructions.

This makes agent behavior more predictable and auditable.

Why Is AAIF in Such a Hurry? This Is a Race for Standards

Let’s compare with history:

Kubernetes’ predecessor Borg ran internally at Google for over a decade; K8s was open sourced and donated to CNCF two years later.
PyTorch joined the Linux Foundation six years after its release.
MCP was donated to AAIF just over one year after its launch.

AAIF is not about “mature technology entering a foundation”, but staking out the key position early.

The reasons are practical:

Prevent agent ecosystem fragmentation Today, there are many competing “tool invocation protocols”, which could become incompatible silos in three years.
Protocol layer is easier to reach global consensus than model layer Model competition is inevitable, but protocols can be standardized, open sourced, and avoid vendor lock-in.
A necessary move in global tech competition Putting the foundational standards for Agentic AI into the Linux Foundation is both a gesture of cooperation and a strategic move.

AAIF vs CNCF: Not Competition, But Two Pieces of the Puzzle

CNCF’s role:

“What infrastructure do agent workloads run on?” Kubernetes, Service Mesh, observability, AI Gateway, RAG Infra—all at this layer.

AAIF’s role:

“How do agents collaborate, invoke tools, and get governed?” Protocols, runtimes, and behavioral standards—all at this layer.

Analogy:

Domain	Responsibilities
AAIF	Semantic and collaboration layer of Agentic Runtime
CNCF/CNAI	Resource and execution layer of AI Native Infra

Table 1: AAIF vs CNCF Comparison

This matches the upper semantic and lower infrastructure layers in my ArkSphere architecture diagram.

In the long run, the two sides will be tightly coupled: CNCF’s KServe, KAgent, and AI Gateway will natively support MCP / AGENTS.md, AAIF’s Runtime will run on Cloud Native infrastructure by default.

The Real Challenge: Not Protocols, But Organizations and People

Most enterprises will get stuck on:

How much responsibility can agents actually take?
Who is accountable when things go wrong?
How are audit, SLOs, and compliance defined?
How is multi-agent collaboration visualized?
How are tool invocation permissions controlled?

In other words, agent adoption is not a “technical migration”, but an “organizational migration”.

If AAIF cannot provide:

Phased adoption methodologies
Typical organizational migration paths
Engineering best practices
Failure cases and anti-patterns

It will be difficult for AAIF to achieve the industry impact that CNCF did.

Summary: AAIF Is the Moment When Boundaries Are Drawn

For me, the establishment of AAIF feels like:

“The battlefield boundaries of the agent world have finally been drawn. Now it’s up to the engineering community to make it work.”

CNCF solved “how to run Cloud Native”, AAIF is now trying to solve “how agents collaborate”.

In the next five years, whoever can truly connect these two worlds will stand at the gateway to the next generation of infrastructure.

That’s why I started a dedicated “Agentic Runtime + AI Native Infra” research track in ArkSphere.

The ‘Three-Body’ Architecture of the AI Native Era

Finally, a personal note—my thoughts on ArkSphere.

Figure 1: AAIF × CNCF: Three-Layer Architecture of Agentic AI in the AI Native Era

This diagram shows the three-layer structure of the AI Native era:

CNCF (bottom layer): Provides the Cloud Native foundation required for agent operation, including Kubernetes, Service Mesh, GPU scheduling, and security systems.
AAIF (middle layer): Defines the runtime semantics and standards for agents, including the MCP protocol, Goose reference runtime, and AGENTS.md behavioral standard.
ArkSphere (bridging layer): Aligns the “Agentic Runtime semantic layer” with the “AI Native Infra infrastructure layer”, forming an engineerable agent architecture standard.

In short:

Infra is responsible for “running”, Runtime for “how to act”, and ArkSphere for “how to assemble a system”.

KCD Beijing + vLLM 2026: Kubernetes × AI × LLM Inference, A Community-Driven Tech Event

Jimmy Song — Fri, 05 Dec 2025 18:46:36 +0800

As Kubernetes becomes the de facto standard for AI infrastructure, and as large language model inference enters engineering and scaling stages, Cloud Native and AI are truly converging.

Therefore, KCD Beijing and the vLLM community have decided to do something together: bring the Kubernetes community and the LLM inference community to the same stage. KCD Beijing + vLLM 2026 is officially launching 🚀

KCD Beijing logo

About KCD Beijing + vLLM 2026

Kubernetes Community Days (KCD) is a CNCF-sponsored, community-organized Kubernetes technology conference that emphasizes community-driven collaboration, real-world practices, and engineering experience sharing.

KCD Beijing + vLLM 2026 will be co-hosted by the KCD Beijing community and the vLLM community, representing a deep collaborative partnership between the Cloud Native community and the LLM inference community.

We hope this will be more than just a conference—it’s a technical connection around these themes:

Real-world experience with Kubernetes in production
Systematic thinking about AI/ML Infra and AI-related practices
vLLM engineering practices in LLM inference
Complete chain from cluster scheduling to model serving

Event Details

Event Name: KCD Beijing + vLLM 2026
Date: March 21, 2026
Co-organizers:
- Kubernetes Community Days Beijing
- vLLM Community

Three Parallel Sessions · Complete Technical Stack Coverage

This conference will feature three parallel sessions, forming a complete loop from infrastructure to model inference:

☸️ Kubernetes Track

Kubernetes production practices
Platform engineering / multi-cluster governance
Networking, storage, security, scheduling
CNCF ecosystem project experiences

🤖 AI / ML Track

AI / ML Infra architecture design
GPU / heterogeneous computing scheduling
Training and inference platform construction
AI-related technology practices
Integration of AI with Cloud Native

🚀 vLLM Track

vLLM architecture and core principles
High-performance LLM inference practices
vLLM + Kubernetes deployment cases
Inference performance optimization and resource management
vLLM ecosystem and future directions

If you’re running AI workloads on Kubernetes, if you’re using vLLM to build inference services, if you’re researching and using AI-related technologies, then this is your stage.

Call for Proposals (CFP)

We sincerely invite engineers, architects, maintainers, and community contributors to submit proposals.

Session Formats

Standard Presentation: 30 minutes · solo or duo speakers
Lightning Talk: 10 minutes · quick, direct, focused on a single point

We value real-world experience, actual problems, and clear thinking over “perfect stories.”

Important Dates

CFP Opens: December 5, 2025
CFP Closes: February 24, 2026
Acceptance Notification: Within 1–2 weeks after the deadline

How to Submit a Proposal

Prepare a clear abstract: What will you talk about? Who is it for? What will attendees gain?
Choose your track and session format
Submit your proposal: https://sessionize.com/kcd-beijing-2026/

Thank You to Our Sponsors and Partners

KCD Beijing + vLLM 2026 would not be possible without the support of our sponsors and community partners.

Special thanks (in no particular order):

AWS
Red Hat
Huawei
Ant Open Source
OceanBase
KubeEvents
AtomGit

Your continued investment in the Cloud Native and AI open source ecosystem enables our community to go further.

Detailed information about sponsors and partners will be released in the next article. Stay tuned.

Why Should You Participate?

This is a co-hosted event by KCD Beijing × vLLM communities
This is a direct convergence of Kubernetes and LLM inference
This is an engineering practice-focused community conference
This is a stage where frontline engineers’ voices are heard

Whether you are:

Kubernetes / Platform Engineer
AI / ML Infra Developer
vLLM user or contributor
Long-term participant in Cloud Native or AI communities

KCD Beijing + vLLM 2026 welcomes you.

Register to attend: https://www.bagevent.com/event/kcd-beijing-2026

Please fill out the form below to apply as a volunteer.

Volunteer Application

Meet the KCD Beijing Organizers

KCD Beijing Organizers

Miley Fu: WasmEdge DevRel, CNCF Ambassador
Jintao Zhang: CNCF Ambassador, Kong
Iceber Gu: CNCF Ambassador, DaoCloud
Jimmy Song: CNCF Ambassador, VP of Open Source Ecosystem at Dynamia.ai
Dinah Zhang: OceanBase DevRel
Qing Hao: CNCF Ambassador, Red Hat
Zhenyu Zheng: Senior Engineer at Huawei, Head of openEuler Operations
Shuangkun Tian: Argo Maintainer, Alibaba Cloud
Addo Zhang: CNCF Ambassador
Betty Zheng: Senior Developer Advocate, AWS

Stay tuned for further updates. For sponsorship, open source booths, and other collaborations, please contact:

Please fill out the form below to apply as a partner community.

Partner Community Application

Please share this with your colleagues and friends, and we welcome you to take the stage yourself.

CFP Open · Speaker Recruitment · Community Building

See you in Beijing on March 21, 2026.

Bun Acquired by Anthropic: A Structural Signal for AI-Native Runtimes

Jimmy Song — Wed, 03 Dec 2025 05:21:28 +0000

The shifting ownership of runtimes is reshaping the underlying logic of AI programming and infrastructure.

After the announcement of Bun’s acquisition by Anthropic, my focus was not on the deal itself, but on the structural signal it revealed: general-purpose language runtimes are now being drawn into the path dependencies of AI programming systems. This is not just “a JS project finding a home,” but “the first time a language runtime has been actively integrated into the unified engineering system of a leading large model company.”

This event deserves a deeper analysis.

Bun’s Engineering Features and Current Status

Before examining Bun’s industry significance, let’s outline its runtime characteristics. The following list summarizes Bun’s main engineering capabilities:

High-performance JavaScript/TypeScript runtime
Built-in bundler, test framework, and package manager
Single-file executable
Extremely fast cold start
Node compatibility without Node’s legacy dependencies

These capabilities have formed measurable performance barriers.

However, it should be noted that Bun currently lacks the core attributes of an AI Runtime, including:

Permission model
Tool isolation
Capability declaration protocol
Execution semantics understandable by models
Sandbox execution environment

Therefore, Bun’s “AI Native” properties have not yet been established, but Anthropic’s acquisition provides an opportunity for it to evolve in this direction.

The Significance of a Leading Model Company Acquiring a General-Purpose Runtime

Historically, it is not uncommon for model companies to acquire editors, plugins, or IDEs, but in known public cases, mainstream large model vendors have never directly acquired a mature general-purpose language runtime. Bun × Anthropic is the first clear event pulling the runtime into the AI programming system landscape. This move sends two engineering-level signals:

The speed of AI code generation continues to increase, amplifying the need for deterministic execution environments. The generate→execute→validate→destroy cycle intensifies the problem of environment non-repeatability.
Models require a “controllable execution substrate” rather than a traditional operating system. Agents are not suited to run tools in an uncontrollable, unpredictable OS layer.
The runtime needs to be embedded into the model’s internal engineering pipeline. Future IDEs, agents, and auto-repair pipelines may directly invoke the runtime’s API.

This is not a short-term business integration, but a manifestation of the trend toward compressed engineering pipelines.

Runtime Requirements Differentiation in the AI Coding Era

Based on observations of agentic runtimes over the past year, runtime requirements in the AI coding era are diverging. The following list summarizes the main engineering abstractions trending in this space:

Determinism: AI-generated code is not reviewed line by line; execution results must be consistent across machines and over time.
Minimal distribution unit: Users no longer install language environments and numerous dependencies. Verifiable, replicable, and portable single execution units are becoming the norm.
Tool isolation: Models cannot directly access all OS capabilities; the context and permissions visible to tools must be strictly defined.
Short-lived execution: Agent invocation patterns resemble “batch jobs” rather than long-running services.
Capability declaration: The runtime must expose “what I can do,” rather than the entire OS interface.
Embeddable self-testing pipeline: After generating code, models need to immediately execute tests, collect errors, and iterate. The runtime must provide observability and diagnostic primitives.

These requirements are not unique to Bun, nor did Bun originate them, but Bun’s “monolithic and controllable” runtime structure is more conducive to evolving in this direction.

Figure 1: Minimal execution loop of an AI-native runtime

Bun’s Potential Role Within Anthropic’s System

If Bun is seen merely as a Node.js replacement, the acquisition is of limited significance. But if it is viewed as the execution foundation for future AI coding systems, the logic becomes clearer:

Code is generated by models
Building is handled by the runtime’s built-in toolchain
Testing, validation, and repair are performed by models repeatedly invoking the runtime
All execution behaviors are defined by the runtime’s semantics
The runtime forms Anthropic’s internal “minimal stable layer”

This model is similar to the relationship between Chrome and V8: the execution engine and upper-layer system co-evolve over time, with performance and semantics advancing in sync.

Whether Bun can fulfill this role depends on Anthropic’s architectural choices, but the event itself has opened up possibilities in this direction.

Industry Trends and Future Evolution

Combining facts, signals, and engineering trends, the following directions can be anticipated:

The “Agent Runtime” category will gradually become more defined
The boundaries between bundler, runtime, and test runner will continue to blur
Cloud vendors will launch controllable runtimes with capability declarations
Permission models and secure sandboxes will move down to the language runtime layer
Runtimes will become part of the model toolchain, rather than an external environment

These trends will not all materialize in the short term, but they represent the inevitable path of engineering evolution.

Summary

The combination of Bun × Anthropic is not about “an open-source project being absorbed,” but about a language runtime being actively integrated into the engineering pipeline of a large model system for the first time. Competition at the model layer will continue, but what truly reshapes software is the structural transformation of AI-native runtimes. This is a foundational change worth long-term attention.

Agentic Runtime Realism: Insights from McKinsey Ark on 2026 Infrastructure Trends

Jimmy Song — Tue, 02 Dec 2025 12:07:45 +0000

Statement

ArkSphere has no affiliation or association with McKinsey Ark.

The value of Agentic Runtime lies not in unified interfaces, but in semantic governance and the transformation of engineering paradigms. Ark is just a reflection of the trend; the future belongs to governable Agentic Workloads.

Recently, the ArkSphere community has been focusing on McKinsey’s open-source Ark (Agentic Runtime for Kubernetes). Although the project is still in technical preview, its architecture and semantic model have already become key indicators for the direction of AI Infra in 2026.

This article analyzes the engineering paradigm and semantic model of Ark, highlighting its industry implications. It avoids repeating the reasons for the failure of unified model APIs and generic infrastructure logic, instead focusing on the unique perspective of the ArkSphere community.

Ark’s Semantic Model and Engineering Paradigm

Ark’s greatest value is in making Agents first-class citizens in Kubernetes, achieving closed-loop tasks through CRD (Custom Resource Definition) and controllers (Reconcilers). This semantic abstraction not only enhances governance capabilities but also aligns closely with the Agentic Runtime strategies of major cloud providers.

Ark’s main resources include:

Agent (inference entity)
Model (model selection and configuration)
Tools (capability plugins/MCP, Model Capability Plugin)
Team (multi-agent collaboration)
Query (task lifecycle)
Evaluation (assessment)

The diagram below illustrates the semantic relationships in Agentic Runtime:

Figure 1: Agentic Runtime Semantic Relationships

Architecture and Community Activity

Ark’s architecture adopts a standard control plane system, emphasizing unified runtime semantics. The community is highly active, engineer-driven, and the codebase is well-structured, though production readiness is still being improved.

Figure 2: Ark Architecture and Control Plane Flow

ArkSphere’s Boundaries and Inspirations

The emergence of Ark has clarified the boundaries of ArkSphere. ArkSphere does not aim for unified model interfaces, multi-cloud abstraction, a collection of miscellaneous tools, or a comprehensive framework layer. Instead, it focuses on:

The semantic system of Agentic Runtime (tasks, states, tool invocation, collaboration graphs, etc.)
Enterprise-grade runtime governance models (permissions, auditing, isolation, multi-tenancy, compliance, cost tracking)
Integration capabilities for domestic ecosystem tools
Engineering paradigms from a runtime perspective

ArkSphere is an ecosystem and engineering system at the runtime level, not a “model abstraction layer” or an “agent development framework.”

Key Changes in 2026

2026 will usher in the era of Agentic Runtime, where Agents are no longer just classes but workloads that require governance rather than mere importation. Ark is just one example of this trend, and the direction is clear:

Semantic models and governability become highlights
Closed-loop tasks are the core value

Summary

Ark’s realism teaches us that the future belongs to runtime, semantics, governability, and workload-level Agents. The industry will no longer pursue unified APIs or framework implementations, but will focus on governable runtime semantics and engineering paradigms.

In-Depth Analysis of Ark: Kubernetes for the AI Era or a New Engineering Paradigm Shift?

Jimmy Song — Tue, 02 Dec 2025 10:54:34 +0000

Statement

ArkSphere has no affiliation or association with McKinsey Ark.

The greatest value of Ark lies in reshaping engineering paradigms, not just its features. It points the way for AI Infra and leaves vast space for community ecosystems.

Recently, many members in our ArkSphere community have started exploring McKinsey’s open-source Ark (Agentic Runtime for Kubernetes).

Some see it as radical, some think it’s just a consulting firm’s experiment, and others quote a realistic maxim:

What we need now is “agentic runtime realism,” not “unified model romanticism.”

I strongly agree with this sentiment.

I’ve spent some time analyzing Ark’s source code, architecture, and design philosophy, combined with our community discussions. My conclusion is:

Ark’s significance is not in its features, but in its paradigm.

It’s not the answer, but it points toward the future of AI Infra.

Below is my interpretation of Ark, focusing on engineering, architecture, trends, and its inspiration for ArkSphere.

What Exactly Is Ark?

Ark’s core positioning is: A runtime that treats Agents as Kubernetes Workloads.

It’s not a framework, not an SDK, not an AutoGen-style multi-agent tool, but a complete system including:

Control plane (Controller)
Custom resource models (CRD, Custom Resource Definition)
API service
Dashboard
CLI
Python SDK

Essentially, Ark is the control plane for Agents.

Ark defines seven core CRDs in Kubernetes. The following flowchart shows the relationships among these resources:

Figure 1: Ark CRD Resource Relationships

Through this set of CRDs, Ark makes Agent systems resource-oriented and declarative, enabling capabilities such as:

Lifecycle management
Multi-tenant isolation
RBAC (Role-Based Access Control)
Observability
Upgradability
Extensibility (tools, models, MCP)

In other words, Ark is not about “how to write Agents,” but “how to operate Agents in enterprise-grade systems.”

Three-Layer Architecture: Mixed Languages and Components, but a Complete System

Ark’s overall architecture is divided into three layers, each with different tech stacks and responsibilities. The following flowchart illustrates the relationships among components in each layer:

Figure 2: Ark Three-Layer Architecture Components

This is not a “wrapper project,” but a fully operational AI Runtime system, with a level of engineering far beyond most agent frameworks on the market.

Is It the Kubernetes of the AI Era?

Let’s revisit Kubernetes’ core value:

Kubernetes was never about “unifying cloud APIs”; it unified the “application runtime model.”

Cloud provider APIs aren’t unified, nor are networking or storage. What is unified: Pod, Deployment, Service—these application models.

Kubernetes succeeded because:

It provides a stable application abstraction on top of diversity.

Ark’s goal is not to unify all large language models (LLMs), MCPs, or tool formats, but rather:

Agent resource model (CRD) + control plane (Reconciler) + lifecycle.

From this perspective, Ark offers a prototype of a “declarative application model” for the AI era.

Whether it will become “Kubernetes for AI” is still too early to say, but it has already planted a seed.

Comparison with Other Frameworks: Not on the Same Level

Current mainstream agent frameworks like LangChain, CrewAI, AutoGen, MetaGPT, etc., address problems fundamentally different from Ark.

The table below compares the positioning and limitations of each framework:

Name	What Problem Does It Solve	Core Limitation
LangChain	Agent/Tool composition	Doesn’t address deployment or governance
AutoGen	Multi-agent conversations	Lacks control plane and lifecycle
CrewAI	Workflow-style multi-agent	Missing scheduling, RBAC, resource model
MetaGPT	Agent SOP	Just execution logic, not a platform
OpenDevin	AI IDE/Dev Assistant	Not an Agent Runtime
Ark	Agent control plane + resource system	Functionality not yet mature

Table 1: Mainstream Agent Frameworks vs. Ark

In short:

Other tools focus on “how to write Agents.”
Ark focuses on “how Agents run, schedule, govern, observe, and extend.”

That’s an architectural difference.

Execution Flow: Agents Scheduled Like Pods

Ark’s execution flow closely resembles the Kubernetes controller model. The following sequence diagram shows the core process:

Figure 3: Ark Agent Execution Flow

You can see Ark’s process logic is transparent, with a clear engineering path, bringing agent systems into a “controllable” state.

Production Readiness: Right Direction, Still a Tech Preview

According to official notes and code maturity, Ark currently offers:

Runnable
Learnable
Extensible
But not recommended for large-scale production use yet

Main reasons include:

CRD structures may change
APIs are not yet stable
MCP ecosystem is still forming
Memory service is still basic
Multi-agent team execution strategies are primitive

But the engineering system is already taking shape, which is crucial.

Community Activity: Small but Elite, Strong McKinsey Drive

From GitHub data:

Stars: 222
Forks: 50
Contributors: 48
Commit frequency is steady
The vast majority of contributions come from within McKinsey

Note: Data as of December 2, 2025.

High stability, but limited openness.

This is also ArkSphere’s opportunity:

The paradigm is right, but the ecosystem needs community-driven growth.

Trends for 2026: From Framework Era to Runtime Era

After deep analysis, I’m increasingly convinced:

2023–2024: Large model API call era
2024–2025: Agent framework era
2025–2027: Agent Runtime / Control Plane era (Ark’s direction)

While everyone is writing Python scripts for agents, the real value lies in:

Multi-agent task scheduling
Tool registration and governance
Session/Memory lifecycle
Result reproducibility
RBAC, auditing, tenant isolation
Observability
Enterprise internal personalized agent systems

Ark is providing a practical path forward.

Inspiration for ArkSphere

Ark’s inspiration for ArkSphere is both critical and direct:

ArkSphere Should Focus on “Paradigm Building,” Not “Feature Stacking”

Ark offers a prototype for future Agentic Runtime:

Resource model
Control plane
Tool registration
Multi-agent collaboration
Evaluation and governance

ArkSphere’s role should be:

Aggregate paradigms, produce standards, incubate ecosystems, not rewrite Ark itself.

This is the “CNCF (Cloud Native Computing Foundation) for the AI-native era.”

Huge Potential for Localization in China

Localization opportunities include but are not limited to:

Integration with domestic large language models (e.g., Qwen, DeepSeek, Zhipu)
Enterprise privatization scenarios
Local tool/MCP discovery ecosystem
Multi-cluster/edge inference
Enterprise-grade RBAC, auditing, data isolation
AgentSpec enhancements for industrial scenarios
Enhanced versions of Runtime/Controller

In other words:

Ark solves the “model,” while ArkSphere can solve the “ecosystem.”

What We Need Is Not “Kubernetes for the LLM Era,” But an “Industry-Grade Cognition System for AI Runtime”

The biggest takeaway from dissecting Ark:

The future of AI-native is not a pile of tools, but an engineering system.

ArkSphere can be the initiator of this system.

Summary

Ark is not a “universal runtime,” nor is it the “ultimate Kubernetes for the AI era.”

But it has done one crucial thing right:

It abstracts all the pain points people faced when writing Python agent scripts into Kubernetes resources and controllers.

It represents engineering, not just a demo.

It’s not mature yet, but it’s heading in the right direction.

It’s not the end, but it gives us a clear roadmap.

For the ArkSphere community I’m running, Ark provides a clear inspiration:

The future belongs to Runtime, to Control Plane, to governable agent systems.

And the ones who can truly scale this system are not McKinsey, but the community.

ArkSphere Community Launch

Jimmy Song — Sun, 30 Nov 2025 16:39:08 +0800

ArkSphere Community is now officially launched, built for developers working on AI Infrastructure, runtime systems, and intelligent agent execution environments. The goal is not discussion for its own sake, but the construction of verifiable, evolvable, and running open-source systems.

Core Focus

AI-native runtime and agent execution layer
GPU, inference serving, and distributed execution architecture
AI infrastructure stack and production-grade engineering patterns
Open-source prototypes and specification-driven development

Join

https://arksphere.dev/community/

Statement

ArkSphere grows from cloud-native experience, but it is not a continuation — it is a forward move. Past work remains accessible, future work is built here.

Participation means contribution: design proposals, architecture validation, OSS mapping, implementation work, and real systems running at scale.

From Using AI to Relying on AI: Why the Era of AI Engineering Has Yet to Begin

Jimmy Song — Sat, 29 Nov 2025 12:40:54 +0000

The real inflection point for AI engineering is not “how many people use it,” but “how many people cannot do without it.” Only when not using AI leads to direct loss of opportunity and efficiency, can we say the era of AI engineering has truly arrived.

Starting Point: Predictions for AI in 2026

Recently, I came across two predictions for 2026 from Amazon CTO Werner Vogels that struck me the most:

Renaissance Developer: Developers must span code, product, business, and social impact.
Personalized Learning: AI will reshape education, focusing on differentiated paths rather than a unified curriculum.

Both point to the same trend: AI is not just a tool, but is redefining how people grow and how they are defined.

There is a gap between prediction and reality, and it is worth exploring.

Correction: Will AI Really Be “Saturated” by 2026?

My initial prediction was that AI usage would reach saturation by 2026. Reality has shown me this is too optimistic.

By the end of 2025, even among internet professionals, most people’s use of AI remains at the “heard of it” or “tried it a few times” stage. It is still far from being a daily workflow necessity.

More importantly, this judgment is conditional: infrastructure supply, regulation, and compute costs must not reverse in the next 3–6 years. If any variable breaks down (costs double, models go offline, policy shifts), the adoption curve will be disrupted.

The Truth About the Inflection Point: From “Using” to “Relying On”

“Relying on” is a vague term. A more precise definition requires measurable indicators.

Here is a diagram that visualizes the metrics for being truly dependent on AI:

This diagram visualizes the quantitative metrics for being truly dependent on AI, comparing target thresholds with current status:

Figure 1: Quantitative Definition of AI Dependency

Most industries have not reached the “cannot operate without” stage, unlike the internet, mobile, or payment inflection points. Most metrics are still far below the threshold, which is why the most likely outcome for 2026 is: more people will use AI, but those who truly rely on it will remain a minority.

Using ≠ Building: The Five-Level Capability Ladder

This difference is not binary, but a clear progression.

The following table shows the five-level model of AI capability maturity.

Level	Name	Description	Scarcity
1	Tool User	ChatGPT/Claude, Coding, Copywriting, Accelerator, Optional	Low
2	Integrator	LLM API + Vector DB, AI layered on existing systems, Usable, not critical	Low
3	Settler	Restructuring data flow, business decisions, AI becomes critical path	Rising
4	Engineering Abstraction	Extracting frameworks, runtimes, providing infra for ecosystem	Extremely High
5	Autonomous System	Self-feedback, self-optimizing, redefining human-AI relationship	Future

Table 1: Five-Level Model of AI Capability Maturity

Currently, the biggest gap is at Level 3 and Level 4. Most people are stuck at Level 1 or 2, with very few reaching Level 4. This means high-value scarcity will not disappear, but will continue to rise.

Why the Era of AI Engineering Has Not Arrived: Three-Dimensional Delaying Factors

It is not technology alone that is holding things back, but constraints in three dimensions.

The following diagram illustrates the three main constraints delaying AI engineering maturity:

This diagram illustrates the three main constraints (technical, institutional, and organizational) that are delaying AI engineering maturity, along with their delay metrics:

Figure 2: Three-Dimensional Constraints on AI Engineering Maturity

The key observation: If any one dimension is stuck, the entire ecosystem’s maturity will be delayed. Currently, none of the three dimensions have fully mature solutions.

The Realistic Window: Three Paths for Capability Advancement

The next three years will not be “winner takes all,” but rather a period where multiple capability levels appreciate simultaneously.

Below is a table comparing the value and bottlenecks of different capability advancement paths:

Capability Path	Short-Term Value	Long-Term Outlook	Bottleneck
Level 1→2 (Tool→Integration)	⭐⭐ Rapid Depreciation	⭐ Saturation	Low barrier, fierce competition
Level 2→3 (Integration→Settlement)	⭐⭐⭐⭐ Scarce	⭐⭐⭐⭐ Continual Appreciation	Requires industry depth, long-term iteration
Level 3→4 (Settlement→Abstraction)	⭐⭐⭐⭐⭐ Extremely Scarce	⭐⭐⭐⭐⭐ Defines Ecosystem	Large cognitive leap, needs community influence

Table 2: AI Capability Advancement Paths and Value Comparison

Key conclusion: While the number of “AI users” is rapidly increasing (depressing Level 1 value), due to the three-dimensional delaying factors, scarcity at Level 3 and 4 will only rise.

What I’m Doing on arksphere.dev

Based on the above judgment, I focus on exploring the architectural evolution of AI Native Infrastructure. The goal is not to catalog model usage, but to study the foundational capability stack supporting scalable intelligent systems: scheduling, storage, inference, Agent Runtime, autonomous control, observability, and reliability.

The content is no longer a collection of courses or tips, but a continuous record of evolution around Infra → Runtime → System Abstraction. arksphere.dev is the site for this experiment and settlement.

Summary

The inflection point for the era of AI engineering is not “how many people use it,” but “how many people cannot do without it.” The latter requires five measurable indicators to reach their thresholds, and we are still far from that.

“Using ≠ Building” is not a binary, but a five-level progression. Scarcity at Level 3 and 4 will rise as the number of Level 1 users increases—this is the biggest opportunity window in the next three years.

But the width of this window depends largely on how technology, institutions, and organizations evolve together. I hope more people working on AI engineering will not only focus on technical innovation, but also invest equal thought into institutional development, talent growth, and risk governance—these “invisible engineering” challenges.

Antigravity VS Code Setup Guide: Build a Practical AI IDE Workflow

Jimmy Song — Thu, 20 Nov 2025 03:55:30 +0000

The biggest pain point when switching IDEs is user habits. By installing a series of plugins and tweaking configurations, you can make Antigravity feel much more like VS Code—preserving familiar workflows while adding Open Agent Manager capabilities.

If you searched for a practical Antigravity VS Code setup, this walkthrough is optimized for that exact use case. The goal is not to replicate VS Code pixel by pixel, but to restore a familiar extension marketplace, keep your daily coding ergonomics, and still use Antigravity’s stronger agent-style execution. I focus on the concrete setup steps that materially change productivity: marketplace migration, AMP and CodeX installation, editor behavior alignment, and the trade-offs versus GitHub Copilot in real daily work.

Figure 1: Antigravity IDE UI

Continue Reading

Below are the configurations and steps I actually use. Feel free to follow along.

First Impressions of Antigravity

A few subjective observations:

The interface is split between agent management and editor views, somewhat like AgentHQ + VS Code.
Agents modify code very quickly, with a much higher completion rate than typical “chat-based” assistants.
The editor and context windows are large, ideal for long diffs and logs.
By default, it uses OpenVSX / OpenVSCode Gallery, so the extension ecosystem isn’t identical to my VS Code setup.

All subsequent steps focus on one goal: keep Antigravity’s agent features while maintaining my VS Code workflow.

Switching the Extension Marketplace to VS Code Official

Antigravity is essentially a VS Code fork, so you can directly change the Marketplace configuration.

In Antigravity:

Go to Settings -> Antigravity Settings -> Editor, and update the following URLs to point to VS Code:

Marketplace Item URL:

https://marketplace.visualstudio.com/items

Marketplace Gallery URL:

https://marketplace.visualstudio.com/_apis/public/gallery

Figure 2: VSCode Marketplace Configuration

Restart Antigravity.

After this change, searching and installing extensions works just like the official VS Code Marketplace. Installing AMP, GitHub Theme, VS Code Icon, etc., all follow this process.

Installing the AMP Extension

AMP isn’t officially supported on Antigravity yet, but you can install it directly via the VS Code Marketplace.

Steps:

Open the Extensions panel (the same icon as in VS Code).
Search for the AMP extension and install it as usual.
Log in using your AMP API Key.

Currently, Antigravity doesn’t support one-click account login like VS Code; you have to use the API key.

Summary

Once installed, AMP works almost identically in Antigravity as in VS Code—completion and refactoring features are available. The only difference is manual login configuration.

I recommend AMP because it offers a free mode. In my experience, it’s great for writing documentation, running scripts, and as a daily command-line tool. It’s fast, and especially useful for optimizing prompts.

Importing the CodeX Extension

CodeX doesn’t provide a direct VSIX download link on the web. My approach is to export it from VS Code and then import it into Antigravity.

Figure 3: Exporting Codex Extension in VS Code

Steps:

Install the CodeX extension in VS Code (if you haven’t already).
In VS Code’s extension manager, find CodeX and export it as a .vsix file.
Switch to Antigravity, open the Extensions panel, and select “Install from VSIX”.
Choose the exported codex-x.x.x.vsix file to complete installation.

Tip

Since my local VS Code is already logged into CodeX, importing it into Antigravity automatically reuses the login state—I didn’t need to log in again.

Optimizing Editor Settings

Beyond the marketplace and plugins, a few tweaks make the experience even closer to VS Code:

Theme: Choose the same color scheme as VS Code to minimize visual switching. I use GitHub Theme and vscode-icons.
Editor Settings: In “Open Editor Settings”, set indentation, formatting, line width, etc., to match your VS Code preferences. I define these in the workspace’s settings.json, so no migration is needed.

After these changes, the editing area is essentially “VS Code with an agent console”.

Remaining Issues

To fully migrate from VS Code/GitHub Copilot to Antigravity, I think there are still several key challenges:

Limited Customization: Antigravity can’t support custom prompts and agents like Copilot Chat. Currently, only “rules” configuration is available, which limits workflow flexibility.
Model Ecosystem Needs Improvement: Antigravity hasn’t natively integrated the latest models from major vendors (OpenAI, Anthropic, Microsoft, xAI, etc.), whereas GitHub Copilot excels here.
Cost Considerations:
- Future pricing may start at $20/month.
- No free models are supported, unlike GitHub Copilot (even Copilot Pro users have free model options).
Stability Issues: Agents often encounter “Agent terminated due to error” during operation, requiring manual retries or new sessions. This affects workflow smoothness, though I expect improvements in the future.

GitHub Copilot VS. Antigravity

Although Antigravity excels in several areas, there is still significant room for improvement compared to the combination of GitHub Copilot and VS Code.

The large language models (LLMs, Large Language Models) I frequently use are all supported in VS Code:

Figure 4: Copilot-supported LLMs (partial)

My long-accumulated custom prompts:

Figure 5: Copilot Chat enables quick access to custom prompts

My collection of agents:

Figure 6: Copilot Chat allows selection of custom agents

Here are some personal experiences using VS Code and Copilot that, for now, are hard to replace with other IDEs:

The Ask/Edit/Agent/Plan workflow perfectly fits my working habits.
Support for custom prompts and agents is essential. Many of my prompts and agents have been refined over time and are deeply integrated into my daily workflow—it’s hard to find alternatives elsewhere.
New models are integrated at lightning speed. Whenever a new model is released, GitHub Copilot is among the first to support it.
The integration with VS Code is seamless—no extra configuration required, making it extremely convenient.
Frequent updates: just a few days ago, a bug I reported to VS Code was fixed the same night.
Copilot Chat’s keyboard shortcuts make it easy to quickly access various features.
GitHub has granted me a free Pro account. Although the monthly premium quota is only 300 calls, combining Copilot with other plugins like AMP, Codex, Droid, and Qwen enables a highly efficient workflow. Even if I upgrade to a paid account in the future, the $10/month fee is very cost-effective compared to similar products.

Practical Experience

A few subjective tips from my actual usage—take them as reference:

Don’t treat Antigravity as “VS Code + chat box”. Use its agent features for complete tasks: let the agent propose a plan, then execute changes.
For major changes, always create a new Git branch and restrict agent actions to that branch. Handle all diffs via standard Pull Request (PR) workflows.
Ask agents to produce “artifacts” (plans, proposals, test descriptions), not just final code. This makes it easier to review and track changes.
Plugins you’re already comfortable with in VS Code (like AMP, CodeX) can be migrated directly, reducing cognitive load and letting you focus on new agent workflows.

Summary

My current experience: Antigravity delivers powerful agent capabilities and multi-view consoles. By following these steps to align the interface and plugin ecosystem with VS Code, you can smoothly transition your daily development workflow.

Cloudflare November 18 Global Outage: The Dangers of Implicit Assumptions in Modern Infrastructure

Jimmy Song — Wed, 19 Nov 2025 18:56:34 +0800

The greatest risks to modern internet infrastructure often aren’t in the code itself, but in those implicit assumptions and automated configuration pipelines that go undefined. Cloudflare’s outage is a wake-up call every Infra/AI engineer must heed.

Yesterday (November 18), Cloudflare experienced its largest global outage since 2019. As this site is hosted on Cloudflare, it was also affected—one of the rare times in eight years that the site was inaccessible due to an outage (the last time was a GitHub Pages failure, which happened the year Microsoft acquired GitHub).

Figure 1: jimmysong.io was down for 27 minutes due to the Cloudflare outage

This incident was not caused by an attack or a traditional software bug, but by a seemingly “safe” permissions update that triggered the weakest link in modern infrastructure: implicit assumptions (Implicit Assumption) and automated configuration pipelines (Automated Configuration Pipeline). Cloudflare has published a blog post Cloudflare outage on November 18, 2025 explaining the cause.

Here is the chain reaction process of the outage:

A permissions adjustment led to metadata changes;
The metadata change doubled the lines in the feature file;
The doubled lines triggered the proxy module’s memory limit;
The memory limit caused the core proxy to panic;
The proxy panic led to a cascade failure in downstream systems.

This kind of chain reaction is the most typical—and dangerous—systemic failure mode at today’s internet scale.

Root Cause: Implicit Assumptions Are Not Contracts

Let’s first look at the core hidden risk in this incident. The Bot Management feature file is automatically generated every five minutes, relying on a default premise:

The system.columns query result contains only the default database.

This assumption was not documented or validated in configuration—it existed only in the engineer’s mental model.

After a ClickHouse permissions update, the underlying r0 tables were exposed, instantly doubling the query results. The file size exceeded the FL2 preset of 200 features in memory, ultimately causing a panic.

Once an implicit assumption is broken, the system lacks a buffer and is highly prone to cascading failures.

Configuration Pipelines Are Riskier Than Code Pipelines

This incident was not caused by code changes, but by data-plane changes:

SQL query behavior changed;
Feature files were automatically generated;
The files were broadcast across the network.

A typical phenomenon in modern infrastructure: data, schema, and metadata are far more likely to destabilize systems than code.

Cloudflare’s feature file is a “supply chain input,” not a regular configuration. Anything entering the automated broadcast path is equivalent to a system-level command.

Language Safety Can’t Eliminate Boundary Layer Complexity

A former Cloudflare engineer summarized it well:

Rust can prevent a class of errors, but the complexity of boundary layers, data contracts, and configuration pipelines does not disappear.

The FL2 panic stemmed from a single unwrap(). This isn’t a language issue, but a lack of system contracts:

No upper-bound validation for feature count;
File schema lacked version constraints;
Feature generation logic depended on implicit behavior;
Core proxy error mode was panic, not graceful degradation.

Most incidents in modern distributed systems (Distributed System) come from “bad input,” not “bad memory.”

Core Proxies Need Controllable Failure Paths

FL/FL2 are Cloudflare’s core proxies; all requests must pass through them. Such components should not fail with a panic, but have the following capabilities:

Ignore abnormal features;
Truncate over-limit fields;
Roll back to previous versions;
Fail-open or fail-close;
Skip the Bot module and continue processing traffic.

As long as the proxy “stays alive,” the entire network won’t be completely paralyzed.

Data Changes Are More Uncontrollable Than Code Changes

The essence of this incident:

Subtle permission changes;
ClickHouse default behavior changed;
Query results propagated to distributed systems;
Automated publishing amplified the error;
Edge proxies crashed due to uncontrolled input.

Future AI Infra (AI Infrastructure) will be even more complex: models, tokenizers, adapters, RAG indexes, and KV snapshots all require frequent updates.

In future AI infrastructure, data-plane risks will far exceed those of the code-plane.

Recovery Process Shows Engineering Maturity

During the incident, Cloudflare took several measures:

Stopped generating erroneous feature files;
Force-distributed the previous version of the file;
Rolled back Bot module configuration;
Ran Workers KV and Access outside the core proxy;
Restored traffic in stages.

Restoring hundreds of PoPs worldwide simultaneously demonstrates a high level of engineering maturity.

Lessons for Infra/AI/Cloud Native Engineers

The Cloudflare event highlights four common risks in large-scale systems:

Implicit assumptions fail;
Configuration supply chain contamination;
Automated publishing amplifies errors;
Core proxies lack graceful degradation paths.

For AI Infra practitioners, these risks are even more relevant:

Model weight updates without schema validation;
Adapter merges may be contaminated;
RAG index incremental builds are unstable;
Inference graph configuration may be broken by bad data;
Automatically rolled-out models may propagate errors network-wide.

AI engineering is replaying Cloudflare’s infrastructure dilemmas—just at greater speed and scale.

Summary of Former Cloudflare Engineer’s Views

His insights pinpoint the hardest problems in distributed systems:

The issue isn’t code, but missing contracts;
Not the language, but undefined input boundaries;
Not modules, but lack of validation in the configuration supply chain;
Not bugs, but absence of fail-safe mechanisms.

This incident proves: The real fragility in modern infrastructure lies in “behavioral boundaries,” not “memory boundaries.”

Summary

The Cloudflare November 18 outage was not a coincidence, but an inevitable result of modern internet infrastructure evolving to large-scale, highly automated stages.

Key takeaways from this event:

System assumptions must be made explicit;
Configuration pipelines must be validated;
Automated publishing needs “dead-end” mechanisms;
Core proxies must be designed with controllable failure paths;
Data-plane contracts must be stricter than code-plane contracts.

In the AI-native Infra era, these requirements will only become more stringent.

References

The Second Half of Cloud Native: The Era of AI Native Platform Engineering Has Arrived

Jimmy Song — Mon, 17 Nov 2025 11:07:40 +0000

The second half of cloud native isn’t about being replaced by AI, but being rewritten by it. The future of platform engineering will revolve around models and agents, reshaping the tech stack and developer experience.

Since I first encountered Docker and Kubernetes in 2015, I’ve followed the cloud native journey: from writing Deployments in YAML, to exploring Service Mesh and observability, and in recent years, focusing on AI Infra and AI-Native Platforms. Looking back from 2025, the years 2015–2025 can be seen as the “first half” of cloud native. Marked by KubeCon / CloudNativeCon NA 2025, the industry is collectively entering the “second half”: the era of AI-Native Platform engineering.

This article reviews the past decade of cloud native, and, combined with KubeCon NA 2025, outlines key turning points and the technical coordinates for the next ten years.

2015–2025: The “First Half” of Cloud Native

Over the past decade, cloud native technology themes have evolved through three main stages. The following flowchart illustrates the progression.

The flowchart below illustrates the progression of cloud native technology themes over the past decade:

Figure 1: Cloud Native Decade Technology Evolution Flow

The first stage focused on containerization and orchestration standardization.

Docker realized the engineering dream of “build once, run anywhere”
Kubernetes won the orchestration wars and became the de facto standard
CNCF was founded, with Prometheus, Envoy, and other projects joining
Enterprises focused on migrating applications to Kubernetes

Typical tasks during this phase involved moving Java services from VMs to containers and K8s, emphasizing understanding of Deployment, Service, and Ingress.

The second stage, 2018–2020, saw complexity shift from “deployment” to “communication” and “operations”.

Service Mesh (Istio / Linkerd / Consul) addressed east-west traffic management
The observability trio (Logs / Metrics / Traces) became default configurations
Multi-cluster and multi-region practices matured
Enterprises focused on managing large microservice systems

During this period, I spent significant time researching Istio, service mesh, and traffic management, and authored Kubernetes and Istio books. The focus shifted to system stability, observability, and reliability.

The third stage, 2021–2025, is defined by Platform Engineering and GitOps.

As microservices and tools proliferated, platform complexity began to overwhelm developers, making Platform Engineering a key industry term.

GitOps (Argo CD / Flux) drove declarative delivery processes
Internal Developer Platforms (IDP) became priorities for large enterprises
“Platform as a product” philosophy spread
FinOps, cost management, and compliance auditing became platform concerns
DevOps evolved from “tool practice” to “organizational + platform capability”

My takeaway: simply giving developers a pile of tools isn’t enough. End-to-end delivery paths and stable abstraction layers are needed so developers can focus on business, not tool integration.

The table below summarizes the main features of each “first half” stage.

Stage	Core Challenge	Key Tech Stack	Typical Issues
2015–2017 Orchestration	Migrating from VM to containers	Docker, Kubernetes, CNI	Reliable deployment, rolling upgrades
2018–2020 Mesh	Microservice scale, complex communication & observability	Istio/Linkerd, Prometheus, Jaeger	Troubleshooting, fragmented observability
2021–2025 Platform	Tool sprawl, declining developer experience	GitOps, IDP, FinOps, Policy-as-Code	Developer fatigue, platform team overload

Table 1: Cloud Native First Half Stage Features

KubeCon NA 2025: Signals of Cloud Native’s “Second Half”

The main theme of KubeCon 2025 is no longer “how to use Kubernetes well,” but how to reconstruct Kubernetes and the cloud native ecosystem into AI-Native Platforms for the AI era.

Key signals from KubeCon NA 2025 include:

CNCF released the Certified Kubernetes AI Conformance Program
Dynamic Resource Allocation (DRA) entered mainstream discussions
Model Runtime / Agent Runtime projects became conference hotspots
Vendors focused on AI SRE, AI-assisted development, AI security, and supply chain governance
Speakers like Alex Zenla openly stated that Kubernetes’ underlying structure needs rethinking

Together, these mark a clear dividing line: cloud native has officially entered its “second half.”

First Half vs Second Half: Shifting the Cloud Native Narrative

If 2015–2025 is the “first half,” then 2025–2035 is likely the “second half.” The table below compares their core differences.

It highlights changes in platform objects, goals, abstraction layers, and more.

Dimension	First Half (2015–2025)	Second Half (2025–2035, AI Native)
Core Objects	Containers, Pods, Microservices	Models, inference tasks, Agents, data pipelines
Platform Goals	Stable application delivery	Efficient, continuous AI workload & agent orchestration
Abstraction Layers	Deployment / Service / Ingress / Job	Model / Endpoint / Graph / Policy / Agent
Resource Scheduling	CPU / Memory / Node	GPU / TPU / ASIC / KV Cache / Bandwidth / Power
Engineering Focus	DevOps / GitOps / Platform Engineering 1.0	AI Native Platform Engineering / AI SRE
Security & Compliance	Image security, CVE, supply chain SBOM	Model security, data security, AI supply chain & “hallucination dependencies”
Runtime Forms	Container + VM + Serverless	Container + WASM + Nix + Agent Runtime

Table 2: Core Differences: First vs Second Half of Cloud Native

From a developer’s perspective, the most direct change is: future platforms will no longer treat “services” as first-class citizens, but will center on “models + agents.”

Example: Technical Layers of an AI Native Platform

To clarify the structure of an AI-Native Platform, the following layered diagram shows the relationships between technical levels.

The layered diagram below shows the relationships between different technical levels in an AI-Native Platform:

Figure 2: AI Native Platform Layering Diagram

Historically, cloud native focused on L0 + L2 (Kubernetes + platform engineering), but in the AI Native era, L1 (Model Runtime, Agent Runtime, heterogeneous resource scheduling) becomes the new battleground.

Key Change 1: From “Container-Centric” to “Model-Centric”

In the first half, cloud native’s main object was the application process, with containers as packaging. The second half requires handling:

Model version management and canary releases
Balancing inference performance, latency, and cost
Multi-model composition, routing, A/B testing
Relationships between models, data, features, and vector indexes

At KubeCon NA 2025, CNCF’s AI Conformance Program aims to standardize model workloads, managing them like Deployments. Platform engineering will gain new abstractions—not just “deploying services,” but “deploying model capabilities.”

Key Change 2: DRA and the Golden Window for Heterogeneous Resource Scheduling

Previously, writing a Deployment meant focusing on CPU and memory. Now, GPU inference, training, and Agent Runtime scenarios demand more than static quotas.

Dynamic Resource Allocation (DRA) brings:

Pluggable resource types (GPU/TPU/FPGA/ASIC)
Topology-aware, NUMA, and memory fragmentation scheduling
Binding inference requests to compute allocation for fine-grained QoS
Cost optimization and power control in scheduling decisions

This is the most significant “resource perspective” upgrade since Kubernetes’ inception. The scheduler is no longer just a cluster component, but the AI platform’s policy engine.

Key Change 3: Agent Runtime as the New Generation of Runtime

KubeCon showcased several representative projects:

Edera: Minimal, verifiable runtime redesign
Flox: Nix-based “uncontained” runtime environment
Golem: WASM-based large-scale agent orchestration

The consensus: AI agents aren’t suited to traditional container runtime models. Agents have these traits:

Strong statefulness: context, memory, sessions
High concurrency but fine granularity: massive lightweight tasks
Extremely sensitive to latency and cold starts
Need to resume after failure

Next-gen runtimes focus on reliably executing, managing state, and auditing “hundreds of thousands of agents,” not just “spinning up more Pods.”

Key Change 4: AI SRE and AI Security

At KubeCon NA 2025, security and operations topics were amplified by AI:

Software supply chain attacks and CVEs continue to rise
LLM-assisted coding introduces “hallucination dependencies” and “vibecoded vulnerabilities”
AI-driven artifact scanning, dependency auditing, and license analysis
“AI SRE” is now a formal product category

Traditional cloud native already emphasized security and SRE, but now must address model weights, datasets, vector stores, and agent workflows. AI-Native Platform engineering must answer:

Are code and dependencies secure?
Are models and data trustworthy?
Are agent behaviors controllable?

This will drive deep integration of Policy-as-Code, MCP, graph permission systems, and AI.

Key Change 5: Open Source Participation Becomes a Baseline

In interviews, platform engineering leaders noted:

Hiring increasingly values upstream contributions to Kubernetes and related projects
Open source involvement shortens ramp-up time
New AI Native projects (Model Runtime, Agent Runtime, Scheduler) are also open source

For career growth, contributing to AI Native open source projects will become a basic requirement for platform engineering and AI Infra roles—not just a resume bonus.

The Contours of Cloud Native’s “Second Half”

The table below summarizes the technical focus and essential differences of the “second half.”

It highlights the key coordinates of AI-Native Platform engineering.

Direction	Technical Focus	Essential Difference from First Half
AI Native Platform	Models/Agents as first-class citizens, unified abstraction & governance	Objects shift from services to models & inference
Resource Scheduling	DRA, heterogeneous compute, topology awareness, power & cost	From static quotas to dynamic, policy-driven
Runtime	Container + WASM + Nix + Agent Runtime	From “process containerization” to “execution graph containerization”
Platform Engineering	IDP + AI SRE + Security + Cost + Compliance	From toolset to “autonomous platform”
Security & Supply Chain	LLM dependencies, model weights, datasets, vector store governance	Protection expands from images to “all AI engineering assets”
Open Source & Ecosystem	AI Infra / Model Runtime / Agent Runtime upstream collaboration	Not just “using open source,” but “building the future in open source”

Table 3: Cloud Native Second Half Technical Coordinates

Summary

Over the past decade, cloud native evolved from container orchestration to platform engineering 1.0. With KubeCon NA 2025 as a milestone, the industry systematically brings AI into cloud native technology and organizational stacks:

Kubernetes is no longer just “infrastructure for microservices,” but “runtime for AI workloads”
Platform Engineering is no longer just “tool integration,” but “autonomous platforms for models and agents”
Security, SRE, runtime, scheduling, and networking will all be reimagined under AI

For me, the past ten years were about “making applications more stable in the cloud native world.” The next ten will focus on “making AI better, safer, and more controllable in the cloud native world.” This is, in my view, the opening whistle for cloud native’s “second half.”

NotebookLM: My Most Recommended AI Tool for Learning and Knowledge Organization

Jimmy Song — Mon, 17 Nov 2025 08:44:45 +0000

NotebookLM is the most tailored AI tool I’ve used for knowledge workers. It truly helps me structure massive information and dramatically boosts my learning and content creation efficiency.

As a lifelong learner who reads technical specs and researches open-source projects, I’ve always sought a tool that can “shortcut” my way through mountains of material, reduce mechanical reading, and help me quickly build a global understanding. NotebookLM has been the smoothest and most reliable experience for me over the past year.

It’s not a traditional “chat-style AI tool”—it’s more like an AI-native learning and content organization system that ingests your materials, organizes them, and presents them in various structured formats. The more I use it, the more I realize its help in learning new technologies, understanding unfamiliar fields, organizing large project documents, and building teaching materials—things that general large language models (LLM, Large Language Model) simply can’t match.

The Core Value NotebookLM Brings Me

NotebookLM has significantly improved my workflow, especially in learning new technologies, organizing documents, and content creation.

Quickly Understanding New Technologies: Feed in Complex Materials, Get a “Learnable Version”

My most frequent and indispensable scenario is learning a completely unfamiliar technology or development framework. Faced with dozens or even hundreds of pages of documentation, my typical approach is:

Add official docs, README files, design documents, and architecture diagrams into a single Notebook
Let NotebookLM generate:
- Study guides
- Briefings
- Key knowledge points
- FAQs
- Quizzes
Ultimately, I get a clearly structured “learning entry point” instead of a flood of raw materials.

The following flowchart illustrates how NotebookLM compresses complex documents into a learnable structure:

Figure 1: NotebookLM Document Structuring Flow

In the end, what I gain is an “organized knowledge system” rather than a pile of PDFs waiting to be consumed.

Generating MindMaps: Instantly Turning Large Documents into Structured Knowledge Graphs

I rely heavily on MindMaps to build the “skeleton of knowledge.” NotebookLM’s MindMap feature stands out for:

Automatically identifying relationships between topics
Interactive node expansion and collapse
Integrating multiple source documents

Although it currently only exports PNG, the logical structure itself is already an excellent “knowledge compression.”

The table below compares the auto-generation and visualization capabilities of different tools:

Tool	Auto-Generation	Multi-Doc Integration	Visualization Quality	Export Formats
NotebookLM	Strong	Strong	Good	PNG only (SVG not yet supported)
Common LLM Tools	Weak	Weak	Poor	Depends on tool
MindMap Software (Manual)	None	None	Strong	Fully supported

Table 1: Comparison of MindMap Capabilities in Mainstream Tools

NotebookLM’s greatest advantage is automation.

Generating Teaching Outlines, Training Scripts, and Book Structures: Truly Saving Me Time

NotebookLM is more than just “summarization”—it can generate formal teaching structures based on my prompts. By feeding in project docs, API references, architecture designs, case studies, videos, and blogs, and prompting it to generate:

Teaching outlines
Project training manuals
Course structures
Book chapter frameworks
Slide text
Training case descriptions

For anyone who needs to create content, conduct training, or give presentations, this feature is a huge time-saver.

Below is a typical prompt I actually use:

Based on the provided content excerpts, write a detailed training manual that systematically explains the core principles covered. The manual should use a professional and instructional tone, breaking down complex concepts into actionable steps and lessons. Ensure all content is strictly based on the source material and covers every aspect mentioned.

The training manual should include:
1. Training objectives and expected outcomes
2. Training content and structure
3. Training methods and tools
4. Training evaluation and feedback
5. Training summary and follow-up actions
6. Training cases and examples
7. Training resources and references

The results are often surprisingly good.

Multi-Format Input Capability: The Most Stable I’ve Seen

NotebookLM supports direct ingestion of various material types, with extremely stable parsing. The table below summarizes my actual experience:

Input Type	My Actual Experience
PDF	Most stable, clear structure parsing
Google Docs	Syncs instantly, very smooth
Word / PPT	Recognized normally
YouTube Video	Auto-summary + key content extraction, very useful
Website URL	Depends on site structure, high success rate
Plain Text	No issues
Images	Partial success, sufficient for screenshots

Table 2: NotebookLM Multi-Format Input Experience

By contrast, other tools often have format parsing issues, garbled text, missing content, or skipped paragraphs. NotebookLM is especially stable in “multi-format ingestion.”

My Most Common NotebookLM Workflow

The following flowchart shows my daily workflow with NotebookLM:

Figure 2: NotebookLM Daily Workflow

Essentially: let AI help me grasp the big picture → then dive deeper → then output content.

My Suggestions and Minor Regrets

NotebookLM is already excellent, but I still have some strong expectations for future improvements:

MindMap Export Formats Should Support SVG or Text-Based (Markmap)

Currently, only PNG is supported, which gets blurry when enlarged. The table below lists my expectations for future features:

Expected Feature	Purpose
SVG Export	For writing books, making slides, scalable without loss
Markmap Output	Most friendly for Markdown writers
Raw JSON	Allows custom rendering

Table 3: Expected MindMap Export Formats

I’m especially looking forward to NotebookLM supporting Markmap format export, which would be extremely friendly for users who write blogs and docs in Markdown.

Recently, Google also launched CodeWiki, similar to DeepWiki, which auto-generates image-rich Wikis for GitHub projects, but currently does not support Mermaid or Markmap.

Conversation History Should Support Long-Term Saving

Currently:

Chats are not persistently saved
Only manually “add to notes” preserves results

This causes some knowledge context to be lost. I hope to see a “Notebook conversation history” feature in the future.

Slide Generation Should Support Templates for Content Creators

Currently, Video Overview offers various visual styles, but cannot:

Upload custom PPT templates
Apply enterprise/personal branding templates

If PPT template support is added, NotebookLM could become the “video generation hub” for content creators.

Deep Research Should Launch Soon and Be Fully Open

I’m especially looking forward to this feature, as it could upgrade NotebookLM from a “knowledge organization tool” to a “research-grade tool.” I hope it will:

Reliably crawl more public web pages
Ensure citation quality
Integrate with existing Notebook materials

This is a major upgrade I personally care about.

Mobile Experience Should Be Enhanced Beyond Content Playback

Currently, the mobile experience is minimal, only allowing:

Listening to audio
Viewing Notebook Guide summaries
Simple Q&A

I hope mobile will soon support:

Editing Notebooks
Deep conversations
MindMap interaction
Content output (generating docs, outlines, etc.)

Summary

NotebookLM is truly one of the AI tools I use every single day because it achieves a critical goal:

Organizing information, structuring knowledge, so I don’t have to start from scratch with massive documents.

Whether it’s:

Learning new technologies
Reading long documents
Creating courses
Conducting training
Writing books
Drafting speeches
Summarizing content

It saves me a huge amount of time upfront, letting me focus on “understanding” and “creating.”

I’ll continue to use NotebookLM as one of my essential tools and keep an eye on its progress in Deep Research, template systems, and mobile features.

This is a tool truly designed for “knowledge workers” and deserves to be known by more people.

Helm v4: Paradigm Convergence and Plugin System Rebuild

Jimmy Song — Fri, 14 Nov 2025 11:18:30 +0000

The release of Helm 4 is not just a technical upgrade, but a deep convergence of cloud-native delivery paradigms. The rebuilt plugin system and supply chain governance capabilities make Helm once again a driving force in the Kubernetes ecosystem.

Since its first release in 2016, Helm has been one of the most important application distribution tools in the Kubernetes ecosystem. Helm v4 is not a “minor enhancement,” but a comprehensive update around delivery methods, extension mechanisms, and supply chain approaches.

This article reconstructs Helm’s historical context and focuses on why Helm 4 represents a paradigm-converging release.

Helm: From Tiller to Declarative Delivery

Below is a textual timeline showing key milestones from Helm v2 to v4, helping you understand its technical evolution:

2016: Helm v2 released, using the Tiller architecture.
2017: Chart Hub expands, major projects begin providing official Charts.
2018: Security model controversies intensify, Tiller’s permission issues become apparent.
2019: Helm v3 released, Tiller removed, OCI support introduced.
2021: GitOps becomes widespread, Server-Side Apply (SSA) becomes the mainstream delivery semantic.
2023: kstatus widely adopted for controller status assessment and health calculation.
2025: Helm v4 released, bringing SSA, WASM plugins, reproducible builds, and content hash caching.

Each major Helm release closely follows Kubernetes paradigms, driving progress in declarative delivery and ecosystem tooling.

Fundamental Changes in Helm v4

This section analyzes the core technical upgrades and paradigm shifts in Helm v4.

Delivery Paradigm Update: Default Server-Side Apply (SSA, Server-Side Apply)

In Helm v3 and earlier, Helm used a “three-way merge” model for resource delivery. Helm v4 fully switches to Server-Side Apply (SSA, Server-Side Apply), meaning the API Server determines field ownership.

This shift brings several direct results:

Full semantic alignment with kubectl apply and GitOps controllers (such as Argo, Flux)
When multiple controllers manage the same object, silent overrides are avoided and conflicts are explainable
Helm’s behavior now follows Kubernetes’ officially recommended declarative delivery paradigm

The following flowchart compares the delivery semantics of Helm v3 and v4.

Figure 1: Helm v3/v4 Delivery Semantics Comparison

Helm is now aligned with the delivery semantics of modern Kubernetes versions, improving predictability and safety in resource management.

kstatus-Driven Wait Behavior and Readiness Annotations

In Helm 3, --wait could only make fuzzy status judgments on limited resources, lacking extensibility and explainability.

Helm 4 introduces kstatus (Kubernetes Status) as the basis for health status parsing, and supports two key annotations:

helm.sh/readiness-success
helm.sh/readiness-failure

Chart authors can precisely define conditions for installation success or failure. Helm’s waiting model now offers “explainability + extensibility,” upgrading from a “templating tool” to a true “deployment orchestrator.”

Extension System Rebuild: WASM Plugin System

Helm 4 thoroughly reconstructs the plugin model, mainly including:

Typed and Structured Plugins

Arbitrary scripts are no longer allowed; plugins must follow typed and structured standards

WebAssembly Plugin Runtime (Extism)

More secure (sandbox isolation)
Cross-language support
Easy unified management in CI/CD and enterprise platforms
Predictable and testable

Post-renderer Integrated into Plugin System

Moves beyond the “external executable black box” era
Helm becomes a programmable platform, not just a template renderer

Engineering Capabilities Upgrade: Reproducible Builds, Content Hash Caching, chart API v3

Helm v4 brings the following engineering improvements:

Chart packaging is reproducible (supports signing, SBOM, SLSA, etc. for supply chain governance)
Local cache uses content hashes, avoiding version-based conflicts
chart API v3 (experimental) is stricter and more flexible
SDK logging system upgraded to Go slog (modern logging)

These capabilities enable Helm charts to enter serious software supply chain governance.

Feature Comparison (Helm v3 → v4)

The table below compares core features between Helm v3 and v4 for a quick understanding of the upgrade value.

Area	Helm 3	Helm 4
Apply Model	Three-way merge	Default SSA
Wait Behavior	Fuzzy, not extensible	kstatus + annotation
Plugin System	Script, uncontrollable	WASM, typed plugins
Post-renderer	External executable	Plugin subsystem
Build	Not reproducible	Reproducible build
Cache	name/version	Content hash
Chart API	v2	v2 + v3 (experimental)
SDK Logs	stdlib log	slog

Table 1: Helm v3 vs v4 Feature Comparison

This is a release that “repays technical debt in bulk + aligns with contemporary Kubernetes semantics.”

Why Is Helm v4 a Paradigm Convergence Event?

The release of Helm v4 is not just a feature upgrade, but a deep convergence of delivery paradigms, mainly in three aspects:

Kubernetes Delivery Semantics Unified to SSA

Previously: kubectl, GitOps, and Helm each had their own logic. Now: All unified to SSA, consistent delivery behavior, smoother ecosystem collaboration.

Plugin System Enters the Platform Era

WASM (WebAssembly) brings a secure, universal, and controllable plugin runtime. Infrastructure projects widely adopt WASM: Envoy → WASM Filters, Kubernetes → WASM CRI/OCI, and now Helm joins the platform camp.

Charts Enter Supply Chain Governance

Reproducible builds and digest verification allow Helm charts to be managed as seriously as container images, greatly enhancing supply chain security.

The entire ecosystem moves to a unified capability baseline, driving cloud-native delivery standardization.

My Helm History and Observations

As an early user from the Helm v2 era, I have experienced the following stages:

Tiller security controversies
v3 migration (state stored in secrets)
Large-scale chart consolidation in the community
OCI adoption
Today’s SSA / WASM / reproducible build

Each major Helm version upgrade is not about chasing trends, but proactively aligning with Kubernetes paradigms:

v3 aligns with K8s “no cluster-side runtime” principle
v4 aligns with SSA, kstatus, WASM, OCI, and other advances from the past five years

Helm exemplifies the evolution rhythm of infrastructure projects: not by piling on features, but by evolving in semantic alignment with the platform.

Summary

The release of Helm v4 marks a new paradigm for Kubernetes application delivery. SSA, WASM plugins, kstatus, and reproducible builds make Helm not just a templating tool, but a core for supply chain governance and platform extensibility. For cloud-native developers and platform teams, Helm v4 is a paradigm upgrade worth attention.

References

Kimi K2 Thinking: The True Awakening of China's Thinking Model

Jimmy Song — Fri, 14 Nov 2025 08:25:26 +0000

China’s large language models have finally moved from “writing like humans” to “thinking like humans.” The open-sourcing of Kimi K2 is a watershed moment for China’s AI trajectory.

The narrative around China’s large language models is shifting from “Chat-style models” to “Thinking models (Thinking Model, Thinking Model).”

Moonshot AI’s open-sourcing of Kimi K2 Thinking marks the first real landing of this transition. K2 is not just another iteration like ChatGLM or Qwen; it’s the first time a Chinese team has unified “deep reasoning + long context + tool invocation continuity” in training. This is the core of the thinking model approach and the reason why models like Claude and Gemini have led the field.

The Significance of K2’s Open Source: China Enters the Era of Thinking Models

Why is K2’s open source a turning point? Because it enables Chinese models to achieve the following capabilities for the first time:

Stable execution of 200–300 tool invocations (toolchain reasoning stability)
Deep, multi-stage reasoning chain execution (CoT Consistency, Chain-of-Thought Consistency)
256k context as a “working memory” (Working Memory, Working Memory)
Native INT4 acceleration + MoE activation sparsity scheduling

This is a completely different path from “stacking parameters → stacking benchmarks,” emphasizing reasoning ability over parameter scale.

In short:

K2 is the first time a Chinese model has entered the sequence of thinking models (Thinking Model, Thinking Model).

Dissecting K2’s Technical Approach

K2’s technical approach can be broken down into five key points, each directly impacting the model’s reasoning ability and ecosystem adaptability.

MoE Expert Division: Cognitive Division Rather Than Parameter Expansion

K2’s MoE (Mixture of Experts, Mixture of Experts) design philosophy is distinct from previous models. The core is not about activating fewer parameters or running larger models more cheaply, but about assigning different cognitive sub-skills to different experts. For example:

Mathematical reasoning expert
Planning expert
Tool invocation expert
Browser task expert
Code generation expert
Long-chain retention expert

This division aligns directly with Claude 3.5’s cognitive layering (Cognitive Layering, Cognitive Layering) approach. K2’s MoE is about “dividing thinking among the model,” not just “making computation cheaper.”

256K Context: Building the Model’s Working Memory

K2’s ultra-long context is not just a parameter showcase; it’s designed to build the model’s “thinking buffer.” It allows the entire process to retain reasoning chains, tool invocation states, multi-stage reflection, and uninterrupted long tasks (such as research or code refactoring), stably executing multi-stage agent workflows. Long-term thinking requires long-term memory support, and K2’s long context is the “memory” for sustained reasoning chains.

Intertwined Training of Tool Invocation and Reasoning Chains

K2 excels in the intertwined training of tool invocation and reasoning chains. Traditional open-source models typically follow this process:

Generate reasoning
Output JSON function call
Tool returns result
Continue reasoning

In this approach, the reasoning chain and invocation chain are separated. K2’s training allows the reasoning chain to invoke tools at any time and feed tool results back into the reasoning chain for the next stage of thinking. It supports 200–300 consecutive tool invocations without interruption, fully aligning with Claude 3.5’s Interleaved CoT + Tool Use.

Native INT4 Quantization: Ensuring Reasoning Chain Stability

K2’s INT4 (INT4, 4-bit Integer Quantization) approach is not ordinary post-quantization. Its purpose is not only to reduce memory usage and increase throughput, but more importantly, to ensure that deep reasoning chains do not break due to insufficient computing power. The biggest killer of deep thinking chains is timeout, freezing, or unstable workers. INT4 enables Chinese GPUs (non-H100) to run complete reasoning chains, which is highly significant for China’s ecosystem.

MoE + Long Context + Toolchain: Unified Training Rather Than Module Stitching

K2’s most important feature is its holistic training approach: expert division, long context-driven consistency, tool invocation trained through real execution, browser tasks and long-step task reinforcement, and INT4 entering the training loop. It’s not a “ChatLLM + Memory + RAG + Tools” patchwork, but an integrated reasoning system.

Alignment and Differences Between K2 and International Mainstream Approaches

K2 is highly aligned with international mainstream models (such as Claude, Gemini, OpenAI) in cognitive reasoning, ultra-long context, and tool invocation mechanisms, but also has unique advantages for Chinese models:

Native INT4 + adaptation to Chinese computing power is rare globally
Toolchain continuity is more stable than most open-source models
Higher degree of open source, stronger ecosystem reusability

Collaborative Value of China’s AI Infra: K2 × RLinf × Mem-alpha

A series of important open-source infrastructures have emerged in the K2 ecosystem. The table below summarizes these project types and their value to K2:

Here is a comparison table of the collaborative value of each infrastructure with K2:

Project	Type	Value to K2
RLinf	Reinforcement Learning	Used to train stronger planning/browser task capabilities
Mem-alpha	Memory Enhancement	Can be combined with K2 to form long-term memory agents
AgentDebug	Agent Error Debugging	Used to analyze K2’s toolchain errors
UI-Genie	GUI Agent Training	Can serve as an experimental field for K2’s agent capability expansion

Table 1: Collaborative Value of China’s AI Infra Ecosystem

This combination is already forming a China AI Agent Infra Stack.

Personal View: The Significance of K2’s Approach

I believe the significance of K2 lies not in the model itself, but in its technical approach:

K2 marks the first time Chinese models have shifted from “language generation competition” to “thinking ability competition.”

For the past three years, the main line of China’s open-source models has been evaluation scores, parameter scale, instruction following, and alignment data. But K2 is the first to clearly take the path of deep reasoning, tool intertwining, cognitive division, long-term task chains, and native performance optimization. This means China’s model trajectory is now synchronized with the US, rather than chasing old paths.

Key Directions to Watch in K2’s Ecosystem Over the Next Year

K2’s future ecosystem influence will depend on several key points:

Whether it opens the tool registry (Tool Registry, Tool Registry)
Whether it supports dynamic memory (Mem-alpha integration)
Whether it opens the MoE expert structure
Whether it can form a Chinese reasoning chain optimization path with vLLM / llm-d / KServe
Whether it supports fault tolerance for multi-node continuous reasoning chains

These capabilities will determine K2’s ecosystem influence and technical extensibility.

K2 Thinking Model Architecture Diagram

The following flowchart illustrates the core architecture of the K2 thinking model and its collaboration with external agents/applications:

Figure 1: K2 Thinking Model Architecture

Summary

K2 is the first time China’s model trajectory is heading in the right direction:

From “writing like humans” to “thinking like humans.”

The era of thinking models is coming, and Chinese models are finally standing on the same roadmap as the international forefront.

Jimmy Song – Jimmy Song's Blog

Core Model Overview

Four-Layer Model

Model Interactions

Model Application Value

The Yin-Yang Layer: Dynamic Balance of System States

Three Typical Pairs of Yin-Yang Tensions

Expansion ↔ Constraint

Innovation ↔ Governance

Speed ↔ Stability

The Art of Yin–Yang Balance

Practical Cases

Dynamic Transformation of Yin–Yang States

Five Elements Layer: Classification and Collaboration of System Roles

Engineering Mapping of Five Elements

Water – Data Flow and Quality

Wood – Model Growth and Capability Expansion

Fire – Compute Conversion and Work Efficiency

Earth – Platform Support and Orchestration Governance

Metal – Hardware Constraints and Physical Boundaries

Five Elements Generation Relationships

Five Elements Overcoming Relationships

Five Elements Balance Diagnosis

Common Imbalance Patterns

Balance Principles

The Yun Layer: Stages and Cycles of System Evolution

Exploration Period (Initial Stage)

Platform Period (Growth Stage)

Scale Period (Mature Stage)

Rebalancing/Substitution Period (Renewal Stage)

Evolutionary Cycle

The Art of Following the Momentum

Error Examples

Stage Assessment Checklist

Qi Layer: Effective System Flow and Pressure Fields

The Essence of Qi: Overall State of Affairs

Two States of Qi

Qi Flow: System Active

Qi Stagnation: System Pathological

Qi’s Flow Path

Two Forms of Qi

Healthy Flow

Pressure Propagation

Application of Qi Layer in Operations

Not Just Whether There’s a Problem, But How It’s Trending

Qi Disorder Precedes Major Incidents

Strategies for Guiding Qi Flow

Architecture Level

Strategy Level

Agent System Special Attention

Qi Layer Monitoring Practices

Summary

Dynamic Relationship Modeling: Five Elements Flow Under Yin-Yang Balance

Yin-Yang × Five Elements: Intrinsic Tension of Elements

Five Elements × Qi: Dynamic Network of Flow

Yun × Yin-Yang Five Elements: Boundary Conditions for Stage Evolution

Comprehensive Analysis Case: GPU Scheduling Scenario

Solutions

Value of Dynamic Modeling

Practical Recommendations

Engineering Practice Guide: Architecture Decisions Guided by Theory

Balance Yin and Yang, Avoid Extremes

Complete the Five Elements, Identify and Fill Weaknesses

Follow the Trend, Align with the Movement

Observe Qi Field, Optimize Flow

Dynamic Adjustment, Continuous Rebalancing

Practice Case: Large-Scale GPU Training Cluster Optimization

Scenario Application Quick Reference Table

Summary

System Diagnosis Principles: Criteria for Health Status

Five-Dimensional Diagnosis Framework

Five Elements Balance Check

Diagnosis Method

Identify Problems

Typical Symptoms

Qi Flow Smoothness Check

Diagnosis Method

Qi Smooth vs. Qi Not Smooth

Diagnosis Points

Yin-Yang Dynamics Check