Jimmy Song – Blog

Kubernetes as the GPU Control Plane: HAMi v2.9 and Next-Gen AI Infra

Jimmy Song — Thu, 14 May 2026 06:34:19 +0000

Recently, I’ve been following the progress of domestic GPU scheduling and Kubernetes AI resource models. With the release of HAMi v2.9, I want to share several observations on how the AI Infra control plane is evolving.

Figure 1: Kubernetes as the GPU Control Plane for AI

Why Discuss This Topic Now

When DeepSeek R1 was released in early 2025, most people focused on the fact that it trained a model competitive with OpenAI o1 for just $5.6 million. What struck me, however, was that as inference costs plummeted, GPU utilization issues would quickly come to the forefront.

As models became more useful and inference demand exploded, “one GPU per model” rapidly became a luxury. Meanwhile, the NVIDIA H200 export saga accelerated the adoption of domestic compute. First, a sales ban; then, at the end of 2025, a 25% tariff under Trump; and by January 2026, Chinese customs had cleared zero units. Policy now mandates that over 40% of data center chips must be domestically produced by 2026.

The reality is harsh: not only are GPUs scarce, but you must also learn to use NVIDIA, Ascend, Cambricon, Hygon, and other very different platforms simultaneously.

That’s why I believe HAMi v2.9 is more significant than it appears on the surface.

GPUs Are No Longer Just About the Card

Kubernetes has always managed GPUs in a rather crude way:

resources:
 limits:
 nvidia.com/gpu: 1

This was sufficient in 2019, when the main question was simply whether a Pod needed a GPU or not.

But that’s no longer enough. An inference service might only need 4GB of VRAM, multiple small models can share a single card, training jobs care about GPU topology and interconnect bandwidth, and multi-tenancy requires fault domain isolation. Treating GPUs as integer resources is like using an abacus for statistical analysis—not impossible, but the mental model is out of sync with reality.

The most notable feature in HAMi v2.9 is the HAMi-core mode for the Ascend 910C. Previously, sharing Ascend cards relied on SR-IOV hardware virtualization, which was coarse-grained and inflexible. HAMi-core takes a different approach: it uses LD_PRELOAD to intercept ACL calls in user space, enabling memory isolation at the MB level and compute throttling by percentage.

In short: it’s managed by software, not hardware slicing.

This is reminiscent of how SDN abstracted the network control plane from hardware devices—GPU partitioning is shifting from a hardware capability to a cluster control plane capability. Considering that Huawei shipped 810,000 Ascend 910C cards last year—nearly half of all domestic chips—this capability has significant real-world impact.

DRA: Kubernetes Finally Has a Robust Device Resource Model

Kubernetes v1.34 (September 2025) officially promoted DRA (Dynamic Resource Allocation) to GA, and Red Hat OpenShift 4.21 followed suit. This is a big deal.

The Device Plugin solved “how to connect GPUs to K8s,” but not “how to express complex AI resource requirements.” Device Plugins only know how many cards are on a node, not how much VRAM you need, what topology, or what isolation level.

DRA standardizes device resource declaration, allocation, and management via ResourceClaim and DeviceClass. HAMi-DRA takes a pragmatic approach: it doesn’t require users to change how they declare resources. Instead, it uses a Mutating Webhook to automatically convert existing Device Plugin-style declarations into the DRA model. Legacy systems don’t need to change, but can still leverage new capabilities.

I liken this to what CSI did for storage: it didn’t eliminate vendor differences, but allowed Kubernetes to consume different storage capabilities in a unified way. DRA does the same for AI accelerators—NVIDIA, Ascend, AMD, Vastai cards will never be identical, but the scheduling layer should speak a common language.

A Complete Control Plane Path

If we look beyond individual features and consider HAMi-core, DRA, CDI, and the scheduler together, they actually correspond to different layers of GPU resource management:

HAMi-core: How to partition and isolate devices internally
DRA: How to declare, allocate, and bind resources
CDI: How to standardize device injection into container runtimes
Scheduler/Webhook: How to schedule, admit, and observe

Connecting these layers, from top to bottom, forms the complete Kubernetes GPU Control Plane:

Figure 2: Kubernetes as the GPU Control Plane for AI Workloads

This is a complete control plane path. In v2.9, Volcano vGPU was upgraded to v0.19 with enhanced CDI support. While this may seem like a minor improvement in device injection, it actually completes a critical link in this chain.

Heterogeneity Is the Main Battlefield for Domestic AI Infra

The reality for domestic AI clusters: you can’t build infrastructure around just one type of GPU.

Enterprise environments often have NVIDIA, Ascend, Biren, Cambricon, Hygon, Muxi, Kunlunxin, Vastai, and other devices coexisting. Each card has different drivers, runtimes, virtualization capabilities, and monitoring methods. HAMi v2.9 adds support for Vastai, covering more than ten types of heterogeneous compute devices. Mixed training and inference, online and offline workloads, domestic and overseas GPUs, multi-team and multi-tenant resource pools—in these scenarios, unified scheduling is far more important than single-card performance.

Key Judgments

GPU sharing will shift from a cost-saving measure to a default requirement. After the explosion of inference workloads, not every workload deserves exclusive access to an entire card. Exclusive allocation will increasingly become a luxury.

DRA is the way forward, but migration will be gradual. The Device Plugin ecosystem is too large to disappear overnight. HAMi-DRA’s compatibility layer shows the project team understands this.

Heterogeneous scheduling will become the core challenge for AI Infra. Whoever can abstract different vendor devices into a unified scheduling language will control the key position in the control plane.

Kubernetes will be reshaped by AI workloads. From scheduling semantics to resource models, AI requires much greater expressiveness than traditional web services. DRA, CDI, topology-aware scheduling—these are not isolated evolutions, but all point to one thing: Kubernetes is evolving from a container orchestrator to the control plane for AI computing.

Summary

The significance of HAMi v2.9 is not just in supporting a particular device or partitioning method, but in making one thing clear: the next generation of AI infrastructure competition is not just about model frameworks or GPU counts, but about the control plane.

GPUs are shifting from external devices on nodes to native resources within the Kubernetes control plane. Whoever defines the resource model for the AI era will define the long-term boundaries of AI Infra.

Kubernetes's Anxiety and Rebirth in the AI Wave

Jimmy Song — Fri, 03 Apr 2026 05:20:28 +0000

Kubernetes hasn’t been replaced by AI, but it’s being redefined by it. Anxiety is the prelude to rebirth.

After attending KubeCon EU 2026 in Amsterdam, I’ve been pondering a key question: Kubernetes isn’t obsolete, but it’s no longer “enough”; it hasn’t been replaced by AI, but it’s being redefined by AI.

Figure 1: KubeCon EU 2026 slogan: Keep Cloud Native Moving. This event had over 13,000 registrations, making it the largest KubeCon to date.

This was my third time attending KubeCon in Europe. Over the past few years, you can actually see the community’s mindset shift through the event slogans:

2024 Paris: La vie en Cloud Native

→ Cloud Native has become a “way of life,” the default state
2025 London: No slogan, just the 10th anniversary

→ Kubernetes reached a milestone, focusing on retrospection rather than moving forward
2026 Amsterdam: Keep Cloud Native Moving

→ But the question is: where is it moving?

The absence of a slogan in 2025 was a signal in itself:

When an ecosystem starts commemorating the past instead of defining the future, it’s already at an inflection point.

This article doesn’t recap the talks, but instead distills my observations at KubeCon into insights about Kubernetes’ anxiety and rebirth in the AI wave.

The Root of Anxiety: Is Kubernetes Facing a “Crisis”?

The biggest change at KubeCon was that AI has completely replaced traditional cloud native topics. The focus shifted from service optimization and microservices management to how to deploy and manage AI workloads on Kubernetes, especially inference tasks and GPU scheduling.

Figure 2: Before KubeCon officially started, the Maintainer Summit was all about AI.

Kubernetes, as the foundational infrastructure, was once the core of the cloud native world. With the explosive growth of AI models, the question now is whether Kubernetes can still serve as a “universal” platform for everything, which has become a new source of anxiety.

The AI boom brings real challenges: Can Kubernetes’ “universality” adapt to the complexity of AI workloads?

The Focus Brought by the AI Boom

AI’s popularity has shifted the cloud native spotlight entirely to artificial intelligence. AI coding, OpenClaw, large language models, and generative models have all drawn widespread attention. AI has become the core computing demand in the real world.

This surge in demand raises the question: Can Kubernetes continue to serve as the infrastructure platform for complex tasks? Especially with issues like GPU sharing, inference model scheduling, VRAM allocation, and device attribute selection, is the traditional Kubernetes resource model sufficient?

In the past, Kubernetes handled compute, storage, and networking as foundational infrastructure. But with the rapid development of AI, its “universality” is being challenged. Particularly for inference tasks, Kubernetes’ model appears thin.

Comparing with OpenStack: Will Kubernetes Repeat History?

OpenStack once aimed to be a complete open-source cloud platform, but ultimately failed to sustain growth due to complexity and a lack of flexibility in adapting to new technologies.

Will Kubernetes follow the same path? I believe Kubernetes has different strengths: as a container and microservices orchestration platform, it’s widely adopted and has strong community and vendor support. It doesn’t try to replace all cloud provider capabilities but serves as an infrastructure control plane to help users manage resources.

Figure 3: Cloud native contributors remain active. The crowd at the KubeCon EU 2026 Maintainer Summit shows the community’s vitality.

However, as AI workloads become mainstream, Kubernetes must find a new position to avoid being replaced by “AI-optimized platforms.”

Kubernetes’ Challenge: The GPU Resource Management Gap

At KubeCon, NVIDIA announced the donation of the GPU DRA (Dynamic Resource Allocation) driver to the CNCF, marking the upstreaming of GPU resource management. GPU sharing and scheduling have become urgent issues for Kubernetes.

Traditionally, Kubernetes relied on the Device Plugin model to schedule GPUs, only supporting allocation by device count (e.g., nvidia.com/gpu: 1). But for AI inference tasks, more information is needed for resource scheduling, such as VRAM size, GPU topology, and sharing strategies. NVIDIA DRA makes GPU resource management more flexible and intelligent, gradually easing the “GPU resource crunch” in AI workloads.

This shift means Kubernetes is no longer just a “container orchestration platform,” but is becoming the infrastructure layer for AI-specific resource scheduling.

Against this backdrop, both the community and industry are exploring finer-grained GPU resource abstraction and scheduling mechanisms. For example, the open-source project HAMi is building a GPU resource management layer for AI workloads on top of Kubernetes, supporting GPU sharing, VRAM-level allocation, and heterogeneous device scheduling.

Figure 4: HAMi demo at KubeCon EU 2026 Keynote

These efforts are not about replacing Kubernetes, but about filling the resource model gaps for the AI era. In the long run, this layer may evolve into a “GPU Abstraction Layer” similar to CNI/CSI, becoming a key part of AI-native infrastructure.

The Production “Gap”: Many AI PoCs, Few in Production

A common post-event summary was: Many PoCs, but “everyday production deployments” are still rare. Pulumi summarized it as:

lots of working demos, very few production setups people trust

This shows that while many AI workload solutions succeed in technical demos, the transition from experimentation to production remains difficult. Whether it’s GPU resource sharing or inference request scheduling, whether Kubernetes as the foundation can support this transformation is still an open question.

The Rise of Inference Systems: Kubernetes’ Scheduling Boundaries Are Challenged

Another major event at this KubeCon was llm-d being contributed to the CNCF as a Sandbox project.

If GPU DRA represents the upstreaming of device resource models, then llm-d represents another critical evolution: Distributed LLM inference capabilities are moving from proprietary engineering implementations to standardized, community-driven collaboration in cloud native.

This is significant not just because it’s another open-source project, but because it shows that Kubernetes’ challenges in the AI era are no longer just about “how to schedule GPUs,” but also “how to host inference systems themselves.” As prefill/decode separation, request routing, KV cache management, and throughput optimization move into the infrastructure layer, Kubernetes’ boundaries are being redefined.

Traditionally, the Kubernetes scheduler focused on Pod scheduling. But in AI inference scenarios, scheduling is not just about picking a node—it’s about selecting the most suitable inference instance based on request characteristics. Factors like model state, request queue depth, and cache hit rate all need to be considered. This process is increasingly managed by inference runtimes, forming new “request-level scheduling” systems.

This leads to an overlap between the Kubernetes scheduler and inference systems, forcing Kubernetes to rethink its role: should it keep expanding, or collaborate with inference systems?

AI-Native Infrastructure: The Key Challenge for Production

At the AI Native Summit, the real needs for AI-native infrastructure were especially clear. The focus was no longer “can it run on Kubernetes,” but how to make AI workloads routine, stable, and production-ready on Kubernetes.

Figure 5: At the AI Native Summit after KubeCon, Linux Foundation Chairman Jonathan said cloud native is entering the AI-native era.

The core challenge is delivery. Unlike traditional apps, AI model weights are often huge—tens of GB or even TB—making model delivery and data management extremely complex. Traditional container delivery systems (like image layers) struggle with such massive data and complex versioning.

A key direction for Kubernetes is to standardize model weight and data delivery, using ImageVolume and OCI artifacts to solve AI model delivery and version management on Kubernetes. This not only reduces “cold start” times but also provides infrastructure support for multi-tenancy and compliance.

Summary

Kubernetes won’t be replaced by AI, but it’s being reshaped as the core of infrastructure. This anxiety is the force driving its evolution—it’s moving from a “general-purpose infrastructure platform” to an “AI-powered multifunctional base”. Some even call it the AI operating system.

In the future, Kubernetes’ core competitiveness will no longer be just container management, but how effectively it can schedule and manage AI workloads, and how it can make AI a routine part of operations. This was my biggest takeaway from the AI Native Summit and KubeCon, and it’s what I look forward to in the Kubernetes ecosystem over the next few years.

References

Day One in Amsterdam: Kubernetes Is Rethinking AI

Jimmy Song — Sun, 22 Mar 2026 20:41:19 +0000

Today marks my first day at KubeCon Europe 2026. The most striking feeling is: the world is vast, but this community is truly small.

Figure 11: Jimmy on the first day of KubeCon EU 2026

One strong impression stands out:

The world is big, but this circle is really small.

Old Friends, New Cycle

At the Maintainer Summit, I met many familiar faces—

Colleagues from Ant Group, friends from Tetrate, and some people I’ve known for nearly a decade. Together, we’ve journeyed from the early days of Kubernetes, Service Mesh, and cloud native infrastructure to today.

In a sense, this generation has fully experienced:

The rise of Kubernetes
The standardization of Cloud Native
The microservices and service mesh boom
And now, the era of AI Infrastructure

This isn’t about “new people entering the field,” but rather—

The same group stepping into a new technology cycle.

What Is the Maintainer Summit Discussing?

If you ask:

What is the Kubernetes community most concerned about right now?

Today’s answer is very clear:

👉 How to run AI workloads better on Kubernetes

Figure 12: The Maintainer Summit’s main topic is AI Infra

Many topics at the Maintainer Summit revolved around:

Scheduling models for LLM / AI workloads
GPU / accelerator resource management
Integrating inference systems with Kubernetes
Redefining the roles of data plane vs. control plane
How observability tools like OTel monitor AI workloads

In other words:

Kubernetes hasn’t been replaced by AI; it’s actively “absorbing” AI.

Key Signal: GPUs Are Becoming the “Infrastructure Layer”

Today, I had an in-depth discussion with CNCF TOC, Red Hat, and the vLLM community.

The core question was:

How should GPUs be “platformized”?

Some consensus is already clear:

GPUs are no longer just devices
They are now a schedulable, partitionable, and shareable resource layer

Figure 13: TOC meeting discussing GPU resource management and LLM Serving integration

At the Maintainer Summit in Amsterdam, we had deep discussions with CNCF TOC, Red Hat, and the vLLM community about GPU resource management and LLM Serving integration in Kubernetes scenarios, and explored potential collaboration between vLLM and HAMi.

Behind this is a major paradigm shift:

Past	Now
GPU = Node resource	GPU = Infrastructure layer
Exclusive use	Multi-tenant sharing
Static binding	Dynamic scheduling
Managed within frameworks	Unified management at the platform layer

This is exactly what we’ve been working on in HAMi.

HAMi: From “Project” to “Reference Pattern”

Another interesting change today:

HAMi is no longer just a “community project”—it’s becoming:

A reference implementation (reference pattern) for AI Infra

Figure 14: Li Mengxuan, CTO of Dynamia, sharing HAMi’s design and practice at KubeCon EU 2026 Maintainer Summit

This is reflected in several ways:

Invited to present at the Maintainer Summit
Participating in CNCF TOC discussions
Involved in incubating review demos
Exploring joint content with the vLLM community (even discussing a joint blog 👀)

Especially in conversations with Red Hat and vLLM, a clear trend emerged:

GPU resource management and LLM serving are becoming coupled

That is:

Upper layer: vLLM / inference frameworks
Lower layer: GPU scheduling / sharing

A new “interface layer” is gradually forming.

This is a direction worth betting on.

Figure 15: At the TAG Workshop, HAMi was discussed as an Incubating demo

A Caution: The AI Infra Startup Boom Hasn’t Really Begun

At the same time, I have a somewhat “counterintuitive” observation:

We haven’t yet seen a large wave of AI Infra (K8s-focused) startups.

Most companies I saw today:

Many are pivoting from CI/CD, Service Mesh, or Gateway
Many are traditional cloud vendors extending into AI
Many are working on models, agents, or even lower-level tech

But those truly focused on:

“Making AI workloads run better on Kubernetes”

There are actually not many startups at this layer.

This could mean two things:

1) This Layer Isn’t Fully Formed Yet

Currently, most activity is at:

The model layer (LLM / foundation models)
The application layer (Agent / Copilot)

But not at:

The scheduling layer
The resource layer
The runtime layer

2) Or, the Barrier to Entry Is Very High

Because at its core, this is:

The intersection of Cloud Native × GPU × AI workload

It’s not just “wrapping AI,” but a fundamental re-architecture at the infrastructure level.

My Take

If we break down the AI technology stack:

Agent / Application
 ↓
LLM Serving (vLLM, etc.)
 ↓
AI Runtime / Scheduling
 ↓
GPU Resource Layer
 ↓
Hardware

Most innovation today is concentrated in:

The top two layers (Agent / LLM)

But the real long-term moat lies in:

The middle two layers (Runtime + Resource Layer)

And Kubernetes is very likely to remain:

The default platform for this middle layer

Summary

Today’s takeaway:

Kubernetes is not obsolete; it’s being redefined.

And our generation is shifting from:

“Cloud Native Builders”

to:

“AI Infrastructure Builders”

More to come tomorrow.

HAMi Website Refactor: Why HAMi Docs and Website Underwent a Complete Redesign

Jimmy Song — Tue, 17 Mar 2026 08:55:52 +0800

This redesign is more than a style update—it’s a step toward clearer technical communication and better user experience. Try the new HAMi website at https://project-hami.io and submit issues here.

Over the past two months, I conducted a thorough refactor of the documentation website (see GitHub). Externally, it looks like a “visual redesign”, but from the perspective of community maintainers and content builders, it’s a comprehensive upgrade of information architecture, content system, and frontend experience.

This article aims to systematically explain three things: why we did this refactor, what exactly changed, and what these changes mean for the HAMi community.

Why Refactor the Website and Documentation

HAMi is a CNCF-hosted open source project initiated and contributed by Dynamia, with growing influence in GPU virtualization, heterogeneous compute scheduling, and AI infrastructure. The community content is expanding, and user types are becoming more diverse: from first-time visitors to engineers and enterprise users seeking deployment docs, architecture diagrams, case studies, and ecosystem information.

The original site was functional, but as content grew, several issues became apparent:

The homepage lacked information density, making it hard to quickly grasp the project’s overall value.
Connections between docs, blogs, and community info were not smooth; content entry points were scattered.
Search experience was unstable; external solutions were not ideal in practice.
Mobile experience had many details needing improvement, especially navigation, card layouts, and footer areas.
Visual style was inconsistent, making it hard to convey community influence and engineering maturity.

For a fast-evolving open source community, the website is not just a “place for docs”, but the public interface of the community. It needs to serve as project introduction, knowledge gateway, adoption proof, community connector, and brand expression.

So the goal of this refactor was clear: not just superficial beautification, but to truly upgrade the website into HAMi’s systematic community entry point.

What Was Done in This Refactor

This update was not a single-point change, but a series of systematic improvements.

Homepage Redesign and Complete Information Architecture Overhaul

The most obvious change is the homepage.

We redesigned the homepage structure, moving away from simply stacking content blocks, and instead organizing the page around the main narrative: “Project Positioning → Core Capabilities → Ecosystem Entry → Content Accumulation → Community Trust”.

Specifically, the homepage received several key upgrades:

Rebuilt the Hero section to strengthen first-screen information delivery and action entry.
Optimized CTA design so users can quickly access docs, blogs, and resources.
Added and enhanced multiple homepage sections to showcase project value and community reach in a more structured way.
Adjusted visual hierarchy, background atmosphere, and scroll rhythm, transforming the homepage from a “content list” into a “narrative page”.

These changes include Hero animations and atmosphere layers, research/story sections, new resource entry sections, refreshed CTAs, unified background design, and ongoing reduction of visual noise. Together, they solve a core problem: enabling visitors to understand what HAMi is and why it’s worth exploring further within seconds.

Architecture Diagrams

Key diagrams were redrawn for clearer technical communication. This helps users grasp HAMi’s role in AI infrastructure.

Figure 1: HAMi website homepage architecture diagram

For HAMi, this change is critical. The community faces not just a single feature, but a set of system-level challenges involving Kubernetes, schedulers, GPU Operators, heterogeneous devices, and enterprise platforms. Improved diagrams make the website a better technical entry point.

Added Case Studies, Community, and Ecosystem Sections to Make Impact Visible

Another important direction was strengthening the “community proof” layer.

Many open source project sites fall into the trap of having complete docs, but users can’t tell if the project is truly adopted, if the community is active, or if the ecosystem is expanding. The HAMi website redesign consciously addresses this.

Figure 2: HAMi ecosystem and device support

Figure 3: HAMi adopters

Figure 4: HAMi contributor organizations

Blog & Reading Experience

Blog cards, lists, and metadata were unified for easier reading and sharing. Blogs are now a core communication layer.

Figure 5: HAMi website blog list page

Mobile Optimization

Navigation, card layouts, footer, and search were improved for smoother mobile browsing.

Figure 6: HAMi website mobile view

Footer layout was enhanced for better navigation and credibility. Built-in search replaced unreliable external solutions, improving content accessibility.

Figure 7: HAMi website footer

Figure 8: HAMi website built-in search

What This Redesign Means for the HAMi Community

From screenshots, it looks like “the website looks better”. But from a community-building perspective, its significance is deeper.

First, HAMi’s external expression is more systematic.

The website is no longer just a collection of scattered pages, but is forming a complete narrative chain: users can understand project value from the homepage, capability details from docs, practical paths from blogs, and community impact from ecosystem modules.

Second, community content assets are reorganized.

Previously, valuable articles, diagrams, and explanations existed but were hard to find. Now, through homepage sections, navigation, and search refactor, these contents are more effectively connected.

Third, HAMi’s community image is more mature.

A mature open source project needs not just an active code repository, but clear, stable, and sustainable website expression. Structure, style, and usability are part of the community’s engineering capability.

Fourth, this lays the foundation for expanding case studies, adopters, contributors, and ecosystem content.

With the framework sorted, adding more case studies, collaboration entry points, or showcasing more adopters and partners will be more natural and easier for users to understand.

As a Community Contributor, My Top Three Takeaways from This Redesign

In summary, I believe this refactor got three things right:

Upgraded the website from a “content dump” to a “community gateway”.
Combined visual optimization with information architecture adjustment, not just a skin change.
Improved basic experiences like search, mobile, navigation, and footer.

These may not be as flashy as launching a new feature, but they directly impact content dissemination, user comprehension, and the project’s long-term image.

For infrastructure projects like HAMi, technical capability is fundamental, but clearly communicating, organizing, and continuously presenting that capability is also a form of infrastructure.

Summary

This HAMi documentation and website refactor is essentially an upgrade to the community’s “expression layer” infrastructure.

It improves visual and reading experience, reorganizes content, homepage narrative, search paths, mobile access, and community signal display. Homepage redesign, architecture diagram redraw, unified blog style, mobile optimization, enhanced footer, and switching from external to built-in search together constitute a true “refactor”.

Externally, it helps more people quickly understand HAMi; internally, it provides a stable platform for the community to accumulate case studies, expand the ecosystem, and serve adopters and contributors.

The website is not an accessory to the open source community, but part of its long-term influence. HAMi’s redesign is about taking this seriously.

If you’re interested in Kubernetes GPU virtualization, add me on WeChat jimmysong or scan the QR code below.

Check out the HAMi project on GitHub

GTC 2026 Eve: AI is Becoming the New Infrastructure

Jimmy Song — Sun, 15 Mar 2026 11:34:06 +0800

AI is quietly reshaping the infrastructure landscape, and GTC 2026 may become a key node in this transformation.

Next week, one of the most important technology conferences in the AI industry, NVIDIA GTC 2026, will be held in San Jose, USA.

For many people, GTC is just a GPU technology conference. But if you follow the development of the AI industry over the past few years, you’ll find an interesting phenomenon:

Many important narratives about AI infrastructure are gradually taking shape at GTC.

From CUDA, DGX, to AI Factory, and most recently Jensen Huang’s proposed AI Five-Layer Cake, NVIDIA is constantly attempting to redefine the computing infrastructure of the AI era.

This is why many people call GTC:

AI’s “Woodstock.”

Figure 1: NVIDIA GTC Conference

This year’s GTC (March 16-19) is expected to cover various levels of the AI stack, including:

AI Chips
AI Data Centers
AI Agents
Robotics
Inference Computing

According to NVIDIA’s official blog, this year’s keynote will focus on the complete AI stack from chips to applications.

If we put these signals together, we can actually see a larger trend:

AI is transforming from an “applied technology” into “infrastructure.”

The Perspective of Industrial Revolutions

From a longer time scale, the technological revolutions in human history are essentially infrastructure revolutions.

We usually divide industrial revolutions into four times.

In the table below, you can see the infrastructure corresponding to each industrial revolution:

Industrial Revolution	Infrastructure
Steam Revolution	Steam Engine
Electrical Revolution	Power Grid
Digital Revolution	Computer
Internet Era	Network

Table 1: Industrial Revolutions and Corresponding Infrastructure

First Industrial Revolution: Steam

The steam engine allowed humans to utilize mechanical power on a large scale for the first time. Production no longer relied on human or animal power, but on machines.

Second Industrial Revolution: Electricity

Electricity changed not only the source of power, but also the organization of production. Assembly lines, large-scale manufacturing, and modern industrial systems are all built on the foundation of the power grid.

Third Industrial Revolution: Computers

Computers allowed information to be processed digitally. Software became a production tool.

Fourth Industrial Revolution: Internet and Intelligence

The internet connects all computers together. Cloud computing transforms computing resources into infrastructure. And AI gives machines a certain degree of “cognitive ability.”

The True Significance of AI

If we observe these industrial revolutions, we discover a pattern:

Each industrial revolution produces a new General Purpose Infrastructure.

And AI is likely to become the next-generation infrastructure.

NVIDIA even directly stated in a recent article:

AI is essential infrastructure, like electricity and the internet.

In other words:

AI is no longer just an applied technology, but a new factor of production.

NVIDIA’s Five-Layer Cake

Recently, Jensen Huang proposed a very interesting concept: AI Five-Layer Cake.

Figure 2: AI Five Layer Cake (Image source: NVIDIA)

AI is broken down into five layers:

Energy
Chips
AI Infrastructure
Models
Applications

This model actually illustrates one thing:

AI is a complete industrial system.

Jensen Huang even described AI at Davos as:

“One of the largest-scale infrastructure constructions in human history.”

Signals GTC 2026 May Release

This year’s GTC is expected to release several important directions.

Inference Computing

The focus of AI in the past was training. But the main load of AI in the future is likely to be Inference.

Analysts expect that by 2030, 75% of computing demand in the AI data center market will come from inference.

Agentic AI

The past AI model was:

User → Model → Answer

The Agent model is more complex:

User → Agent → Tools → Model → Action

The flowchart below shows the main interaction paths in the Agent model:

Figure 3: Agentic AI Interaction Flow

AI is no longer just answering questions, but executing tasks.

Agent Platform

Recent media reports suggest that NVIDIA may launch a new Agent platform: NemoClaw, aimed at helping enterprises deploy AI Agents.

If this project is truly released, it means NVIDIA’s stack will become the following structure:

Figure 4: NVIDIA Agent Platform Architecture

This is actually a complete AI stack.

Agents Change Computing Workloads

The emergence of Agents brings new computing workload issues.

Past AI workloads were mainly:

Training
Inference

But Agents bring a third type of workload:

Agent Workloads

The figure below shows the diverse workload types related to Agents:

Figure 5: Agent Workloads Structure

The characteristic of this workload is highly fragmented. GPUs are no longer occupied for long periods, but rather face many small requests. This poses new challenges for infrastructure.

AI-Native Infrastructure

For the past few years, I’ve been thinking about a question:

What is AI-native infrastructure?

It is clearly not just “Kubernetes with GPUs.” I’m more inclined to believe it needs to possess several characteristics.

GPU as a First-Class Resource

In the cloud computing era, CPU is the core resource. In the AI era, GPU is the core resource.

Heterogeneous Computing

Real-world AI chips are not limited to NVIDIA:

NVIDIA
Ascend
Cambricon
Metax
Moore Threads

Future AI infrastructure must be able to manage heterogeneous computing.

GPU is a very expensive resource. If it cannot be shared, utilization will be very low. This is why GPU virtualization and slicing are becoming increasingly important.

AI Scheduling

AI scheduling includes not only traditional CPU and Memory, but also:

GPU
VRAM
Topology
Bandwidth

A Possible AI Tech Stack

Combining the above trends, the future AI stack may present the following structure:

Figure 6: AI Tech Stack Evolution

This structure is very close to NVIDIA’s Five-Layer Cake.

My Judgment

Combining signals from GTC, AI Factory, Agents, and AI Five-Layer Cake, we can see a very obvious trend:

AI is rewriting computing infrastructure.

Future competition may not just be “who has the best model,” but:

Who has the best AI Infrastructure.

Just like the past few decades:

Electricity determines industrial capability
Internet determines information capability
Cloud computing determines software capability

The future may be:

AI Infrastructure determines intelligence capability.

Summary

If we stretch the time scale a bit longer, we may be in a new historical stage.

AI is no longer just a technological tool. It is becoming new infrastructure.

Just like:

Electricity
Internet
Cloud computing

And AI-native infrastructure is likely to become one of the most important technology directions for the next decade.

When GPUs Move Toward Open Scheduling: Structural Shifts in AI Native Infrastructure

Jimmy Song — Fri, 13 Feb 2026 14:32:46 +0000

The future of GPU scheduling isn’t about whose implementation is more “black-box”—it’s about who can standardize device resource contracts into something governable.

Figure 1: GPU Open Scheduling

Introduction

Have you ever wondered: why are GPUs so expensive, yet overall utilization often hovers around 10–20%?

Figure 2: GPU Utilization Problem: Expensive GPUs with only 10-20% utilization

This isn’t a problem you solve with “better scheduling algorithms.” It’s a structural problem - GPU scheduling is undergoing a shift from “proprietary implementation” to “open scheduling,” similar to how networking converged on CNI and storage converged on CSI.

In the HAMi 2025 Annual Review, we noted: “HAMi 2025 is no longer just about GPU sharing tools—it’s a more structural signal: GPUs are moving toward open scheduling.”

By 2025, the signals of this shift became visible: Kubernetes Dynamic Resource Allocation (DRA) graduated to GA and became enabled by default, NVIDIA GPU Operator started defaulting to CDI (Container Device Interface), and HAMi’s production-grade case studies under CNCF are moving “GPU sharing” from experimental capability to operational excellence.

This post analyzes this structural shift from an AI Native Infrastructure perspective, and what it means for Dynamia and the industry.

Why “Open Scheduling” Matters

In multi-cloud and hybrid cloud environments, GPU model diversity significantly amplifies operational costs. One large internet company’s platform spans H200/H100/A100/V100/4090 GPUs across five clusters. If you can only allocate “whole GPUs,” resource misalignment becomes inevitable.

“Open scheduling” isn’t a slogan—it’s a set of engineering contracts being solidified into the mainstream stack.

Standardized Resource Expression

Before: GPUs were extended resources. The scheduler didn’t understand if they represented memory, compute, or device types.

Figure 3: Open Scheduling Standardization Evolution

Now: Kubernetes DRA provides objects like DeviceClass, ResourceClaim, and ResourceSlice. This lets drivers and cluster administrators define device categories and selection logic (including CEL-based selectors), while Kubernetes handles the full loop: match devices → bind claims → place Pods onto nodes with access to allocated devices.

Even more importantly, Kubernetes 1.34 stated that core APIs in the resource.k8s.io group graduated to GA, DRA became stable and enabled by default, and the community committed to avoiding breaking changes going forward. This means the ecosystem can invest with confidence in a stable, standard API.

Standardized Device Injection

Before: Device injection relied on vendor-specific hooks and runtime class patterns.

Now: The Container Device Interface (CDI) abstracts device injection into an open specification. NVIDIA’s Container Toolkit explicitly describes CDI as an open specification for container runtimes, and NVIDIA GPU Operator 25.10.0 defaults to enabling CDI on install/upgrade—directly leveraging runtime-native CDI support (containerd, CRI-O, etc.) for GPU injection.

This means “devices into containers” is also moving toward replaceable, standardized interfaces.

On this standardization path, HAMi’s role needs redefinition: it’s not about replacing Kubernetes—it’s about turning GPU virtualization and slicing into a declarative, schedulable, governable data plane.

Data Plane Perspective

HAMi’s core contribution expands the allocatable unit from “whole GPU integers” to finer-grained shares (memory and compute), forming a complete allocation chain:

Device discovery: Identify available GPU devices and models
Scheduling placement: Use Scheduler Extender to make native schedulers “understand” vGPU resource models (Filter/Score/Bind phases)
In-container enforcement: Inject share constraints into container runtime
Metric export: Provide observable metrics for utilization, isolation, and more

This transforms “sharing” from ad-hoc “it runs” experimentation into engineering capability that can be declared in YAML, scheduled by policy, and validated by metrics.

Scheduling Mechanism: Enhancement, Not Replacement

HAMi’s scheduling doesn’t replace Kubernetes—it uses a Scheduler Extender pattern to let the native scheduler understand vGPU resource models:

Filter: Filter nodes based on memory, compute, device type, topology, and other constraints
Score: Apply configurable policies like binpack, spread, topology-aware scoring
Bind: Complete final device-to-Pod binding

This architecture positions HAMi naturally as an execution layer under higher-level “AI control planes” (queuing, quotas, priorities)—working alongside Volcano, Kueue, Koordinator, and others.

Figure 4: HAMi Scheduling Architecture (Filter → Score → Bind)

CNCF public case studies provide concrete answers: in a hybrid, multi-cloud platform built on Kubernetes and HAMi, 10,000+ Pods run concurrently, and GPU utilization improves from 13% to 37% (nearly 3×).

Figure 5: CNCF Production Case Studies: Ke Holdings 13%→37%, DaoCloud 80%+ utilization, SF Technology 57% savings

Here are highlights from several cases:

Case Study 1: Ke Holdings (February 5, 2026)

Environment: 5 clusters spanning public and private clouds
GPU models: H200/H100/A100/V100/4090 and more
Architecture: Separate “GPU clusters” for large training tasks (dedicated allocation) vs “vGPU clusters” with HAMi fine-grained memory slicing for high-density inference
Concurrent scale: 10,000+ Pods
Outcome: Overall GPU utilization improved from 13% to 37% (nearly 3×)

Case Study 2: DaoCloud (December 2, 2025)

Hard constraints: Must remain cloud-native, vendor-agnostic, and compatible with CNCF toolchain
Adoption outcomes:
- Average GPU utilization: 80%+
- GPU-related operating cost reduction: 20–30%
- Coverage: 10+ data centers, 10,000+ GPUs
Explicit benefit: Unified abstraction layer across NVIDIA and domestic GPUs, reducing vendor dependency

Case Study 3: Prep EDU (August 20, 2025)

Negative experience: Isolation failures in other GPU-sharing approaches caused memory conflicts and instability
Positive outcome: HAMi’s vGPU scheduling, GPU type/UUID targeting, and compatibility with NVIDIA GPU Operator and RKE2 became decisive factors for production adoption
Environment: Heterogeneous RTX 4070/4090 cluster

Case Study 4: SF Technology (September 18, 2025)

Project: EffectiveGPU (built on HAMi)
Use cases: Large model inference, test services, speech recognition, domestic AI hardware (Huawei Ascend, Baidu Kunlun, etc.)
Outcomes:
- GPU savings: Large model inference runs 65 services on 28 GPUs (37 saved); test cluster runs 19 services on 6 GPUs (13 saved)
- Overall savings: Up to 57% GPU savings for production and test clusters
- Utilization improvement: Up to 100% GPU utilization improvement with GPU virtualization
Highlights: Cross-node collaborative scheduling, priority-based preemption, memory over-subscription

These cases demonstrate a consistent pattern: GPU virtualization becomes economically meaningful only when it participates in a governable contract—where utilization, isolation, and policy can be expressed, measured, and improved over time.

Strategic Implications for Dynamia

From Dynamia’s perspective (and as VP of Open Source Ecosystem), the strategic value of HAMi becomes clear:

Two-Layer Architecture: Open Source vs Commercial

HAMi (CNCF open source project): Responsible for “adoption and trust,” focused on GPU virtualization and compute efficiency
Dynamia enterprise products and services: Responsible for “production and scale,” providing commercial distributions and enterprise services built on HAMi

Figure 6: Dynamia Dual Mechanism: Open Source vs Commercial

This boundary is the foundation for long-term trust—project and company offerings remain separate, with commercial distributions and services built on the open source project.

Global Narrative Strategy

The internal alignment memo recommends a bilingual approach:

First layer: Lead globally with “GPU virtualization / sharing / utilization” (Chinese can directly use “GPU virtualization and heterogeneous scheduling,” but English first layer should avoid “heterogeneous” as a headline)

Second layer: When users discuss mixed GPUs or workload diversity, introduce “heterogeneous” to confirm capability boundaries—never as the opening hook

Core anchor: Maintain “HAMi (project and community) ≠ company products” as the non-negotiable baseline for long-term positioning

The Right Commercialization Landing

DaoCloud’s case study already set vendor-agnostic and CNCF toolchain compatibility as hard constraints, framing vendor dependency reduction as a business and operational benefit—not just a technical detail. Project-HAMi’s official documentation lists “avoid vendor lock” as a core value proposition.

In this context, the right commercialization landing isn’t “closed-source scheduling”—it’s productizing capabilities around real enterprise complexity:

Systematic compatibility matrix
SLO and tail-latency governance
Metering for billing
RBAC, quotas, multi-cluster governance
Upgrade and rollback safety
Faster path-to-production for DRA/CDI and other standardization efforts

Forward View: The Next 2–3 Years

My strong judgment: over the next 2–3 years, GPU scheduling competition will shift from “whose implementation is more black-box” to “whose contract is more open.”

The reasons are practical:

Hardware Form Factors and Supply Chains Are Diversifying

OpenAI’s February 12, 2026 “GPT‑5.3‑Codex‑Spark” release emphasizes ultra-low latency serving, including persistent WebSockets and a dedicated serving tier on Cerebras hardware
Large-scale GPU-backed financing announcements (for pan-European deployments) illustrate the infrastructure scale and financial engineering surrounding accelerator fleets

These signals suggest that heterogeneity will grow: mixed accelerators, mixed clouds, mixed workload types.

Low-Latency Inference Tiers Will Force Systematic Scheduling

Low-latency inference tiers (beyond just GPUs) will force resource scheduling toward “multi-accelerator, multi-layer cache, multi-class node” architectural design—scheduling must inherently be heterogeneous.

Open Scheduling Is Risk Management, Not Idealism

In this world, “open scheduling” isn’t idealism—it’s risk management. Building schedulable governable “control plane + data plane” combinations around DRA/CDI and other solidifying open interfaces, ones that are pluggable, multi-tenant governable, and co-evolvable with the ecosystem—this looks like the truly sustainable path for AI Native Infrastructure.

The next battleground isn’t “whose scheduling is smarter”—it’s “who can standardize device resource contracts into something governable.”

Conclusion

When you place HAMi 2025 back in the broader AI Native Infrastructure context, it’s no longer just the year of “GPU sharing tools”—it’s a more structural signal: GPUs are moving toward open scheduling.

Figure 7: Open Scheduling Future Vision

The driving forces come from both ends:

Upstream: Standards like DRA/CDI continue to solidify
Downstream: Scale and diversity (multi-cloud, multi-model, even accelerators beyond GPUs)

For Dynamia, HAMi’s significance has transcended “GPU sharing tool”: it turns GPU virtualization and slicing into declarative, schedulable, measurable data planes—letting queues, quotas, priorities, and multi-tenancy actually close the governance loop.

AI Learning Resources: 44 Curated Collections from Our Cleanup

Jimmy Song — Sun, 08 Feb 2026 12:20:05 +0000

“The best way to learn AI is to start building. These resources will guide your journey.”

Figure 1: AI Learning Resources Collection

In my ongoing effort to keep the AI Resources list focused on production-ready tools and frameworks, I’ve removed 44 collection-type projects—courses, tutorials, awesome lists, and cookbooks.

These resources aren’t gone—they’ve been moved here. This post is a curated collection of those educational materials, organized by type and topic. Whether you’re a complete beginner or an experienced practitioner, you’ll find something valuable here.

Why Remove Collections from AI Resources?

My AI Resources list now focuses on concrete tools and frameworks—projects you can directly use in production. Collections, while valuable, serve a different purpose: education and discovery.

By separating them, I:

Keep the resources list actionable and focused
Create a dedicated space for learning materials
Make it easier to find what you need

📚 Awesome Lists (14 Collections)

Awesome lists are community-curated collections of the best resources. They’re perfect for discovering new tools and staying updated.

Must-Explore Awesome Lists

Awesome Generative AI

Models, tools, tutorials, and research papers
Great for: Comprehensive overview of generative AI landscape

Awesome LLM

LLM resources: papers, tools, datasets, applications
Great for: Deep dive into large language models

Awesome AI Apps

Practical LLM applications, RAG examples, agent implementations
Great for: Real-world implementation examples

Awesome Claude Code

Claude Code commands, files, and workflows
Great for: Maximizing Claude Code productivity

Awesome MCP Servers

MCP servers for modular AI backend systems
Great for: Building with Model Context Protocol

Specialized Awesome Lists

Awesome ChatGPT Prompts - Prompt examples for various scenarios
Awesome LLM Apps - LLM applications with code examples
Awesome Multimodal LLM - Multimodal model resources
Awesome MCP Clients - MCP client tools and SDKs
Awesome Claude Skills - Claude Skills and workflows
Awesome GitHub Copilot - Copilot customizations
Awesome Nano Banana Pro - Image model prompts and examples
Awesome SaaS - AI platform templates
Awesome Claude Code Subagents - Claude Code subagents

🎓 Courses & Tutorials (9 Curricula)

Structured learning paths from universities and tech companies.

Microsoft’s AI Curriculum

AI for Beginners

12 weeks, 24 lessons covering neural networks, deep learning, CV, NLP
Great for: Complete AI foundation
Format: Lessons, quizzes, projects

Machine Learning for Beginners

12-week, 26-lesson curriculum on classic ML
Great for: ML fundamentals without deep math
Format: Project-based exercises

Generative AI for Beginners

18 lessons on building GenAI applications
Great for: Practical GenAI development
Format: Hands-on projects

AI Agents for Beginners

11 lessons on agent systems
Great for: Understanding autonomous agents
Format: Project-driven learning

EdgeAI for Beginners

Optimization, deployment, and real-world Edge AI
Great for: On-device AI applications
Format: Practical tutorials

MCP for Beginners

Model Context Protocol curriculum
Great for: Building with MCP
Format: Cross-language examples and labs

Official Platform Courses

Hugging Face Learn Center

Free courses on LLMs, deep RL, CV, audio
Great for: Hands-on Hugging Face ecosystem
Format: Interactive notebooks

OpenAI Cookbook

Runnable examples using OpenAI API
Great for: OpenAI API best practices
Format: Code examples and guides

PyTorch Tutorials

Basics to advanced deep learning
Great for: PyTorch mastery
Format: Comprehensive tutorials

🍳 Cookbooks & Example Collections (5 Collections)

Practical code examples and recipes.

Claude Cookbooks

Notebooks and examples for building with Claude
Great for: Anthropic Claude integration
Format: Jupyter notebooks

Hugging Face Cookbook

Practical AI cookbook with Jupyter notebooks
Great for: Open models and tools
Format: Hands-on examples

Tinker Cookbook

Training and fine-tuning examples
Great for: Fine-tuning workflows
Format: Platform-specific recipes

E2B Cookbook

Examples for building LLM apps
Great for: LLM application development
Format: Recipes and tutorials

arXiv Paper Curator

6-week course on RAG systems
Great for: Production-ready RAG
Format: Project-based learning

📖 Guides & Handbooks (5 Resources)

In-depth guides on specific topics.

Prompt Engineering Guide

Comprehensive prompt engineering resources
Great for: Mastering prompt design
Format: Guides, papers, lectures, notebooks

Evaluation Guidebook

LLM evaluation best practices from Hugging Face
Great for: Assessing LLM performance
Format: Practical guide

Context Engineering

Design and optimize context beyond prompt engineering
Great for: Advanced context management
Format: Practical handbook

Context Engineering Intro

Template and guide for context engineering
Great for: Providing project context to AI assistants
Format: Template + guide

Vibe-Coding Workflow

5-step prompt template for building MVPs with LLMs
Great for: Rapid prototyping with AI
Format: Workflow template

🗂️ Template & Workflow Collections

Reusable templates and workflows.

Claude Code Templates

Code templates for various programming scenarios
Great for: Claude AI development
Format: Template collection

n8n Workflows

2,000+ professionally organized n8n workflows
Great for: Workflow automation
Format: Searchable catalog

N8N Workflows Catalog

Community-driven reusable workflow templates
Great for: Workflow import and versioning
Format: Template catalog

📊 Research & Evaluation

Academic and evaluation resources.

LLMSys PaperList

Curated list of LLM systems papers
Great for: Research on training, inference, serving
Format: Paper collection

Free LLM API Resources

LLM providers with free/trial API access
Great for: Experimentation without cost
Format: Provider list

🎨 Other Notable Resources

System Prompts and Models of AI Tools

Community-curated collection of system prompts and AI tool examples
Great for: Prompt and agent engineering
Format: Resource collection

ML Course CS-433

EPFL Machine Learning Course
Great for: Academic ML foundation
Format: Lectures, labs, projects

Machine Learning Engineering

ML engineering open-book: compute, storage, networking
Great for: Production ML systems
Format: Comprehensive guide

Realtime Phone Agents Course

Build low-latency voice agents
Great for: Voice AI applications
Format: Hands-on course

LLMs from Scratch

Build a working LLM from first principles
Great for: Understanding LLM internals
Format: Repository + book materials

💡 How to Use This Collection

For Complete Beginners

Start with: Microsoft’s AI for Beginners
Practice with: PyTorch Tutorials
Explore: Awesome AI Apps for inspiration

For Developers

Build skills: OpenAI Cookbook + Claude Cookbooks
Find tools: Awesome Generative AI + Awesome LLM
Learn workflows: n8n Workflows Catalog

For Researchers

Stay updated: Awesome Generative AI + LLMSys PaperList
Deep dive: Awesome LLM
Implement: Hugging Face Cookbook

For Product Builders

Find examples: Awesome AI Apps
Learn workflows: n8n Workflows Catalog
Study patterns: Awesome LLM Apps

🔄 What Was NOT Removed

Agent frameworks and production tools remain in the AI Resources list, including:

AutoGen - Microsoft’s multi-agent framework
CrewAI - High-performance multi-agent orchestration
LangGraph - Stateful multi-agent applications
Flowise - Visual agent platform
Langflow - Visual workflow builder
And 80+ more agent frameworks

These are functional tools you can use to build applications, not educational collections. They belong in the AI Resources list.

📝 Summary

I removed 44 collection-type projects from the AI Resources list to keep it focused on production tools:

14 Awesome Lists - Discover new tools and stay updated
9 Courses & Tutorials - Structured learning paths
5 Cookbooks - Practical code examples
5 Guides & Handbooks - In-depth resources
4 Template Collections - Reusable workflows
7 Other Resources - Research and evaluation

These resources remain incredibly valuable for learning and discovery. They just serve a different purpose than the production-focused tools in my AI Resources list.

Next Steps:

Bookmark this post for future reference
Explore the AI Resources list for production tools (agent frameworks, databases, etc.)
Check out my blog for more AI engineering insights

Acknowledgments: This collection was compiled during my AI Resources cleanup initiative. Special thanks to all the maintainers of these awesome lists, courses, and collections for their invaluable contributions to the AI community.

Standing on Giants' Shoulders: The Traditional Infrastructure Powering Modern AI

Jimmy Song — Sun, 08 Feb 2026 08:00:00 +0000

“If I have seen further, it is by standing on the shoulders of giants.” — Isaac Newton

Figure 1: Standing on Giants’ Shoulders: The Traditional Infrastructure Powering Modern AI

In the excitement surrounding LLMs, vector databases, and AI agents, it’s easy to forget that modern AI didn’t emerge from a vacuum. Today’s AI revolution stands upon decades of infrastructure work—distributed systems, data pipelines, search engines, and orchestration platforms that were built long before “AI Native” became a buzzword.

This post is a tribute to those traditional open source projects that became the invisible foundation of AI infrastructure. They’re not “AI projects” per se, but without them, the AI revolution as we know it wouldn’t exist.

The Evolution: From Big Data to AI

Era	Focus	Core Technologies	AI Connection
2000s	Web Search & Indexing	Lucene, Elasticsearch	Semantic search foundations
2010s	Big Data & Distributed Computing	Hadoop, Spark, Kafka	Data processing at scale
2010s	Cloud Native	Docker, Kubernetes	Model deployment platforms
2010s	Stream Processing	Flink, Storm, Pulsar	Real-time ML inference
2020s	AI Native	Transformers, Vector DBs	Built on everything above

Table 1: Evolution of Data Infrastructure

Big Data Frameworks: The Data Engines

Before we could train models on petabytes of data, we needed ways to store, process, and move that data.

Apache Hadoop (2006)

GitHub: https://github.com/apache/hadoop

Hadoop democratized big data by making distributed computing accessible. Its HDFS filesystem and MapReduce paradigm proved that commodity hardware could process web-scale datasets.

Why it matters for AI:

Modern ML training datasets live in HDFS-compatible storage
Data lakes built on Hadoop became training data reservoirs
Proved that distributed computing could scale horizontally

Apache Kafka (2011)

GitHub: https://github.com/apache/kafka

Kafka redefined data streaming with its log-based architecture. It became the nervous system for real-time data flows in enterprises worldwide.

Why it matters for AI:

Real-time feature pipelines for ML models
Event-driven architectures for AI agent systems
Streaming inference pipelines
Model telemetry and monitoring backbones

Apache Spark (2014)

GitHub: https://github.com/apache/spark

Spark brought in-memory computing to big data, making iterative algorithms (like ML training) practical at scale.

Why it matters for AI:

MLlib made ML accessible to data engineers
Distributed data processing for model training
Spark ML became the de facto standard for big data ML
Proved that in-memory computing could accelerate ML workloads

Search Engines: The Retrieval Foundation

Before RAG (Retrieval-Augmented Generation) became a buzzword, search engines were solving retrieval at scale.

Elasticsearch (2010)

GitHub: https://github.com/elastic/elasticsearch

Elasticsearch made full-text search accessible and scalable. Its distributed architecture and RESTful API became the standard for search.

Why it matters for AI:

pioneered distributed inverted index structures
Proved that horizontal scaling was possible for search workloads
Many “AI search” systems actually use Elasticsearch under the hood
Query DSL influenced modern vector database query languages

OpenSearch (2021)

GitHub: https://github.com/opensearch-project/opensearch

When AWS forked Elasticsearch, it ensured search infrastructure remained truly open. OpenSearch continues the mission of accessible, scalable search.

Why it matters for AI:

Maintains open source innovation in search
Vector search capabilities added in 2023
Demonstrates community fork resilience

Databases: From SQL to Vectors

The evolution from relational databases to vector databases represents a paradigm shift—but both have AI relevance.

Traditional Databases That Paved the Way

Dgraph (2015) - Graph database proving that specialized data structures enable new use cases
TDengine (2019) - Time-series database for IoT ML workloads
OceanBase (2021) - Distributed database showing ACID transactions could scale

Why they matter for AI:

Proved that specialized database engines could outperform general-purpose ones
Database internals (indexing, sharding, replication) are now applied to vector databases
Multi-model databases (graph + vector + relational) are becoming the norm for AI apps

Cloud Native: The Runtime Foundation

When Docker and Kubernetes emerged, they weren’t built for AI—but AI couldn’t scale without them.

Docker (2013) & Kubernetes (2014)

GitHub: https://github.com/kubernetes/kubernetes

Kubernetes became the operating system for cloud-native applications. Its declarative API and controller pattern made it perfect for AI workloads.

Why it matters for AI:

Model deployment platforms (KServe, Seldon Core) run on K8s
GPU orchestration (NVIDIA GPU Operator, Volcano, HAMi) extends K8s
Kubeflow made K8s the standard for ML pipelines
Microservice patterns enable modular AI agent architectures

Service Mesh & Serverless

Istio (2016), Knative (2018) - Service mesh and serverless platforms that proved:

Network-level observability applies to AI model calls
Scale-to-zero is essential for cost-effective inference
Traffic splitting enables A/B testing of ML models

Why they matter for AI:

AI Gateway patterns evolved from API gateways + service mesh
Serverless inference platforms use Knative-style autoscaling
Observability patterns (tracing, metrics) are now standard for ML systems

API Gateways: From REST to LLM

API gateways weren’t designed for AI, but they became the foundation of AI Gateway patterns.

Kong, APISIX, KGateway

These API gateways solved rate limiting, auth, and routing at scale. When LLMs emerged, the same patterns applied:

AI Gateway Evolution:

Traditional API Gateway (2010s)
 ↓
Rate Limiting → Token Bucket Rate Limiting
Auth → API Key + Organization Management
Routing → Model Routing (GPT-4 → Claude → Local Models)
Observability → LLM-specific Telemetry (token usage, cost)
 ↓
AI Gateway (2024)

Why they matter for AI:

Proved that centralized API management scales
Plugin architectures enable LLM-specific features
Traffic management patterns apply to prompt routing
Security patterns (mTLS, JWT) now protect AI endpoints

Workflow Orchestration: The Pipeline Backbone

Data engineering needs pipelines. ML engineering needs pipelines. AI agents need workflows.

Apache Airflow (2015)

GitHub: https://github.com/apache/airflow

Airflow made pipeline orchestration accessible with its DAG-based approach. It became the standard for ETL and data engineering.

Why it matters for AI:

ML pipeline orchestration (feature engineering, training, evaluation)
Proved that DAG-based workflow definition works at scale
Prompt engineering pipelines use Airflow-style orchestration
Scheduler patterns are now applied to AI agent workflows

n8n, Prefect, Flyte

Modern workflow platforms that evolved from Airflow’s foundations:

n8n (2019) - Visual workflow automation with AI capabilities
Prefect (2018) - Python-native workflow orchestration for ML
Flyte (2019) - Kubernetes-native workflow orchestration for ML/data

Why they matter for AI:

Multi-modal agents need workflow orchestration
RAG pipelines are essentially ETL pipelines for embeddings
Prompt chaining is DAG-based orchestration

Data Formats: The Lakehouse Foundation

Before we could train on massive datasets, we needed formats that supported ACID transactions and schema evolution.

Delta Lake, Apache Iceberg, Apache Hudi

These table formats brought reliability to data lakes:

Why they matter for AI:

Training datasets need versioning and reproducibility
Feature stores use Delta/Iceberg as storage formats
Proved that “big data” could have transactional semantics
Schema evolution handles ML feature drift

The Invisible Thread: Why These Projects Matter

What do all these projects have in common?

They solved scaling first - AI training/inference needs horizontal scaling
They proved distributed systems work - Modern AI is fundamentally distributed
They created ecosystem patterns - Plugin systems, extension points, APIs
They established best practices - Observability, security, CI/CD
They built developer habits - YAML configs, declarative APIs, CLI tools

The AI Native Continuum

Modern “AI Native” infrastructure didn’t replace these projects—it builds on them:

Traditional Project	AI Native Evolution	Example
Hadoop HDFS	Distributed model storage	HDFS for datasets, S3 for checkpoints
Kafka	Real-time feature pipelines	Kafka → Feature Store → Model Serving
Spark ML	Distributed ML training	MLlib → PyTorch Distributed
Elasticsearch	Vector search	ES → Weaviate/Qdrant/Milvus
Kubernetes	ML orchestration	K8s → Kubeflow/KServe
Istio	AI Gateway service mesh	Istio → LLM Gateway with mTLS
Airflow	ML pipeline orchestration	Airflow → Prefect/Flyte for ML

Table 2: From Traditional to AI Native

Why We’re Removing Them from AI Resources List

This post honors these projects, but we’re also removing them from our AI Resources list. Here’s why:

They’re not “AI Projects”—they’re foundational infrastructure.

Hadoop, Kafka, Spark are data engineering tools, not ML frameworks
Elasticsearch is search, not semantic search
Kubernetes is general-purpose orchestration
API gateways serve REST/GraphQL, not just LLMs

But their absence doesn’t diminish their importance.

By removing them, we acknowledge that:

AI has its own ecosystem - Transformers, vector DBs, LLM ops
Traditional infra has its own domain - Data engineering, cloud native
The intersection is where innovation happens - AI-native data platforms, LLM ops on K8s

The Giants We Stand On

The next time you:

Deploy a model on Kubernetes
Stream features through Kafka
Search embeddings with a vector database
Orchestrate a RAG pipeline with Prefect

Remember: You’re standing on the shoulders of Hadoop, Kafka, Elasticsearch, Kubernetes, and countless others. They built the roads we now drive on.

The Future: Building New Giants

Just as Hadoop and Kafka enabled modern AI, today’s AI infrastructure will become tomorrow’s foundation:

Vector databases may become the new standard for all search
LLM observability may evolve into general distributed tracing
AI agent orchestration may reinvent workflow automation
GPU scheduling may influence general-purpose resource management

The cycle continues. The giants of today will be the foundations of tomorrow.

Conclusion: Gratitude and Continuity

As we clean up our AI Resources list to focus on AI-native projects, we don’t forget where we came from. Traditional big data and cloud native infrastructure made the AI revolution possible.

To the Hadoop committers, Kafka maintainers, Kubernetes contributors, and all who built the foundation: Thank you.

Your work enabled ChatGPT, enabled Transformers, enabled everything we now call “AI.”

Standing on your shoulders, we see further.

Acknowledgments: This post was inspired by the need to refactor our AI Resources list. The 27 projects mentioned here are being removed—not because they’re unimportant, but because they deserve their own category: The Foundation.

My First Month at Dynamia: Why AI Native Infra Is Worth the Investment

Jimmy Song — Fri, 06 Feb 2026 12:56:35 +0000

Time flies—it’s already been a month since I joined Dynamia. In this article, I want to share my observations from this past month: why AI Native Infra is a direction worth investing in, and some considerations for those thinking about their own career or technical direction.

Introduction

After nearly five years of remote work, I officially joined Dynamia last month as VP of Open Source Ecosystem. This decision was not sudden, but a natural extension of my journey from cloud native to AI Native Infra.

But this article is not just about my personal choice. I want to answer a more universal question: In the wave of AI infrastructure startups, why is compute governance a direction worth investing in?

For the past decade, I have worked continuously in the infrastructure space: from Kubernetes to Service Mesh, and now to AI Infra. I am increasingly convinced that the core challenge in the AI era is not “can the model run,” but “can compute resources be run efficiently, reliably, and in a controlled manner.” This conviction has only grown stronger through my observations and reflections during this first month at Dynamia.

This article answers three questions: What is AI Native Infra? Why is GPU virtualization a necessity? Why did I choose Dynamia and HAMi?

What Is AI Native Infra

The core of AI Native Infrastructure is not about adding another platform layer, but about redefining the governance target: expanding from “services and containers” to “model behaviors and compute assets.”

I summarize it as three key shifts:

Models as execution entities: Governance now includes not just processes, but also model behaviors.
Compute as a scarce asset: GPU, memory, and bandwidth must be scheduled and metered precisely.
Uncertainty as the default: Systems must remain observable and recoverable amid fluctuations.

In essence, AI Native Infra is about upgrading compute governance from “resource allocation” to “sustainable business capability.”

Why GPU Virtualization Is Essential

Many teams focus on model inference optimization, but in production, enterprises first encounter the problem of “underutilized GPUs.” This is where GPU virtualization delivers value.

Structural idleness: Small tasks monopolize large GPUs, leaving them idle for long periods.
Pseudo-isolation risks: Native sharing lacks hard boundaries, so a single task OOM can cause cascading failures.
Scheduling failures: Some users queue for GPUs while others occupy but do not use them, leading to both shortages and idleness.
Fragmentation waste: There may be enough total GPU, but not enough full cards, making efficient packing impossible.
Vendor lock-in anxiety: Proprietary, tightly coupled solutions make migration costs uncontrollable.

In short: GPUs must not only be allocatable, but also splittable, isolatable, schedulable, and governable.

The Relationship Between HAMi and Dynamia

This is the most frequently asked question. Here is the shortest answer:

HAMi: A CNCF-hosted open source project and community focused on GPU virtualization and heterogeneous compute scheduling.
Dynamia: The founding and leading company behind HAMi, providing enterprise-grade products and services based on HAMi.

Open source projects are not the same as company products, but the two evolve together. HAMi drives industry adoption and technical trust, while Dynamia brings these capabilities into enterprise production environments at scale. This “dual engine” approach is what makes Dynamia unique.

What HAMi Provides

HAMi (Heterogeneous AI Computing Virtualization Middleware) delivers three key capabilities on Kubernetes:

Virtualization and partitioning: Split physical GPUs into logical resources on demand to improve utilization.
Scheduling and topology awareness: Place workloads optimally based on topology to reduce communication bottlenecks.
Isolation and observability: Support quotas, policies, and monitoring to reduce production risks.

Currently, HAMi has attracted over 360 contributors from 16 countries, with more than 200 enterprise end users, and its international influence continues to grow.

Market Trends: The AI Infrastructure Startup Wave

AI infrastructure is experiencing a new wave of startups. The vLLM team’s company raised $150 million, SGLang’s commercial spin-off RadixArk is valued at $4 billion, and Databricks acquired MosaicML for $1.3 billion—all pointing to a consensus: Whoever helps enterprises run large models more efficiently and cost-effectively will hold the keys to next-generation AI infrastructure.

Against this backdrop, the positioning of Dynamia and HAMi is even clearer. Many teams focus on “model performance acceleration” and “inference optimization” (like vLLM, SGLang), while we focus on “resource scheduling and virtualization”—enabling better orchestration of existing accelerated hardware resources.

The two are complementary: the former makes individual models run faster and cheaper, while the latter ensures that compute allocation at the cluster level is efficient, fair, and controllable. This is similar to extending Kubernetes’ CPU/memory scheduling philosophy to GPU and heterogeneous compute management in the AI era.

Why AI Native Infra Is Worth the Investment

My observations this month have convinced me that compute governance is the most undervalued yet most promising area in AI infrastructure. If you are considering a career or technical investment, here is my assessment:

First, this is a real and urgent pain point

Model training and inference optimization attract a lot of attention, but in production, enterprises first encounter the problem of “underutilized GPUs”—structural idleness, scheduling failures, fragmentation waste, and vendor lock-in anxiety. Without solving these problems, even the fastest models cannot scale in production. GPU virtualization and heterogeneous compute scheduling are the “infrastructure below infrastructure” for enterprise AI transformation.

Second, this is a clear long-term track

Frameworks like vLLM and SGLang emerge constantly, making individual models run faster. But who ensures that compute allocation at the cluster level is efficient, fair, and controllable? This is similar to extending Kubernetes’ success in CPU/memory scheduling to GPU and heterogeneous compute management in the AI era. This is not something that can be finished in a year or two, but a direction for continuous construction over the next five to ten years.

Third, this is an open and verifiable path

Dynamia chose to build on HAMi as an open source foundation, first solving general capabilities, then supporting enterprise adoption. This means the technical direction is transparent and verifiable in the community. You can form your own judgment by participating in open source, observing adoption, and evaluating the ecosystem—rather than relying on the black-box promises of proprietary solutions.

Fourth, this is a window of opportunity that is opening now

AI infrastructure is being redefined. Investing in its construction today will continue to yield value in the coming years. The vLLM team’s company raised $150 million, SGLang’s commercial spin-off RadixArk is valued at $4 billion, Databricks acquired MosaicML for $1.3 billion—all validating the same trend: Whoever helps enterprises run large models more efficiently will hold the keys to next-generation AI infrastructure.

I hope to bring my experience in cloud native and open source communities to the next stage of HAMi and Dynamia: turning GPU resources from a “cost center” into an “operational asset.” This is not just my career choice, but my judgment and investment in the direction of next-generation infrastructure.

Join the HAMi Community

Add me on WeChat (jimmysong) to join the HAMi community focused on GPU virtualization and heterogeneous compute scheduling.

If you are also interested in HAMi, GPU virtualization, AI Native Infra, or Dynamia, feel free to reach out.

Summary

From cloud native to AI Native Infra, my observations this month have only strengthened my conviction: The true upper limit of AI applications is determined by the infrastructure’s ability to govern compute resources.

HAMi addresses the fundamental issues of GPU virtualization and heterogeneous compute scheduling, while Dynamia is driving these capabilities into large-scale production. If you are also looking for a technical direction worth long-term investment, AI Native Infra—especially compute governance and scheduling—is a track with real pain points, a clear path, an open ecosystem, and an opening window of opportunity.

Joining Dynamia is not just a career choice, but a commitment to building the next generation of infrastructure. I hope the observations and reflections in this article can provide some reference for you as you evaluate technical directions and career opportunities.

If you are also interested in HAMi, GPU virtualization, AI Native Infra, or Dynamia, feel free to reach out.

The True Inflection Point of ADD: When Spec Becomes the Core Asset of AI-Era Software

Jimmy Song — Tue, 20 Jan 2026 07:51:36 +0000

The role of Spec is undergoing a fundamental transformation, becoming the governance anchor of engineering systems in the AI era.

The Essence of Software Engineering and the Cost Structure Shift Brought by AI

From first principles, software engineering has always been about one thing: stably, controllably, and reproducibly transforming human intent into executable systems.

Artificial Intelligence (AI) does not change this engineering essence, but it dramatically alters the cost structure:

Implementation costs plummet: Code, tests, and boilerplate logic are rapidly commoditized.
Consistency costs rise sharply: Intent drift, hidden conflicts, and cross-module inconsistencies become more frequent.
Governance costs are amplified: As agents can act directly, auditability, accountability, and explainability become hard constraints.

Therefore, in the era of Agent-Driven Development (ADD), the core issue is not “can agents do the work,” but how to maintain controllability and intent preservation in engineering systems under highly autonomous agents.

The ADD Era Inflection Point: Three Structural Preconditions

Many attribute the “explosion” of ADD to more mature multi-agent systems, stronger models, or more automated tools. In reality, the true structural inflection point arises only when these three conditions are met:

Agents have acquired multi-step execution capabilities

With frameworks like LangChain, LangGraph, and CrewAI, agents are no longer just prompt invocations, but long-lived entities capable of planning, decomposition, execution, and rollback.

Agents are entering real enterprise delivery pipelines

Once in enterprise R&D, the question shifts from “can it generate” to “who approved it, is it compliant, can it be rolled back.”

Traditional engineering tools lack a control plane for the agent era

Tools like Git, CI, and Issue Trackers were designed for “human developer collaboration,” not for “agent execution.”

When these three factors converge, ADD inevitably shifts from an “efficiency tool” to a “governance system.”

The Changing Role of Spec: From Documentation to System Constraint

In the context of ADD, Spec is undergoing a fundamental shift:

Spec is no longer “documentation for humans,” but “the source of constraints and facts for systems and agents to execute.”

Spec now serves at least three roles:

Verifiable expression of intent and boundaries

Requirements, acceptance criteria, and design principles are no longer just text, but objects that can be checked, aligned, and traced.

Stable contracts for organizational collaboration

When agents participate in delivery, verbal consensus and tacit knowledge quickly fail. Versioned, auditable artifacts become the foundation of collaboration.

Policy surface for agent execution

Agents can write code, modify configurations, and trigger pipelines. Spec must become the constraint on “what can and cannot be done.”

From this perspective, the status of Spec is approaching that of the Control Plane in AI-native infrastructure.

The Reality of Multi-Agent Workflows: Orchestration and Governance First

In recent systems (such as APOX and other enterprise products), an industry consensus is emerging:

Multi-agent collaboration no longer pursues “full automation,” but is staged and gated.
Frameworks like LangGraph are used to build persistent, debuggable agent workflows.
RAG (e.g., based on Milvus) is used to accumulate historical Specs, decisions, and context as long-term memory.
The IDE mainly focuses on execution efficiency, not engineering governance.

Figure 1: APOX user interface

APOX (AI Product Orchestration eXtended) is a multi-agent collaboration workflow platform for enterprise software delivery. Its core goals are:

To connect the entire process from product requirements to executable code with a governable Agentflow and explicit engineering artifact chain.
To assign dedicated AI agents to each delivery stage (such as PRD, PO, Architecture, Developer, Implementation, Coding, etc.).
To embed manual approval gates and full audit trails at every step, solving the “intent drift and consistency” governance problem that traditional AI coding tools cannot address.
The platform provides a VS Code plugin for real-time sync between local IDE and web artifacts, allowing Specs, code, tasks, and approval statuses to coexist in the repository.
Supports assigning different base models to different agents according to enterprise needs.

APOX is not about simply speeding up code generation, but about elevating “Spec” from auxiliary documentation to a verifiable, constrainable, and traceable core asset in engineering—building a control plane and workflow governance system suitable for Agent-Driven Development.

Such systems emphasize:

An explicit artifact chain from PRD → Spec → Task → Implementation.
Manual confirmation and audit points at every stage.
Bidirectional sync between Spec, code, repository, and IDE.

This is not about “smarter AI,” but about engineering systems adapting to the agent era.

The Long-Term Value of Spec: The Core Anchor of Engineering Assets

This is not to devalue code, but to acknowledge reality:

There will always be long-term differentiation in algorithms and model capabilities.
General engineering implementation is rapidly homogenizing.
What is hard to replicate is: how to define problems, constrain systems, and govern change.

In the ADD era, the value of Spec is reflected in:

Determining what agents can and cannot do.
Carrying the organization’s long-term understanding of the system.
Serving as the anchor for audit, compliance, and accountability.

Code will be rewritten again and again; Spec is the long-term asset.

Risks and Challenges of ADD: Living Spec and Governance Constraints

ADD also faces significant risks:

Can Spec become a Living Spec

That is, when key implementation changes occur, can the system detect “intent changes” and prompt Spec updates, rather than allowing silent drift?

Can governance achieve low friction but strong constraints

If gates are too strict, teams will bypass them; if too loose, the system loses control.

These two factors determine whether ADD is “the next engineering paradigm” or “just another tool bubble.”

The Trend Toward Control Planes in Engineering Systems

From a broader perspective, ADD is the inevitable result of engineering systems becoming “control planes”:

Engineering systems are evolving from “human collaboration tools” to “control systems for agent execution.”

In this structure:

Agent / IDE is the execution plane.
RAG / Memory is the state and memory plane.
Spec is the intent and policy plane.
Gates, audit, and traceability form the governance loop.

This closely aligns with the evolution path of AI-native infrastructure.

Summary

The winners of the ADD era will not be the systems with “the most agents or the fastest generation,” but those that first upgrade Spec from documentation to a governable, auditable, and executable asset. As automation advances, the true scarcity is the long-term control of intent.

AI Voice Dictation Input Methods Are Becoming the New Shortcut Key for the Programming Era

Jimmy Song — Sun, 18 Jan 2026 06:53:08 +0000

Voice input methods are not just about being “fast”—they are becoming a brand new gateway for developers to collaborate with AI.

Warning

On January 12, 2026, due to financial difficulties encountered during operations, the Miaoyan project announced the cessation of operations and the team was disbanded. The application will no longer be updated or maintained, but existing versions can continue to be used on the current device and system, and do not store any audio or transcription content.

Figure 1: Can voice input become the new shortcut for developers? My in-depth comparison experience.

AI Voice Input Methods Are Becoming the “New Shortcut Key” in the Programming Era

I am increasingly convinced of one thing: PC-based AI voice input methods are evolving from mere “input tools” into the foundational interaction layer for the era of programming and AI collaboration.

It’s not just about typing faster—it determines how you deliver your intent to the system, whether you’re writing documentation, code, or collaborating with AI in IDEs, terminals, or chat windows.

Because of this, the differences in voice input method experiences are far more significant than they appear on the surface.

My Six Evaluation Criteria for AI Voice Input Methods

After long-term, high-frequency use, I have developed a set of criteria to assess the real-world performance of AI voice input methods:

Response speed: Does text appear quickly enough after pressing the shortcut to keep up with your thoughts?
Continuous input stability: Does it remain reliable during extended use, or does it suddenly fail or miss recognition?
Mixed Chinese-English and technical terms: Can it reliably handle code, paths, abbreviations, and product names?
Developer friendliness: Is it truly designed for command line, IDE, and automation scenarios?
Interaction restraint: Does it avoid introducing distracting features that interfere with input itself?
Subscription and cost structure: Is it a standalone paid product, or can it be bundled with existing tool subscriptions?

Based on these criteria, I focused on comparing Miaoyan, Shandianshuo, and Zhipu AI Voice Input Method.

Miaoyan: Currently the Most “Developer-Oriented” Domestic Product

Miaoyan was the first domestic AI voice input method I used extensively, and it remains the one I am most willing to use continuously.

Figure 2: Miaoyan is currently my most-used Mac voice input method.

Command Mode: The Key Differentiator for Developer Productivity

It’s important to clarify that Miaoyan’s command mode is not about editing text via voice. Instead:

You describe your need in natural language, and the system directly generates an executable command-line command.

This is crucial for developers:

It’s not just about input
It’s about turning voice into an automation entry point
Essentially, it connects voice to the CLI or toolchain

This design is clearly focused on engineering efficiency, not office document polishing.

Usage Experience Summary

Fast response, nearly instant
Output is relatively clean, with minimal guessing
Interaction design is restrained, with no unnecessary concepts
Developer-friendly mindset

But there are some practical limitations:

It is a completely standalone product
Requires a separate subscription
Still in relatively small-scale use

From a product strategy perspective, it feels more like a “pure tool” than part of an ecosystem.

Note

Shandianshuo: Local-First Approach, Developer Experience Depends on Your Setup

Shandianshuo takes a different approach: it treats voice input as a “local-first foundational capability,” emphasizing low latency and privacy (at least in its product narrative). The natural advantages of this approach are speed and controllable marginal costs, making it suitable as a “system capability” that’s always available, rather than a cloud service.

Figure 3: Shandianshuo settings page

However, from a developer’s perspective, its upper limit often depends on “how you implement enhanced capabilities”:

If you only use it for basic transcription, the experience is more like a high-quality local input tool. But if you want better mixed Chinese-English input, technical term correction, symbol and formatting handling, the common approach is to add optional AI correction/enhancement capabilities, which usually requires extra configuration (such as providing your own API key or subscribing to enhanced features). The key trade-off here is not “can it be used,” but “how much configuration cost are you willing to pay for enhanced capabilities.”

If you want voice input to be a “lightweight, stable, non-intrusive” foundation, Shandianshuo is worth considering. But if your goal is to make voice input part of your developer workflow (such as command generation or executable actions), it needs to offer stronger productized design at the “command layer” and in terms of controllability.

Zhipu AI Voice Input Method: Stable but with Friction

I also thoroughly tested the Zhipu AI Voice Input Method.

Figure 4: Zhipu Voice Input Method settings interface

Its strengths include:

More stable for long-term continuous input
Rarely becomes completely unresponsive
Good tolerance for longer Chinese input

But with frequent use, some issues stand out:

Idle misrecognition: If you press the shortcut but don’t speak, it may output random characters, disrupting your input flow
Occasionally messy output: Sometimes adds irrelevant words, making it less controllable than Miaoyan
Basic recognition errors: For example, “Zhipu” being recognized as “Zhipu” (with a different character), which is a trust issue for professional users
Feature-heavy design: Various tone and style features increase cognitive load

Subscription Bundling: Zhipu’s Practical Advantage

Although I prefer Miaoyan in terms of experience, Zhipu has a very practical advantage:

If you already subscribe to Zhipu’s programming package, the voice input method is included for free.

This means:

No need to pay separately for the input method
Lower psychological and decision-making cost
More likely to become the “default tool” that stays

From a business perspective, this is a very smart strategy.

Main Comparison Table

The following table compares the three products across key dimensions for quick reference.

Dimension	Miaoyan	Shandianshuo	Zhipu AI Voice Input Method
Response Speed	Fast, nearly instant	Usually fast (local-first)	Slightly slower than Miaoyan
Continuous Stability	Stable	Depends on setup and environment	Very stable
Idle Misrecognition	Rare	Generally restrained (varies by version)	Obvious: outputs characters even if silent
Output Cleanliness/Control	High	More like an “input tool”	Occasionally messy
Developer Differentiator	Natural language → executable command	Local-first / optional enhancements	Ecosystem-attached capabilities
Subscription & Cost	Standalone, separate purchase	Basic usable; enhancements often require setup/subscription	Bundled free with programming package
My Current Preference	Best experience	More like a “foundation approach”	Easy to keep but not clean enough

Table 1: Core Comparison of Miaoyan, Shandianshuo, and Zhipu AI Voice Input Methods

User Loyalty to AI Voice Input Methods

The switching cost for voice input methods is actually low: just a shortcut key and a habit of output.

What really determines whether users stick around is:

Whether the output is controllable
Whether it keeps causing annoying minor issues
Whether it integrates into your existing workflow and payment structure

For me personally:

The best and smoothest experience is still Miaoyan
The one most likely to stick around is probably Zhipu
Shandianshuo is more of a “foundation approach” and worth watching for how its enhancements evolve

These points are not contradictory.

Summary

Miaoyan is more mature in engineering orientation, command capabilities, and input control
Zhipu has practical advantages in stability and subscription bundling
Shandianshuo takes a local-first + optional enhancement approach, with the key being how it balances “basic capability” and “enhancement cost”
Who truly becomes the “default gateway” depends on reducing distractions, fixing frequent minor issues, and treating voice input as true “infrastructure” rather than an add-on feature

The competition among AI voice input methods is no longer about recognition accuracy, but about who can own the shortcut key you press every day.

From Spatial Data to AI Open Source: Technical Standards, Data Sovereignty, and the Global Divide

Jimmy Song — Sun, 11 Jan 2026 03:29:28 +0000

The divide in technical standards and data sovereignty determines the global competitive landscape of infrastructure open source in the AI era.

In this article, I will use the differences in air quality data presentation in Apple Maps and Weather as a starting point to explore how technical standards and data sovereignty influence the open source paths of AI in different countries. I will further analyze why, in the AI era, infrastructure-level open source has become the key battleground for ecosystem dominance.

Author’s Note

This article originates from a very everyday observation: Why is air quality data in China shown as “points” in Apple Maps and Weather, while in other countries it is often displayed as “areas”?

Figure 1: Air quality map in Apple Weather, showing point-based data in China and area-based data in other countries

At first glance, it seems like a product experience difference. But when I reconsidered this issue in the context of engineering, standards, and system design, I realized it actually points to a much bigger question: how different countries understand the relationship between technology, standards, openness, and sovereignty.

As an engineer who has long worked in cloud native, AI infrastructure, and open source ecosystems, I gradually realized that this difference is not limited to air quality or map data. In the AI era, it is further amplified, directly affecting how we open source models, build infrastructure, and whether we can participate in the formulation of global rules.

Writing this article is not about judging right or wrong, but about using a concrete example to explain a structural difference and discuss the long-term impact and real opportunities this difference may bring in the AI era.

What is especially important: at the level of AI infrastructure and infra-level open source, the competition has just begun. China is not without opportunities, but the choice of path will become more critical than ever.

Differences in Air Quality Data Presentation: A Microcosm of Technical Standards and Sovereignty

The following image illustrates the divide between spatial data, AI open source, and technical standards. By comparing how air quality data is presented in Apple Maps and Weather in different countries, you can intuitively feel the differences in technical standards and sovereignty strategies behind the scenes.

Figure 2: The divide between spatial data, AI open source, and technical standards

If you regularly use global products such as maps, weather, traffic, or various data services, you may notice a recurring phenomenon that is rarely discussed seriously: the way data is presented in China often differs significantly from global mainstream standards.

A very intuitive example comes from the air quality display in Apple Maps or Weather. In China, air quality is usually shown as discrete points; in the US, Europe, Japan, and other countries, it is often rendered as continuous coverage areas.

At first glance, this seems like a product experience difference, and may even lead people to mistakenly believe that “China’s data is incomplete.” But if you treat it as an engineering or system design issue, you will find: this is not a matter of data capability, but a different choice in technical standards, data sovereignty, and openness strategies.

And this choice is not limited to air quality.

Air Quality Is Just a Slice: Greater Differences in Spatial Public Data

Air quality is just a highly visible and relatively low-risk example. Similar differences have long existed in broader spatial and public data domains.

Maps and coordinate systems
Surveying and high-precision spatial data
Real-time traffic and population movement
Remote sensing, environmental, and urban operation data

In global mainstream systems, such data is usually regarded as public information infrastructure. It is standardized, gridded, API-ified, allows interpolation, modeling, and redistribution, and is widely used in research, business, and product innovation.

In China, this data often takes another form: hierarchical, discrete, strictly defined, and with centralized interpretation authority.

This is not a technical preference in a single field, but a systemic logic of technology and governance.

Three Global Paths

Placing China in a global context, we can see that there are roughly three different paths worldwide regarding “how public data and technical standards are opened.”

Engineering-Open Type: Standards and Ecosystem First

Represented by the US and some European countries, the core features of this system are:

Public data prioritized as infrastructure
Standards and interfaces come first
Encourages engineering autonomy and ecosystem evolution
Tolerates model inference and uncertainty

This path directly shaped the global landscape of foundational software and infrastructure-level open source. Linux, Kubernetes, and the cloud native system are essentially products of openness at the rules layer.

Governance-Sovereignty Type: Control and Auditability First

Represented by China, this path emphasizes:

Sensitivity of spatial and public data
Data as part of governance capability
Standards, definitions, and release methods are highly bound
Emphasizes traceability, accountability, and controllability

In this system, “point data” is not a sign of technological backwardness, but a governable technical form. When a technical system is designed as a governance system, its primary goal is not reusability, but controllability.

Compromise-Coordinated Type: Cautious Openness, Engineering Internationalization

Some countries try to find a balance between the two, maintaining caution in spatial data while being highly internationalized in engineering and industry. This shows that the difference is not about being advanced or backward, but about different objective functions.

The following diagram compares the core characteristics, typical cases, and advantages/challenges of these three paths from a global perspective. The “Engineering-Open Type” on the left shapes the global infrastructure software landscape through standards and ecosystems; the “Governance-Sovereignty Type” in the middle emphasizes data sovereignty and security controllability but has limitations in influence at the rules layer; the “Compromise-Coordinated Type” on the right attempts to find a balance between security and openness. The divide between these three paths directly affects the infrastructure competition landscape of various countries in the AI era.

Figure 3: Global Perspective: Three Paths for Public Data and Technical Standards

The Essence of “Point” vs. “Area” in Air Quality

Among all spatial public data, air quality is an ideal observation window:

Does not directly involve military or core economic security
Highly visible, updated daily, and perceptible to everyone

China does not lack air quality data; on the contrary, the density of monitoring stations is among the highest in the world. The real difference lies in:

Whether interpolation is allowed
Whether model inference is allowed
Whether platforms are allowed to reinterpret the data

“Point” means authenticity and traceability; “area” means models, inference, and redistribution of interpretive authority. This is precisely the watershed between technical standards and data sovereignty.

The following diagram compares two different technical paths. The left side, “Governance-Sovereignty Type,” emphasizes data traceability and controllability, using discrete point-based data presentation. The right side, “Engineering-Open Type,” allows model interpolation and inference, providing more user-friendly experience through continuous area-based coverage. The essence of this difference lies not in the level of technical capability, but in the different choices made between data sovereignty, governance capability, and open ecosystems.

Figure 4: Technical Standards and Sovereignty Divide in Spatial Data Presentation

The Amplification Effect in the AI Era

With the above logic in mind, many phenomena in the AI era become less confusing.

For example:

Why are Chinese AI companies more willing to open source large language model (LLM) weights, while American companies have clearly shifted toward closed source in recent years?
Why is foundational software and infrastructure-level open source still mainly led by the US?

The key is not “whether to open source,” but “which layer is open sourced.”

Model weights are static, declarable assets
Infrastructure, runtimes, protocols, and standards are dynamic, evolving system rules

Open sourcing weights is essentially openness at the asset layer; infrastructure-level open source means relinquishing control over operating rules and interpretive authority.

The following diagram compares two different layers of AI open source. The left side shows “Model Weight Layer Open Source,” which is a typical feature of Chinese path—opening static digital assets with low cost and controllable risk, but not involving rule-making. The right side shows “Infrastructure Layer Open Source,” which is a core strategy of US path—by open sourcing development tools, protocol standards, runtimes, and compute scheduling and other infrastructure, defining how AI is used, thereby mastering ecosystem rules and interpretive authority. Key insight: Open sourcing model weights does not equal mastering AI ecosystem, and the real competitive focus is shifting to the infrastructure layer of “how AI runs.”

Figure 5: Two Layers of AI Era Open Source: Model Weights vs Infrastructure

The US Approach: Focusing on Rules and Runtime Layers

In the past year or two, US-led AI open source and ecosystem initiatives have shown a highly consistent direction: not rushing to open source the strongest models, but focusing on defining “how AI is used.”

The Linux Foundation established AAIF (Agentic AI Foundation), focusing on AI infrastructure, standards, and toolchain collaboration
Protocols like MCP (Model Context Protocol) aim to define common interaction methods between agents and tools/systems
Major tech companies are generally focusing on APIs, platforms, runtimes, and ecosystem binding

The commonality of these actions: competing in model capability, but controlling the usage rules.

China’s Shift: From Model-Oriented to Infrastructure-Oriented

It is important to emphasize that this difference does not mean China is unaware of the issue.

Whether in policy discussions or within industry and research institutions, the risk of “only open sourcing models without controlling infrastructure and standard dominance” has been repeatedly discussed.

The real challenge lies in how to achieve a directional shift within the existing governance logic and risk framework. This shift has already appeared in some concrete practices.

Exploration and Practice at the Infrastructure Layer

In the AI era, infrastructure often starts with the most engineering-driven problems.

HAMi Project

Projects like HAMi do not focus on model capability, but on:

Abstraction, allocation, and isolation of GPU resources
How multi-tenant AI workloads are run
How computing power transitions from hardware assets to governable system resources

The significance of such projects is not about being “SOTA,” but about entering the domain of “how AI runs.”

AI Runtime Reconstruction from a System Software Review

Exploration at the research institution level is also noteworthy. The FlagOS initiative by the Beijing Academy of Artificial Intelligence is a clear signal: AI is being redefined as a system software issue, not just a model or algorithm problem.

Long-Term Tech Stack Investment by Industry Players

In the industry, Huawei’s strategy reflects a similar direction: not simply open sourcing models, but attempting to build a complete, controllable AI tech stack, from computing power to frameworks, platforms, and ecosystems. This is a slower, heavier, but more infrastructure-competitive path.

Realistic Assessment: The Starting Point of AI Infrastructure Competition

Taking a longer view, we find an easily overlooked fact:

At the level of AI infrastructure and infra-level open source, there is no settled pattern between China and the US.

The US advantage lies in:

Mature engineering culture
Standard organizations and foundation mechanisms
High proficiency in openness at the rules layer

China’s variables include:

Huge AI application scenarios
Extreme demand for computing power and system efficiency
Ongoing directional adjustments

The real uncertainty is not “whether we can catch up,” but whether it is possible to gradually open up space for engineering autonomy and standard co-construction while maintaining governance bottom lines.

Summary

The “points” and “areas” of air quality, model weights and the world of operations—behind these appearances lies not a simple technical route dispute, but how a country finds its own balance between openness, standards, and sovereignty.

In the AI era, this issue will not disappear, but will become more concrete and more engineering-driven. And this is precisely where there are still opportunities for China’s AI infrastructure open source.

Joining Dynamia: Embarking on a New Journey in AI Native Infrastructure

Jimmy Song — Wed, 07 Jan 2026 07:49:21 +0000

Compute governance is the critical bottleneck for AI scaling. From hardware consumption to core asset, this long-undervalued path needs to be redefined.

Figure 1: Dynamia.ai

A New Beginning

I have officially joined Dynamia as Open Source Ecosystem VP, responsible for the long-term development of the company in open source, technical narrative, and AI Native Infrastructure ecosystem directions.

Why I Chose Dynamia

I chose to join Dynamia not because it’s a company trying to “solve all AI problems,” but precisely the opposite—it’s because Dynamia focuses intensely on one unavoidable, yet long-undervalued core issue in AI Native Infrastructure: compute, especially Graphics Processing Units (GPU), are evolving from “technical resources” into infrastructure elements that require refined governance and economic management.

Through years of practice in cloud native, distributed systems, and AI infrastructure (AI Infra), I’ve formed a clear judgment: as Large Language Models (LLM) and AI Agents enter the stage of large-scale deployment, the real bottleneck limiting system scalability and sustainability is no longer just model capability itself, but how compute is measured, allocated, isolated, and scheduled, and how a governable, accountable, and optimizable operational mechanism is formed at the system level. From this perspective, the core challenge of AI infrastructure is essentially evolving into a “resource governance and Token economy” problem.

About Dynamia and HAMi

Dynamia is an AI-native infrastructure technology company rooted in open source DNA, driving efficiency leaps in heterogeneous compute through technological innovation. Its leading open source project, HAMi (Heterogeneous AI Computing Virtualization Middleware), is a Cloud Native Computing Foundation (CNCF) sandbox project providing GPU, NPU and other heterogeneous device virtualization, sharing, isolation, and topology-aware scheduling capabilities, widely adopted by 50+ enterprises and institutions.

Dynamia’s Technical Approach

In this context, Dynamia’s technical approach—starting from the GPU layer, which is the most expensive, scarcest, and least unified abstraction layer in AI systems, treating compute as a foundational resource that can be measured, partitioned, scheduled, governed, and even “tokenized” for refined accounting and optimization—aligns highly with my long-term judgment on AI-native infrastructure.

This path doesn’t use “model capabilities” or “application innovation” as selling points in the short term, nor is it easily packaged into simple stories. However, with rising compute costs, heterogeneous accelerators becoming the norm, and AI systems moving toward multi-tenant and large-scale operations, these infrastructure-level capabilities are gradually becoming prerequisites for the establishment and expansion of AI systems.

Future Focus

As Dynamia’s Open Source Ecosystem VP, I will focus on technical narrative of AI-native infrastructure, open source ecosystem building, and global developer collaboration, promoting compute from “hardware resource being consumed” to governable, measurable, and optimizable AI infrastructure core asset, laying the foundation for the scaling and sustainable evolution of AI systems in the next stage.

Summary

Joining Dynamia is an important milestone in my career and a concrete action demonstrating my long-term optimism about AI-native infrastructure. Compute governance is not a short-term trend that yields quick results, but an infrastructure proposition that cannot be bypassed for AI large-scale deployment. I look forward to exploring, building, and landing solutions on this long-undervalued path with global developers.

References

Running Parallel AI Agents on My Mac: Hands-On with Verdent's Standalone App

Jimmy Song — Sun, 04 Jan 2026 02:25:48 +0000

I’ve been spending more time recently experimenting with vibe coding tools on real projects, not demos. One of those projects is my own website, where I constantly tweak content structure, navigation, and layout.

During this process, I started using Verdent’s standalone Mac app more seriously. What stood out was not any single feature, but how different the experience felt compared to traditional AI coding tools.

Figure 1: Verdent Standalone App UI

Verdent doesn’t behave like an assistant waiting for instructions. It behaves more like an environment where work happens in parallel.

A Different Starting Point: Tasks, Not Chats

Most AI coding tools begin with a conversation. Verdent begins with tasks.

When I opened my website repository in the Verdent app, I didn’t start with a long prompt. I created multiple tasks directly: one to rethink navigation and SEO structure, another to explore homepage layout improvements, and a third to review existing content organization.

Each task immediately spun up its own agent and workspace. From the beginning, the app encouraged me to think in parallel, the same way I normally would when sketching ideas on paper or jumping between files.

This framing alone changes how you work.

Built for Multitasking, Without Losing Context

Switching contexts is unavoidable in real development work. What usually breaks is continuity.

Verdent handles this well. Each task preserves its full context independently. I could stop one task mid-way, switch to another, and come back later without re-explaining the problem or reloading files.

For example, while one agent was analyzing my site’s navigation structure, another was exploring layout options. I moved between them freely. Nothing was lost. Each agent remembered exactly what it was doing.

This feels closer to how developers think than how chat-based tools operate.

Safe Parallel Coding with Workspaces

Parallel work only becomes truly safe when code changes are isolated. When parallelism moves from discussion to actual code modification, risk management becomes essential.

Verdent solves this with Workspaces. Each workspace is an isolated, independent code environment with its own change history, commit log, and branches. This isn’t just about separation—it’s about making concurrent code changes manageable.

What this means in practice:

Multiple tasks can write code simultaneously
Changes remain isolated from each other
If conflicts arise, they’re visible and cleanly resolvable

I intentionally let different agents operate on overlapping parts of my project: one modifying Markdown content and links, another adjusting CSS and layout logic. Both ran in parallel. No conflicts emerged. Later, I reviewed the diffs from each workspace and merged only what made sense.

This kind of isolation removes significant anxiety from AI-assisted coding. You stop worrying about breaking things and start experimenting more freely, knowing that each change exists in its own contained environment.

Parallel Agent Execution Feels Like Delegation

Parallelism doesn’t mean that all agents complete the same phase of work at the same time—instead, by isolating and overlapping phases, what was once a strictly sequential process is compressed into a more efficient, collaborative mode.

In Verdent, each agent runs in its own workspace, essentially an automatically managed branch or worktree. In practice, I often create multiple tasks with different responsibilities for the same requirement, such as planning, implementation, and review. But this doesn’t mean they all complete the same phase simultaneously.

These tasks are triggered as needed, each running for a period and producing clear artifacts as boundaries for collaboration. The planning task generates planning documents or constraint specifications; the implementation task advances code changes based on those documents and produces diffs; the review task, according to the established planning goals and audit criteria, performs staged reviews of the generated changes. By overlapping phases around artifacts, the originally strict sequential process is compressed into a workflow that more closely resembles team collaboration.

The value of splitting into multiple tasks is not parallel execution, but parallel cognition and clear collaboration boundaries.

While it’s technically possible to put multiple roles into a single task, this causes planning, implementation, and review to share the same context, which weakens role isolation and the auditability of results.

Configurability and Design Trade-offs

Beyond the workflow model itself, Verdent exposes a surprisingly rich set of configurable capabilities.

It allows users to customize MCP settings, define subagents with configurable prompts, and create reusable commands via slash (/) shortcuts. Personal rules can be written to influence agent behavior and response style, and command-level permissions can be configured to enforce basic security boundaries. Verdent also supports multiple mainstream foundation models, including GPT, Claude, Gemini, and K2. For users who prefer a lightweight coding experience without a full IDE, Verdent offers DiffLens as an alternative review-oriented interface. Both subscription-based and credit-based pricing models are supported.

Figure 2: Verdent Settings

That said, Verdent makes a clear set of trade-offs. It is not built around tab-based code completion, nor does it offer a plugin system. If it did, it would start to resemble a traditional IDE - which does not seem to be its goal. Verdent is not designed for direct, fine-grained code manipulation; most changes are mediated through conversational tasks and agent-driven edits. This makes the experience clean and focused, but it also means that for large, highly complex codebases, Verdent may function better as a complementary orchestration layer rather than a full-time development environment.

Where Verdent Fits Today

There are many AI-assisted coding tools emerging right now. Some focus on smarter editors, others on faster generation.

Verdent feels different because it focuses on orchestration, not just assistance.

It doesn’t try to replace your editor. It sits one level above, coordinating planning, execution, and review across multiple agents.

That makes it particularly suitable for exploratory work, refactoring, and early-stage design - exactly the kind of work I was doing on my website.

Final Thoughts

Using Verdent’s standalone app didn’t just speed things up. It changed how I structured work.

Instead of doing everything sequentially, I started thinking in parallel again - and letting the system support that way of thinking.

Verdent feels less like an AI feature and more like an environment that assumes AI is already part of how development happens.

For developers experimenting with AI-native workflows, that shift is worth paying attention to.

2025 Annual Review: The Transformation Journey from Cloud Native to AI Native

Jimmy Song — Wed, 31 Dec 2025 10:02:01 +0000

The waves of technology keep evolving; only by actively embracing change can we continue to create value. In 2025, I chose to move from Cloud Native to AI Native—this year marked a key turning point for personal growth and system reinvention.

2025 was a turning point for me. This year, I not only changed my technical direction but also the way I approach problems. Moving from Cloud Native infrastructure to AI Native Infrastructure was not just a migration of content, but an upgrade in mindset.

Figure 1: Farewell 2025!

This year, I conducted a large-scale refactoring of the website and systematically organized the content. Beyond the technical improvements, I want to share my thoughts and changes throughout the year.

A Bold Shift: Embracing the AI Native Era

At the beginning of 2025, I made an important decision: to reposition myself from a Cloud Native Evangelist to an AI Infrastructure Architect. This was not just a change in title, but a strategic transformation after careful consideration.

As I witnessed the surge of AI technologies and the rise of Agent-based applications reshaping software, I realized that clinging to the boundaries of Cloud Native might mean missing an era. So, I systematically adjusted the website’s content structure, shifting the focus toward AI Native Infrastructure.

This transformation was not about abandoning the past, but extending forward from the foundation of Cloud Native. Classic content like Kubernetes and Istio remains and is continuously updated, but new topics such as AI Agent and the AI OSS landscape have been added, forming a more complete knowledge map.

Content Creation: From Technical Details to Ecosystem Perspective

AI Agent: Building Systematic Knowledge

Agents represent a major evolution in software for the AI era. When I tried to understand Agent design principles, I found fragmented information everywhere but lacked a systematic knowledge base.

So I created content that analyzes the Agent context lifecycle and control loop mechanisms, summarizing several proven architectural patterns. To make complex knowledge easier to digest, I organized it into logical sections so readers can learn step by step.

AI Tool Ecosystem: Mapping the Open Source Landscape

AI tools and frameworks are emerging rapidly, with new projects appearing daily. To help readers quickly grasp the ecosystem, I built a comprehensive AI OSS database.

This database covers everything from Agent frameworks to development tools and deployment services. I not only included active projects but also established an archive mechanism, preserving detailed information on over 150 historical projects. More importantly, I developed a scoring system to objectively evaluate projects across dimensions like quality and sustainability, helping readers decide which tools are worth investing time in.

Blogging: Capturing Technology Trends Faster

In 2025, I wrote over 120 blog posts. Compared to previous years, these articles focused more on observing and reflecting on technology trends, rather than just technical tutorials.

I started paying attention to deeper questions: How will AI infrastructure evolve? What does Beijing’s open source initiative mean for the AI industry? What ripple effects might a tech acquisition trigger? These articles allowed me and my readers to not only see “what” technology is, but also “why” and “what’s next.”

User Experience: Making Knowledge Easier to Discover and Consume

No matter how good the content is, if it can’t be easily found and read, its value is greatly diminished. In 2025, I invested significant effort into website functionality, with one goal: to provide readers with a smoother reading experience.

Comprehensive Search Upgrade

As the volume of content grew, the original search function could no longer meet demand. I redesigned the search system to support fuzzy search and result scoring, and optimized index loading performance. More importantly, the new search interface is more user-friendly, supporting keyboard navigation and category filtering so users can find what they want faster.

Multi-Device Experience Optimization

Mobile reading experience has improved significantly. I refactored the mobile navigation and table of contents, making reading on phones much smoother. Dark mode is now more refined, fixing several display issues and ensuring images and diagrams look good on dark backgrounds.

Efficiency Revolution in Content Distribution

A major change was optimizing the WeChat Official Account publishing workflow. Previously, publishing website content to WeChat required manual handling of many details; now, it’s almost one-click export. This workflow automatically processes images, metadata, styles, and all details, reducing a half-hour task to just a few minutes.

Additionally, I added a glossary feature for technical term highlighting and tooltips; improved SEO and social sharing metadata; and cleaned up outdated content. These seemingly minor improvements quietly enhance the user experience.

Content Evolution: More Dimensional Knowledge Expression

Looking back at content creation in 2025, I found clear changes in several dimensions.

From Tutorials to Observations

Early content leaned toward technical tutorials and practical guides, showing “how to do.” This year, I focused more on “why” and “what are the trends.” I wrote more technology trend analyses, ecosystem maps, and in-depth case studies. These may not directly teach you how to use an API, but they help you understand the direction of technological evolution.

From Chinese to Bilingual

AI is a global wave and cannot be limited to the Chinese-speaking world. In 2025, I wrote bilingual documentation for almost all new AI tools, and important blog posts also have English versions. This increased the workload, but allowed the content to reach a broader audience.

From Text to Multimedia

Text is efficient, but not all knowledge is best expressed in words. This year, I used many architecture and schematic diagrams to explain complex concepts, adding 59 new charts. These visual elements lower the barrier to understanding, making abstract concepts more intuitive. I also optimized image display in dark mode to ensure consistent visual experience.

Development Approach: Embracing AI-Assisted Programming

2025 was not only a year of shifting content themes toward AI, but also a year of deep practice in AI-assisted programming.

I developed a VS Code plugin and created many prompts to automate repetitive tasks. I experimented with various AI programming tools and settled on a toolchain that suits me. I even migrated the website to Cloudflare Pages and used its edge computing services to develop a chatbot. These practices greatly improved development efficiency, giving me more time to focus on thinking and creating rather than mechanical coding.

This made me realize: AI will not replace developers, but developers who use AI well will replace those who do not. I also shared more insights to help others master AI-assisted programming.

Looking Ahead to 2026: Keep Moving Forward

Looking back at 2025, the site underwent a profound transformation—from a Cloud Native tech blog to an AI infrastructure knowledge base. But this is just the beginning, not the end.

Looking forward to 2026, I plan to continue deepening in several areas:

Enhancing the knowledge system: Continue to supplement GPU infrastructure and AI Agent content, especially practical cases and performance tuning knowledge.
Tracking ecosystem evolution: AI tools and frameworks iterate rapidly; I need to keep up with this fast-changing ecosystem and update content in a timely manner.
Deepening engineering practice: Share more practical AI engineering experience to help readers turn theory into practice.
Exploring knowledge connections: Consider building a knowledge graph to connect different content sections, providing smarter navigation and recommendations.

Summary

2025 was a year of change and growth. From Cloud Native to AI Native, from technical practice to ecosystem observation, both the content and functionality of the site have made qualitative leaps.

What makes me happiest is that this transformation allowed me and my readers to stand at the forefront of the technology wave. We are not just learning new technologies, but thinking about how technology changes the world and the way we write software.

The waves of technology keep evolving; only by actively embracing change can we continue to create value. Thank you to every reader for your companionship and support. I look forward to sharing more insights and practices in 2026.

Further Reading:

The Butterfly Effect After Manus Was Acquired by Meta

Jimmy Song — Tue, 30 Dec 2025 03:30:51 +0000

The success or failure of AI applications often lies not in the technology itself, but in the ability to scale delivery and create a closed loop.

Figure 1: The Butterfly Effect After Manus Was Acquired by Meta

When “Those Who Discuss It” Are Not “Those Who Pay for It”

On December 30, 2025, a piece of news went viral: Manus was acquired by Meta for billions of dollars (Manus Joins Meta for Next Era of Innovation). This startup, founded in China and under pressure from tech giants since its inception, completed a whirlwind journey in less than a year—from explosive growth, relocating to Singapore, to being acquired by a global giant.

According to Manus’s official statement, its products and subscriptions will continue to be available via the app and website, and the company will remain operational in Singapore. The team will join Meta to provide general Agent capabilities for Meta’s consumer and enterprise products (including Meta AI).

Rather than focusing on “who won,” I’m more interested in the chain reaction this event triggered: it activated completely opposite judgment systems among different groups, and this split is reshaping the growth paths and strategies for AI applications and startups.

Two Public Opinion Arenas: Blessings and Doubts Coexist

After Manus was acquired, the mainstream sentiment in social circles was one of congratulations and excitement. Many saw it as a stellar example of a Chinese team going global—achieving remarkable results in the most competitive field in a very short time.

Meanwhile, the comment sections of public accounts became “venting valves for counter-narratives,” with skepticism centering on three main points:

Whether the technology has real barriers (e.g., “there are countless similar products,” “it’s not hard for big companies to build their own”).
Valuation and bubble concerns (e.g., “another case of the AI bubble”).
Distrust in the buyer’s judgment (e.g., “giants making desperate bets,” “history repeating itself”).

This divergence isn’t about who understands AI better, but about different evaluation frameworks: social circles focus on “trajectory and outcome,” while comment sections focus on “legitimacy and worthiness.”

Many people are impressed by Manus’s marketing buzz and controversies, which can lead to skepticism. But if it achieved a “strict $100M ARR” in 10 months, one fact is clear: its revenue doesn’t depend on broad consensus, but comes from a highly concentrated group of global users with strong willingness to pay.

Manus’s core user profile is closer to “individuals as production units,” including freelancers, indie developers, independent researchers, and key deliverers in small and medium businesses. They don’t care about debates over “wrapping” or not; they care about “can I deliver end-to-end tasks,” and “can this help me hire one less person, work fewer late nights, or avoid juggling ten tools.”

This leads to a counterintuitive phenomenon: those who discuss the most may not pay, while those who pay steadily are often silent.

For these users, tools are not identity badges—they are profit levers.

Three Lessons for Entrepreneurs: The Growth Paradigm in the AI Application Era Has Changed

Based on the above, the Manus case offers three lessons for entrepreneurs:

Growth No Longer Equals Positive Reviews

AI applications can commercialize first and build consensus later. Public opinion can remain divided for a long time, but cash flow doesn’t wait for unified recognition.

“Heavy Marketing” Is Becoming a Capability, Not a Stigma

As foundational models and capabilities spread rapidly, differentiation is quickly erased. Being seen, understood, and paid for is itself part of the moat. Not all marketing deserves respect, but “distribution and mindshare” have become unavoidable battlegrounds for AI applications.

Globalization Is No Longer a Bonus, but May Be a Survival Strategy

From payment willingness, compliance boundaries, talent density to valuation systems, market structure means many teams “can only complete the loop overseas.” It’s not romantic, but it’s reality.

A Personal Reflection

As someone long engaged in cloud native and AI infrastructure, I’m used to evaluating products by their “technical barriers.” But cases like Manus remind me: at the AI application layer, barriers may not first appear in models or code, but often in organizational speed, productization capability, delivery loop, and distribution efficiency.

When a system can reliably turn “capability” into “results,” it has built a commercial moat—even if its tech stack doesn’t meet outsiders’ ideals of “purity.”

The biggest butterfly effect of Manus being acquired by Meta may not be the deal itself, but making more entrepreneurs realize: in the AI era, the winning move is shifting from “what model you use” to “whether you can deliver results at scale.”

Summary

The acquisition of Manus by Meta is not just a convergence of capital and technology, but also a microcosm of the changing growth paradigm in the AI application era. For entrepreneurs, understanding and mastering “user structure,” “distribution capability,” and “global closed loops” will be key to future competition.

AI Infra Open Source in China: Analysis of Beijing and Shanghai's Plans

Jimmy Song — Thu, 25 Dec 2025 10:01:13 +0000

Institutionalized open source marks a new starting point for China’s AI Infra, but true breakthroughs and risks lie in the engineering and governance details.

Perspective on Beijing and Shanghai’s Open Source Plans

Using the simultaneous release of open source ecosystem plans by Beijing and Shanghai as a lens, and drawing on China’s past foundation practices and international open source governance experience, this article explores the real opportunities, structural constraints, and potential risks as AI Infrastructure (AI Infra, Artificial Intelligence Infrastructure) enters a new phase of institutionalized open source.

Figure 1: Beijing and Shanghai successively launch open source ecosystem construction plans

Why Compare Beijing and Shanghai Together

It is rare for me to write an article solely because of a local policy document. However, during Christmas, both Beijing and Shanghai’s Bureaus of Economy and Information Technology released their respective open source ecosystem construction plans:

This time, the fact that both cities released their plans on the same day sends a signal worth serious attention: China is attempting to advance open source in a more systematic and institutionalized way, especially regarding open source capabilities related to AI Infra.

If you only look at Beijing’s plan, it is easy to interpret it as a local industrial policy upgrade. But when you consider both Beijing and Shanghai’s plans together, it looks more like a clearly defined “dual-center structure.”

The question is no longer whether to develop open source, but:

In the AI era, what institutional forms, engineering paths, and governance models will open source take?

Open Source as “Industrial Infrastructure Engineering”

Both Beijing and Shanghai’s plans reflect a highly consistent judgment:

Open source is no longer seen as a spontaneous community activity, but as an industrial infrastructure capability that requires systematic construction.

This is especially evident in the field of AI Infra.

Issues such as computing power scheduling, model evaluation, toolchains, data elements, license compliance, and supply chain security—previously hidden in “engineering details”—are now systematically incorporated into policy language for the first time. This at least shows that decision-makers have realized:

AI competition is not only about model parameter scale
It is even more about toolchains, infrastructure, evaluation systems, and engineering capabilities
These capabilities are naturally more suitable for building public foundations through open source

In this respect, Beijing and Shanghai are highly aligned.

Two Open Source Paths: Infra vs. Platform

When we zoom in, the differences between the two plans become clear.

Beijing: “Foundation-Oriented” Open Source Path for AI Infra

Beijing’s plan focuses on:

Heterogeneous computing power scheduling
Model evaluation toolchains
Data elements and data governance
RISC-V software-hardware collaboration
SBOM, license compatibility, open source compliance
Supply chain security and industrial resilience

This is a typical perspective of “treating AI as an infrastructure problem.”

It is less concerned with the number of projects or community size, and more with:

Whether reusable engineering capabilities can be formed
Whether these can be trusted by industry and government over the long term
Whether they can stand up to scrutiny in terms of security, compliance, and governance

To some extent, Beijing is answering the question:

How can open source become a “governable, auditable, and scalable public capability”?

Shanghai: “Scale and Internationalization” Path for AI Platform

In contrast, Shanghai’s plan has a different focus:

Building an international open source community for artificial intelligence
Covering the entire platform chain from development, training, testing, hosting, to operation
Overseas sites, multilingual support, international activities
Resource linkage through computing vouchers and model vouchers
“Open source platform first release / global simultaneous release” dual-release mechanism
Clear targets for community, enterprise, and developer scale

Shanghai cares more about:

How open source can achieve scale effects
How it can support the growth of commercial enterprises
How it can be seen and adopted globally

This is a path of “treating open source as a global digital product and platform capability.”

Together: A Complete but Tension-Filled Structure

When viewed together, Beijing and Shanghai’s plans form a more complete picture:

Beijing is responsible for “making open source solid,” while Shanghai is responsible for “taking open source global.”

Structurally, this is a clear division of labor:

Beijing focuses on institutions, governance, and foundational capabilities
Shanghai focuses on community, commercialization, and international communication

These two paths are not in conflict; in theory, they are even complementary. The real question is whether they can form positive feedback in practice, rather than operating in silos.

Cautious Attitude Toward “Institutionalized, Platformized Open Source”

Precisely because both plans are so “systematic,” I am even more cautious.

The reason is simple: this is not China’s first attempt to promote open source through foundations, associations, or platforms.

Over the past decade, we have seen similar paths repeatedly, and recurring structural problems:

The difficulty of establishing neutrality and multi-party trust is extremely high
There is a huge gap between showcase metrics (quantity, activities, certifications) and ecosystem strength
Commercialization and long-term maintenance mechanisms are hard to sustain

These problems will not disappear just because the plans are more comprehensive.

Four Risks to Watch Under the Dual Plans

If we are to “listen to their words and watch their actions,” I would focus on the following four risks:

Will Metrics Hijack Engineering Reality

When “internationally influential projects,” “star projects,” and “first-release projects” become hard metrics, will this induce packaging, migration, and short-term hype, rather than truly solving engineering problems?

Will It Slide Toward Platform Centralism

The long-term pattern of AI Infra is closer to a model that prioritizes protocols, standards, and interoperability. If it eventually evolves into “a few platforms concentrating resources and discourse power,” it may be efficient in the short term but will suppress external participation and international collaboration in the long run.

Is Internationalization Underestimated as an “Operational Issue”

True international collaboration is never just about language, sites, or events; it also involves governance structures, compliance boundaries, and supply chain trust.

Will Application Demonstrations Become One-Off Projects

If “first plans” and “computing vouchers” are just procurement tactics without continuous iteration and community feedback mechanisms, the long-term benefit to the ecosystem will be very limited.

What Are the “Hard Results” of AI Infra Open Source After Three Years

If we review the success of this round of institutionalized open source after three years, I would look for three types of results:

Whether de facto standards and interoperable ecosystems have emerged, including scheduling interfaces, evaluation benchmarks, Agent tool invocation protocols, and observability semantics.
Whether compliance and supply chain security have become public capabilities—SBOM, license compatibility, vulnerability monitoring—truly productized and service-oriented.
Whether a sustainable maintenance business mechanism has been established, allowing core maintainers to stay long-term, rather than relying on passion and subsidies.

If I were to use a North Star metric to measure the success of these plans, it would be the emergence of several outstanding open source commercial companies rooted in China and serving the world.

Summary

The open source ecosystem plans of Beijing and Shanghai mark a new phase of institutionalization and engineering for AI Infra open source in China. Over the next three years, the real achievements will not be about meeting targets, but about forming sustainable engineering capabilities, de facto standards, and maintenance mechanisms. Only through continuous participation and practice can open source become the public foundation of AI infrastructure.

References

From 2025 Onwards, Software Engineering Shifts from Code-Centric to Runtime and Cost-Centric

Jimmy Song — Wed, 24 Dec 2025 14:59:11 +0000

In 2025, the core of software engineering is no longer just about code itself, but about runtime controllability and cost governance. This shift is fundamentally reshaping the industry’s underlying logic.

Looking back at 2025, I became increasingly aware that this year was not about “code becoming unimportant,” but rather that the value coordinates of engineering have shifted as a whole. For more than a decade, software engineering has focused on code quality, architectural evolution, and delivery efficiency. But starting in 2025, the key to system success is shifting—towards whether the runtime is controllable and whether costs are governable.

This is not just a slogan, but a conclusion repeatedly validated by my real-world experiences throughout the year.

My 2025: From “Platform Engineering” to “Runtime Challenges”

In my annual review, I noted a clear change: I spent less time on “how to write a good system,” and more time on “how to keep the system running stably, reliably, and affordably.”

This shift in focus is a natural extension of a decade of cloud native evolution.

The following timeline diagram illustrates how my focus has changed over recent years:

Figure 1: My Focus Shift Timeline

My focus shifted from cloud native platform engineering to LLM application engineering, then to AI infrastructure, and finally to Agentic Runtime with governance and cost control.

When AI workloads truly enter business scenarios, the core challenges engineers face also change:

Are inference, training, and evaluation competing for the same compute pool?
Is GPU utilization consistently below expectations?
Does cost scale linearly and uncontrollably with concurrency?
Does the system have failure isolation and replay capabilities?

These issues go far beyond the code level.

Industry Consensus: AI Is Shifting the Focus of Engineering

By 2025, an industry consensus is emerging: AI is rewriting software engineering. But the real change is not happening in the IDE or code completion speed—it is reflected in the migration of engineering complexity.

Previously, complexity was concentrated in code and interfaces, and problems were solved through abstraction, refactoring, and testing.

Now, complexity has shifted to the runtime, resource, and cost layers, and must be addressed through scheduling, isolation, observability, and governance.

This is why the same AI tools:

Serve as “accelerators” for junior engineers
But act as “magnifiers” for senior engineers

AI tools amplify whether you truly understand how systems run in production.

Why “Cost” Becomes a First Principle

In traditional cloud native systems, low CPU utilization is often just an efficiency issue; but in AI systems, low GPU utilization is often a cash flow problem.

In 2025, I repeatedly encountered scenarios like:

Resources “seem insufficient,” but utilization is not actually high
Scaling up to solve queuing issues ends up increasing unit costs
The system lacks clear budget and quota boundaries, so throttling becomes the only way to stop the bleeding

The root cause of these phenomena is not model selection, but the lack of a runtime and cost control plane tailored for AI workloads.

The following flowchart visually illustrates the cyclical relationship between GPU resources and cost pressures:

Figure 2: GPU Resource and Cost Cycle in AI Systems

In AI systems, limited GPU supply leads to queuing and waiting, which causes throughput to drop. Attempts to solve this through blind scaling only increase unit costs and create budget pressure, ultimately forcing the adoption of finer scheduling and governance strategies.

Engineering problems ultimately manifest as cost issues.

The Rise of Agents: The Real Challenge Is at Runtime

In 2025, Agent (Intelligent Agent, Agent, Intelligent Agent) became a hot topic; by 2026, it will enter the “can it actually run” stage.

The challenge for Agents has never been about “how smart they are,” but rather:

Whether there are clear permission and data boundaries
Whether they run in an isolated execution environment
Whether they can be observed, evaluated, and replayed
Whether they are subject to explicit cost and budget constraints

These capabilities form the outline of Agentic Runtime (Agentic Runtime, Intelligent Agent Runtime) that I have been trying to clarify throughout the year.

The following flowchart shows the core capability layers of Agentic Runtime:

Figure 3: Agentic Runtime Capability Layers

Agentic Runtime builds from the foundation of Agents and workflows, connecting through orchestration and tool protocols, with the runtime managing state, memory, and evaluation. It provides secure execution environments (Sandbox and Policy), and ultimately implements a resource and cost control plane that unifies GPU, quota, and billing management.

Without a runtime, an Agent is just a demo; without cost constraints, an Agent is just a risk amplifier.

Outlook for 2026: The “Foundation” of Engineering Matters Again

Looking ahead to 2026, I remain cautiously optimistic.

I do not believe the future belongs to “those who write the best prompts,” but more likely to:

Those who understand runtime boundaries
Those who can govern compute as a constrained resource
Those who design AI systems as long-running systems

From 2025 onwards, software engineering is no longer code-centric, but runtime and cost-centric. This is not a regression, but a return: a return to being responsible for the whole system and for real-world constraints.

For me personally, this is both a year-end summary and the direction I will continue to invest in for the coming years.

Summary

In 2025, the focus of software engineering has shifted from code itself to runtime and cost governance. The rise of AI and Agents has not diminished the value of engineering, but has pushed complexity to a higher level. In the future, understanding runtime, managing compute and cost will become the new core competencies for engineers. I hope this year-end review provides some inspiration and reflection for fellow professionals.

From Cloud Native to AI Native: Why Kubernetes Is the Foundation for Next-Gen AI Agents

Jimmy Song — Wed, 24 Dec 2025 12:25:52 +0000

As a long-time practitioner in the cloud native field, I am increasingly convinced of one thing: AI Agents are not just a change in application form, but a migration of infrastructure paradigms.

As artificial intelligence evolves from demos and copilots to systems that truly take on tasks and responsibilities, AI Agents are becoming the new execution units in enterprise IT architectures. They not only “think,” but also act: they can invoke tools, access systems, and collaborate to achieve goals.

This raises an important question:

What kind of infrastructure should such systems run on?

In my view, Kubernetes remains a solid choice for large-scale scenarios—but only if we reimagine Kubernetes in an AI-native way.

Cloud Native Challenges for Production-Grade AI Agents

In real production environments, AI Agents expose infrastructure needs that are fundamentally different from traditional microservices. Agents are not “just another HTTP service”; they have three distinct characteristics:

Behavior is non-deterministic (driven by model inference)
Execution paths are dynamic (tool invocation cannot be fully enumerated in advance)
Decisions must be auditable, constrained, and reviewable

If we simply apply existing cloud native infrastructure, we quickly hit bottlenecks.

The following table summarizes the main challenges and risks AI Agents face in cloud native environments:

Challenge Category	Real Needs of Agents	What Happens If Missing
Policy & Security	Dynamic control of tool and data access based on context, identity, and task	Agents have “superuser” privileges, risks are uncontrollable
Observability	Not just “did it succeed,” but also why was this decision made	Hard to debug, hard to review, hard to hold accountable
Governance & Consistency	Platform-level guardrails enforce organizational policies	Each Agent could become a “shadow AI”

Table 1: Challenges and Risks for AI Agents in Cloud Native Environments

All these issues point to one conclusion:

AI Agents must be treated as first-class citizens in Kubernetes, not just ordinary workloads.

Core Architecture: Making Agents Native Kubernetes Objects

Looking back at the evolution of cloud native technologies, we’ve gone through similar stages:

Physical machines → Virtual machines
Virtual machines → Containers
Containers → Microservices
Microservices → Declarative, governable platforms

AI Agents are simply the next step.

A production-ready AI Agent architecture requires at least three layers:

Agent Orchestration Layer: Declaratively define Agents
Tool Service-ization Layer (MCP Services): Turn capabilities into governable services
AI Native Data Plane / Gateway: Unify policy, security, and protocols

Agent Orchestration Layer: Declarative Agent Management

Agents should no longer be “runtime objects” inside an SDK—they should be managed like Pods or Deployments.

Key concepts:

Agents as Kubernetes Resources

Agents are defined using CRD (CustomResourceDefinition)
Lifecycle managed via kubectl or GitOps
Agent models, tools, and policies are all explicitly declared

A typical Agent definition includes:

Agent logic (inference loop)
Model configuration (specifying which large language model to use)
Callable toolset

This closely mirrors how we once decomposed “applications” into Deployments, Services, and ConfigMaps.

Tool Service-ization Layer: MCP Services Are Essential

In Agent architectures, tools are where real “actions” happen.

Early MCP tools were often:

Local processes
Tightly coupled to a single Agent
Lacking versioning, permissions, and auditing

This is unsustainable in enterprise environments.

The Essence of MCP Service-ization

Tools → Remote services
Services → Kubernetes native workloads
Capabilities → Reusable, governable, auditable

This step is fundamentally similar to how we once turned scripts into microservices.

AI Native Gateway: The “Control Plane Entry” for the Agent World

As the number of Agents grows and tools/models diversify, connectivity itself becomes a system risk.

Traditional API Gateways do not understand scenarios like:

MCP
Agent-to-Agent (A2A) communication
Model invocation context

Thus, we need an AI native gateway dedicated to mediation and governance.

It must understand at least three types of traffic:

A2T: Agent → Tool
A2L: Agent → LLM
A2A: Agent ↔ Agent

And enforce, across these paths:

Identity and authorization
Policy and guardrails
Auditing and rate limiting

Architecture Overview

The diagram below illustrates the core layers and traffic paths of an AI-native system on Kubernetes:

Figure 1: AI Native Architecture Layers and Traffic Paths

Summary

AI Agents do not negate cloud native; on the contrary:

AI Agents are the natural extension of cloud native in the era of intelligence.

Declarative → Agent definitions
Service → MCP Services
Service Mesh → AI Native Gateway

If Kubernetes is the “automated factory,” then AI Agents are the intelligent workers who actually get things done.

And the AI native gateway is the security and governance system tailored for these intelligent workers.

This is not an optional architecture—it is the only path for AI to reach production.

AI Open Source Landscape: A One-Stop Guide to AI Project Navigation and Scoring System

Jimmy Song — Tue, 23 Dec 2025 08:34:05 +0000

The AI Open Source Landscape is not just a project directory, but an innovative attempt to bring transparency and quantifiability to the AI open source ecosystem.

Note: This article is intended for general readers and focuses on platform features and usage scenarios. If you want to see the technical details and formulas behind the scoring, please refer to: AI Project Scoring and Inclusion Criteria.

Project Background and Positioning

The AI Open Source Landscape aims to provide developers, researchers, and enterprise users with a one-stop navigation and evaluation platform for AI open source projects. With the rapid development of large language models (LLM, Large Language Model), multimodal models (Multimodal Model), and other AI technologies, the open source community has seen a surge of innovative projects. However, information is scattered and quality varies, making it difficult for users to filter and make decisions.

Figure 1: AI Open Source Landscape

The AI Open Source Landscape systematically collects mainstream AI open source projects. As of the time of writing, it has included 851 open source projects. This landscape combines a multi-dimensional scoring system to help users efficiently discover, compare, and select the most suitable AI tools and frameworks for their needs. The platform not only focuses on models themselves, but also covers datasets, inference engines, evaluation tools, application frameworks, and the entire ecosystem chain, striving to promote transparency, quantifiability, and sustainable development in the AI open source ecosystem.

Main Interface and Feature Highlights

The platform homepage presents project distribution in both landscape and list views, supporting category filtering, keyword search, and tag navigation to help users quickly locate target projects.

Figure 2: Open Source Project Detail Page

For general readers, the main experience points include:

Card view: One-sentence overview, star rating, and overall score for quick browsing and comparison.
Health card: Displays overall health and key dimensions (activity, community, influence, sustainability) on the project page or sidebar, with the latest update marked for easy assessment of maintenance status.
Detail page: Provides more background information, project links, and application scenarios to help you evaluate suitability for your needs.
Smart badges: Visually display labels such as “Active”, “New Project”, “Popular”, “Archived” on cards, helping you quickly capture key project features.

If you are interested in the specific rules for badge determination or scoring, detailed explanations are available on the Scoring Rules Page.

Scoring and Ranking Mechanism

The platform uses multi-dimensional scores to reflect the overall health and popularity of projects. The main dimensions include: Activity, Community, Quality, Sustainability, and the comprehensive Health score. These scores help you quickly judge whether a project is suitable for production or experimentation.

Data Sources and Update Mechanism

The platform’s data mainly comes from GitHub, project lists, official documentation, and community recommendations. We regularly and automatically synchronize and update metrics to ensure that the “last updated” and scores displayed on the interface reflect the current maintenance status of projects. Projects that have not been updated for a long time or are determined to be “inactive” are moved to the Archived Page. Archived projects remain searchable and retain historical scores, but will not appear in the default view of active rankings, making it easier for readers to focus on projects that are still maintained and active.

For general readers, the key points are:

The page displays key metrics and “last updated” time, helping you quickly judge whether a project is still maintained.
The AI Open Source Landscape continuously iterates on the scoring model to improve fairness and differentiation.

How to Contribute and Correct Data

If you want a project to be included or its data updated, you can:

Submit an AI Open Source Project Inclusion Request
Keep the project’s README, License, documentation, and other information complete in the repository to facilitate our data collection and assessment.
For faster synchronization or if you encounter data issues, contact the maintainers via project issues or raise a request in the site discussion area.

Typical Use Cases or User Feedback

The AI Open Source Landscape has been widely used in various scenarios such as AI developer selection, enterprise technology research, and academic studies. For example:

Developers can quickly filter models or tools that meet their needs through the platform, saving significant research time.
Enterprise technical teams use the ranking lists for competitor analysis and technology planning.
Educational and research institutions refer to the landscape to understand trends in the AI open source ecosystem, supporting course design and topic selection.

Some users have commented that the platform is “comprehensive, well-structured, and fair in scoring,” greatly improving the efficiency of AI project selection and learning. Community suggestions continue to drive ongoing improvements in platform features and content.

Summary

The AI Open Source Landscape systematically and quantitatively organizes the AI open source ecosystem: the backend worker is responsible for reliable data collection and scoring calculations (supporting backfill and migration), while frontend components handle fast rendering and visualization (including smart badges, health cards, and metric explanations).

If you want to learn more about the scoring details or participate in improvements:

The community is welcome to join in evaluation, backfilling historical data, and refining scoring rules, working together to make the AI open source ecosystem more transparent and sustainable.

AI 2026: Infrastructure, Agents, and the Next Cloud-Native Shift

Jimmy Song — Fri, 19 Dec 2025 03:54:31 +0000

The real turning point for AI in 2026 is not autonomy, but the maturity of infrastructure - where agentic runtimes, GPU efficiency, and organizational design will decide who wins.

Introduction: 2026 Is Not an AI Moment, It Is an Infrastructure Moment

Over the past fifteen years, every major shift in software has followed a familiar arc. Microservices were adopted not out of love for distributed systems, but because monoliths reached organizational limits. Kubernetes succeeded not because containers were novel, but because infrastructure finally matched how teams operated. Cloud native was never about YAML—it was about operability at scale.

AI now stands at a similar inflection point.

The central question for 2026 is not whether models will become more autonomous. That debate overlooks the core issue. Instead, the real question is whether AI can become operable, governable, and economically sustainable within real systems.

Most organizations today are limited not by intelligence, but by infrastructure: inefficient GPU utilization, escalating inference costs, fragile agent demos, and a tendency to treat AI as a feature rather than a runtime. The next phase of AI will be shaped not by model breakthroughs, but by the maturity of AI infrastructure and its ability to absorb responsibility.

From Automation to Capability Multiplication — A Familiar Cloud-Native Pattern

Reflecting on early cloud adoption, the dominant narrative was cost reduction: fewer servers, lower CapEx, elastic scaling. Yet, the true payoff emerged later, when teams realized cloud enabled entirely new operating models.

AI is repeating this pattern.

The following diagram illustrates the shift from automation to capability multiplication.

Figure 1: From Automation to Capability Multiplication

The first wave of AI focused on labor replacement. The second wave reframes AI as capability multiplication: the same team, observing more signals, covering broader areas, and acting sooner.

This mirrors the evolution of monitoring, tracing, and SRE practices. Rather than reducing engineers, these systems enabled continuous observation instead of occasional sampling.

Preemptive AI systems—monitoring every interaction, log, and signal—are only viable if the underlying infrastructure can support them. This exposes a critical constraint: AI capability scales faster than AI infrastructure.

Without efficient scheduling, isolation, and utilization, multiplying capability simply multiplies cost.

Agents Are Becoming Distributed Systems, Whether We Admit It or Not

The industry often discusses agents as products. In reality, agents are evolving into distributed systems.

The diagram below highlights this architectural shift.

Figure 2: Agents Are Becoming Distributed Systems

Single-agent designs resemble early monoliths: impressive demos, fragile behavior, and opaque failure modes. As tasks grow in complexity, systems must decompose work into planning, execution, verification, and review—making coordination inevitable.

This is not merely a philosophical change, but an architectural one.

Multi-agent systems introduce challenges familiar from the microservices era:

Coordination and orchestration
Resource contention
Fault isolation
Observability and rollback
Deterministic artifacts between stages

Labeling this as “multi-agent collaboration” can be misleading. What is actually occurring is workload decomposition and control-plane emergence. Agents are transitioning from tools to workloads competing for limited resources.

Recognizing this clarifies why agent progress is inseparable from infrastructure maturity.

AI Infra Is the Missing Layer Between Models and Organizations

Cloud native taught us that abstractions only scale when a control plane exists.

Currently, AI lacks a mature control plane.

The following image demonstrates the gap between models and organizations.

Figure 3: AI Infra Is the Missing Layer Between Models and Organizations

Models are powerful, but the surrounding infrastructure—scheduling, isolation, quota enforcement, cost attribution, observability—remains primitive, especially at the GPU layer.

GPUs are expensive, scarce, and often underutilized. In many environments, utilization remains below 30–40%, while inference costs continue to rise. Training pipelines monopolize resources, inference workloads spike unpredictably, and organizations must choose between waste and throttling innovation.

This is not a model problem. It is fundamentally an AI infrastructure problem.

The next phase of AI will depend on treating GPUs as we learned to treat CPUs:

Fine-grained allocation
Fair sharing
Preemption and prioritization
Clear ownership and accounting

Until GPU utilization becomes a primary design goal, AI systems will remain economically fragile.

Domain Expertise Matters Because Infrastructure Finally Exposes It

As models plateau in general reasoning, differentiation shifts elsewhere.

The diagram below illustrates how infrastructure exposes domain expertise.

Figure 4: Domain Expertise Matters Because Infrastructure Finally Exposes It

In cloud-native systems, competitive advantage eventually moved from frameworks to operational excellence: superior runbooks, incident response, and cost control. AI is following a similar trajectory.

High-value AI systems must operate within dense, rule-heavy domains such as finance, healthcare, manufacturing, and infrastructure operations. What matters is not abstract intelligence, but the ability to encode domain constraints, exceptions, and failure patterns.

Here, domain experts become central—not as prompt engineers, but as system shapers. Their decisions define agent permissions, human intervention points, and error containment strategies.

Infrastructure determines whether this expertise can be safely operationalized.

Simulation Is Becoming the New Staging Environment for AI

One of the most important lessons from cloud-native operations: distributed systems are not tested in production.

AI systems that act, plan, and modify state are no exception.

The following image shows simulation as the new staging environment.

Figure 5: Simulation Is Becoming the New Staging Environment for AI

Training and validating agents directly in live environments is unsustainable. The future lies in simulation-first AI development—sandboxed environments that mirror real systems, workloads, and constraints.

This approach is analogous to staging clusters, chaos engineering, and load testing, but elevated for decision-making systems. Evaluation shifts from static benchmarks to behavioral metrics: intervention rates, rollback frequency, and cost impact.

Organizations that build these environments will advance faster and safer. Those that do not may remain limited by conservative deployments and restricted autonomy.

Summary

Technological revolutions succeed not on novelty alone, but when infrastructure, tooling, and organizational models align.

AI is nearing that pivotal moment.

The leaders in 2026 will be those who:

Treat AI as a runtime, not just a feature
Optimize for resource efficiency, especially GPUs
Recognize agents as distributed systems
Redesign organizations around continuous learning systems
Invest in infrastructure ahead of autonomy

AI is no longer just a model problem. It is an infrastructure challenge—and the next phase will be decided not in labs, but in production systems.

What I Saw at COSCon'25: The Real State of Open Source in China

Jimmy Song — Thu, 18 Dec 2025 06:14:51 +0000

Attending COSCon'25 in Beijing, I observed firsthand how open source in China is shifting: AI is now the default context, discussions are grounded in real engineering, and the community is embracing long-term thinking. These are not just trends—they are the new reality.

In early December this year, I attended COSCon'25, the China Open Source Annual Conference, in Beijing. Although I have worked in open source for many years, this was my first time participating in an event organized by the Open Source Society—and I joined as a sub-forum producer. Previously, I thought such conferences were too high-level or disconnected from reality, but after actually taking part, I found there was much to gain.

A quick note: this article is not an official conference summary or review. The organizers have already published detailed information about the event’s scale, attendee numbers, and forum sessions. If you’re interested in those details, please refer to the official article: COSCon'25: The 10th China Open Source Annual Conference Successfully Concludes in Beijing—A Comprehensive Recap!

Figure 1: 10th COSCon Venue

What I want to share is this: Standing on site, on the engineering front lines, and as an organizer rather than an audience member, I saw real changes happening in Chinese open source.

This COSCon: No More Trying to “Prove Open Source Matters”

One clear impression: Almost no one spent time arguing “why do open source” anymore.

In earlier years, common narratives included:

Is open source safe?
Can open source be commercialized?
Can China create its own open source projects?

But at COSCon'25, these questions were basically assumed as “background conditions.” The focus shifted to those already doing open source, and what comes next.

This doesn’t mean the issues have disappeared, but it does mean:

In China’s engineering circles, open source is no longer a “philosophical choice”—it’s a practical way of working.

AI as Background Noise, Not the Main Character

The theme of this year’s conference was Open Source × Open Intelligence, but interestingly, AI did not take center stage.

Instead, it was more like background noise— Almost every topic touched on AI, but no one was giving talks solely “about AI.”

You would see it repeatedly in areas like:

Cloud native scheduling, focusing on GPU / NPU / heterogeneous resources
Storage and data, focusing on data paths for training and inference
Serverless, focusing on LLM cold starts and elasticity
Observability, focusing on what to do when system complexity gets out of hand

AI was not treated as a “hot trend,” but as a new workload reality. This is a significant change, though not one easily captured in press releases.

Real Impressions as a Cloud Native Sub-forum Producer

I helped organize the cloud native open source sub-forum at this year’s conference. This role gave me a perspective very different from that of a typical attendee.

First, Topics Clearly Converged on “Engineering Problems”

There were almost no talks about Kubernetes concepts; Very few about “architectural philosophies.”

Instead, the focus was on:

What pitfalls did you encounter at what scale?
Why did you choose this solution over another?
Which problems remain unsolved?

Many presentations weren’t “pleasant to hear,” but they were very real.

Second, The Boundary Between Academia and Industry Is Thinning

This was especially evident this year.

Some talks from universities and research institutes were no longer just “from a paper’s perspective,” but directly addressed core issues in industrial systems, such as:

Cold start of serverless LLMs
The real value of RDMA in inference paths
Whether prefill/decode separation is truly feasible in engineering

These topics may not be immediately applicable, but they are now colliding head-on with engineering problems, rather than talking past each other.

Third, Open Source Is No Longer Just About Code

In many discussions, “governance,” “maintenance cost,” and “community collaboration” came up frequently.

This is a signal: When a project is truly being used, code is no longer the hardest part.

Main Forum: More Questions, Not Answers

If I had to sum up the main forum in one sentence: It kept raising questions, but wasn’t in a hurry to provide answers.

For example:

Has the boundary of open source changed in the AI era?
Should models, data, and chips become part of the open source core?
Are developers’ roles being redefined?

There are no standard answers to these questions, but the fact that they are being raised repeatedly shows they have become common concerns, not just the thoughts of a few.

Exhibition Area and Sub-forums: Closer to the Real Ecosystem

Compared to the main forum, I personally paid more attention to the sub-forums and exhibition area.

There, you would see:

Many projects no longer emphasize “who they want to replace”
More discussions about “who they can work with”
Several communities are seriously discussing long-term maintenance, not just releasing versions

This may not be glamorous, but it’s important.

A Personal Judgment

If I had to make a judgment about COSCon'25, I would say:

Chinese open source is shifting from “can we do it” to “can we sustain it for the long term.”

This is a more difficult, but also more realistic, stage.

This COSCon did not try to create a grand narrative. Instead, it felt like a “status exposure” at a particular stage: There are more questions, participants are more diverse, but the discussions are also closer to the real world.

Open source doesn’t depend on a single conference to move forward, but being on site helps you see more clearly: Where exactly are we standing right now?

Decoding Goose: Why It Joined AAIF and What This Means for Agentic Runtime

Jimmy Song — Fri, 12 Dec 2025 08:16:48 +0000

Goose is not a project that excites you at first glance in this wave of Agent innovation, but its entry into AAIF signals a deeper shift in how we think about Agentic Runtime and AI-Native infrastructure.

At first glance, Goose is not a project that immediately excites people.

Figure 1: Goose App UI

It doesn’t have flashy demos, nor does it showcase overwhelming multimodal capabilities, and it certainly doesn’t look like an AI product aimed at consumers. Yet, this seemingly “plain” project became one of the first donations to the Agentic AI Foundation (AAIF), standing alongside Anthropic’s MCP and OpenAI’s AGENTS.md.

This fact alone is worth a closer look.

This article does not aim to prove how powerful Goose is, but rather to answer three more practical questions:

What overlooked but long-term critical problems does Goose actually solve?
Why was it Goose, and not another Agent framework, that entered AAIF?
What does this mean for Agentic Runtime and AI-Native infrastructure, which I care about?

Goose’s True Positioning: It’s Not an IDE or a Chatbot

If you only look at its surface features, Goose is easily mistaken for one of two things:

A “multi-model AI desktop client”
Or an “intelligent programming assistant that can run commands”

But inside Block, it was never designed as a “tool” from the start.

Goose’s origin is closely tied to Block’s engineering environment.

Block (formerly Square) is a classic engineering-driven company: complex systems, high automation needs, many internal tools, and very high execution costs in real production environments. In its recent AI transformation, Block did not focus on “which model to choose” or “which AI tool to introduce,” but directly targeted the engineering execution layer itself.

Goose was born in this context.

Its goal is not to “help people code faster,” but to enable models to stably and controllably take action: run tests, modify code, drive UIs, call internal systems, and operate reliably in real engineering environments.

In short:

Goose is more like an executable Agent Runtime than a conversation-centric product.

Block’s AI Transformation Started with Organization, Not Tools

To understand Goose, you can’t ignore a key organizational shift at Block.

In an interview with Block’s CTO, one signal was very clear: the starting point for AI transformation was not buying tools or stacking models, but the organizational structure itself.

Block shifted from a business-line GM model to a more functionally oriented structure, making engineering and design the company’s core scheduling units again. This is essentially a proactive response to Conway’s Law.

If the organizational structure doesn’t allow technical capabilities to be orchestrated centrally, Agents will ultimately remain “personal assistants” or “engineering toys.”

From this perspective, Goose is not just a tool, but a cultural signal:

Every employee can use AI to build and execute real system behaviors.

This also explains a fact many overlook: Goose was not packaged as SaaS, nor was it rushed to commercialization, but was open-sourced and rapidly standardized.

Because its role inside Block is closer to an “operating system for execution models” than a product that can be sold separately.

Why Did Goose Enter AAIF? Not Because It’s “Technically Strongest”

This is what confuses outsiders the most.

If you only look at flashy features, model support, or community popularity, Goose doesn’t stand out. But AAIF’s choice was not about “maximum capability,” but about whether the position is right.

Looking at the first batch of AAIF projects, a clear chain emerges:

MCP (Anthropic): Defines how models safely and standardly call tools
AGENTS.md (OpenAI): Defines behavioral conventions for Agents in code repositories
Goose (Block): A real, runnable, open-source Agent execution framework

Goose’s role is not to set new protocols, but to serve as the practical carrier and reference implementation for these protocols.

It proves one thing:

MCP is not just a paper standard
Agents are not just research concepts
In real enterprise environments, they can actually run

From this angle, Goose’s “ordinariness” is actually an advantage.

It is not tied to Block’s business moat, nor does it have irreplaceable private APIs; it can be forked, replaced, audited—“boring” enough, and neutral enough.

And that is the most important trait of public infrastructure.

Goose’s Value Lies Not in Today, But in 2–3 Years

From a longer-term perspective, Goose’s value becomes clearer.

What we’re experiencing now is much like the early days of containers: Most Agent projects today are demos, IDE plugins, or workflow wrappers, but what’s really missing is a sustainable, schedulable, observable execution layer.

Goose is already moving in this direction.

Block’s metrics for Goose’s success are straightforward:

How many human hours are saved each week
How much non-technical teams reduce their dependence on engineering teams

Behind this is a judgment I’m increasingly convinced of:

What enterprises truly need is not “smarter models,” but “cheaper execution.”

The long-term value of Agents is not in generation quality, but in execution substitution rate.

AAIF Is an Attempt at Infrastructure-Level Consensus

Just as CNCF did for cloud native, AAIF is not guaranteed to succeed.

But it at least marks a shift: Agents are no longer just application-layer innovations, but are beginning to enter the stage of infrastructure-layer collaboration.

As a reference implementation, Goose is likely to remain in this ecosystem for a long time—even if it is replaced, rewritten, or evolved in the future.

Summary

If you see Goose as a “product,” it is indeed not dazzling.

But if you place it in the long-term evolution path of Agentic AI, its significance becomes clear:

It is not the end, but a necessary intermediate state.

For me, the emergence of Goose further confirms one thing:

Agentic Runtime is not a conceptual problem, but an engineering and organizational one.

And that is one of the most worthwhile directions to invest energy in over the next few years.

ARK: Multi-Agent Systems Are Finally Entering the Engineer's World

Jimmy Song — Thu, 11 Dec 2025 13:19:42 +0000

The deep integration of cloud native and AI, with the ARK platform, provides a new paradigm for engineering multi-agent systems.

Introduction

AI Agents are moving from the “single agent demo” stage to “large-scale operation.” The real challenge does not lie in the model itself, but in engineering issues at runtime: model management, tool invocation, state maintenance, elastic scaling, team collaboration, observability, deployment, and upgrades. These are problems that traditional agent libraries struggle to solve.

ARK (Agentic Runtime for Kubernetes) provides a fully operational, observable, governable, and continuously deliverable multi-agent operating system. It is not a Python library, but a complete runtime platform.

Figure 1: ARK Dashboard

Note: In this article, ARK refers to McKinsey’s open-source ARK Agent Runtime for Kubernetes.

This article, from an engineer’s perspective, will reorganize ARK’s core capabilities and answer the following questions:

What engineering challenges does ARK actually solve?
Why is it worth special attention in the cloud native field?
How is it fundamentally different from frameworks like LangChain and CrewAI?
What insights does it offer for the Agentic Runtime ecosystem?

ARK Architecture: Treating Agents as Kubernetes-Native Workloads

The core idea of ARK is: An agent is not a script, but a schedulable, governable, and observable Kubernetes workload.

The following architecture diagram illustrates ARK’s underlying structure.

Figure 2: ARK Overall Architecture

This diagram highlights ARK’s key design points:

CRDs declare requirements (Agent, Model, Team, Tool, Memory, etc.)
The Controller translates declarations into actual Pods/Services
The API provides a unified communication entry point and team orchestration
Memory supports long-term state management for agents
MCP Server enables external systems to become tools
Dashboard provides visual management and observability

ARK adopts the typical cloud-native Operator pattern and applies it to multi-agent systems.

CRD: ARK’s “Abstraction Layer”

Unlike traditional agent frameworks where “code is logic,” ARK uses CRDs (Custom Resource Definitions) to abstract the components of agent applications.

The main CRD types in ARK include:

Model
Agent
Team
Tool
Memory
Evaluation

These CRDs correspond to all the key components of an agent system.

The following diagram shows the structure of the CRDs:

Figure 3: CRD Structure (Simplified)

Through CRDs, ARK achieves the following engineering features:

All resources are GitOps-ready, supporting declarative management
Changes are auditable, reversible, and continuously deliverable
The evolution of models, tools, and agents does not require business code changes

This is the key gene of ARK’s engineering-oriented system.

Agent Execution Flow: From Query to Tool Invocation

The following image shows how to view query details in the ARK Dashboard.

Figure 4: Viewing Query Details in ARK Dashboard

In ARK, the complete execution flow for an agent receiving a query is as follows:

Figure 5: Agent Execution Flow

This flow has the following characteristics:

Memory modules are naturally involved in the execution flow, without code specialization
Large language model (LLM, Large Language Model) and tool invocation are governed by the runtime
Agents can reside in Pods long-term, not just as one-off processes

This makes ARK more like an “agent microservice platform.”

Below is an example of a request and response:

The True Value of Multi-Agent: Team Orchestration

ARK’s Team CRD allows multiple agents to be woven into a higher-level “system,” enabling multi-agent collaboration.

The following diagram shows the collaboration model of a multi-agent team:

Figure 6: Multi-Agent Team Collaboration

The engineering value of Team is reflected in:

Making “expert collaboration” declarative and configurable
Flexible strategies (such as polling, role assignment, routing, etc.)
A2A Gateway handles message passing
The Team itself is observable (every round of collaboration is logged)

For enterprises, this means the “agent organizational structure” can be standardized, replayed, and tuned.

Fundamental Differences Between ARK and Other Frameworks

Many engineers, upon first seeing ARK, may wonder:

“Is it just LangChain or CrewAI wrapped in Kubernetes?”

In fact, there are fundamental differences. The following diagram compares the structural differences between ARK and mainstream agent frameworks:

Figure 7: ARK vs LangChain / AutoGPT / CrewAI

The table below further summarizes the key differences:

Dimension	Traditional Agent Libraries	ARK
Core Pattern	Write Python code	Write CRDs (declarative)
Deployment	Local/Container	Kubernetes-native scheduling
State	Managed inside code	Memory CR + Service
Tools	Integrated at code level	Tool CR + MCP
Multi-Agent	Dialog managed in code	Team CR + A2A protocol
Observability	Almost none	OTel / Langfuse / Dashboard
Use Cases	Demo / Prototype / Single Agent	Enterprise production / Multi-Agent Systems

Table 1: ARK vs Traditional Agent Libraries

In short:

LangChain is a “library for building agents,” while ARK is a “platform for running agents.”

The two are not in conflict and are, in fact, highly complementary.

The Engineering Value of ARK

To summarize ARK’s engineering value in simple terms:

Turns agents into governable workloads
Unifies models, tools, and memory as reusable resources
Makes multi-agent collaboration structured, observable, and tunable
Brings agent upgrades and iteration into CI/CD + GitOps mode
Enables enterprises to manage agents like microservices

This is a clear evolution path:

Agent → Service → Platform → Runtime → Operating System

ARK is currently positioned at the fourth stage: Runtime.

Insights for Agentic Runtime

ARK provides three direct insights for building Agentic Runtimes:

Unified Scheduling System

The agent runtime must run on a unified scheduling system (Kubernetes, MicroVM, Wasmtime, etc.)

Declarative Capability Boundaries

Must use declarative abstractions to split capability boundaries, including:
- Model Layer
- Tool Layer
- Memory Layer
- Workflow Layer
- Team Layer
- State Layer

Observability

Observability is essential; otherwise, multi-agent systems cannot be engineered
- Langfuse
- OTel
- Logs / Events
- Structured JSON

ARK demonstrates a direction:

Multi-agent systems are an engineering problem, not a prompt engineering problem.

Summary

If you only need to build a simple agent, frameworks like LangChain, CrewAI, and AutoGPT are sufficient.

But if you want to operate a system composed of dozens or hundreds of agents that need to collaborate, run long-term, and support continuous delivery and governance, runtimes like ARK are the inevitable trend.

It provides Agentic AI with:

A cloud-native runtime model
Observable execution paths
Governable abstraction layers
Extensible, componentized architecture

Therefore, ARK deserves to be regarded as an early model for engineering multi-agent systems.

Can Open Source Suddenly Disappear? An AI Chat Dev Tool Went 404 Overnight

Jimmy Song — Thu, 11 Dec 2025 05:20:12 +0000

“Open source” in the AI era is no longer a trustworthy promise. Commercial projects can withdraw their code at any time, and developers must be wary of the gap between appearances and reality.

The Disappearance of Lunary’s Repository: A Real Case of Open Source “Vanishing”

While updating the AI open source project library on my website, I encountered a situation that left me stunned for the first time: An “open-source AI tool” that still promotes itself, with an active website and commercial services, suddenly vanished from GitHub—its repository went straight to 404.

The project is called Lunary.

Original repository address: https://github.com/lunary-ai/lunary It now returns a 404 Not Found.

Notably, the official site lunary.ai remains online, but the core promise of an “open-source codebase” has disappeared.

Lunary’s Positioning and Features

Here is an overview of Lunary’s main features and positioning to help understand its role in the AI tool ecosystem.

Lunary claims to be an Observability and Evaluations platform for large language model (LLM, Large Language Model) applications, focusing on:

LLM conversation and feedback logs
Cost, latency, and metrics analysis
Prompt version management
Distributed tracing
Evaluations
Supports both self-hosted and managed modes
Provides JS / Python SDKs, integrates with LangChain

Its overall positioning is clear: “Development and debugging tools for AI applications.”

In fact, products like this have emerged rapidly over the past year, forming a new AI DevTool track.

The Reality and Risks Behind the “Open Source” Label

The core issue is not the tool itself, but its claim to be “open source.”

Lunary has consistently emphasized:

“Lunary is an open-source platform for developers.”

This statement is great for attracting users, as open source implies transparency, trustworthiness, self-hosting, and community participation.

But now the repository is gone, with only the website continuing its promotion—raising many questions.

Lunary is not a niche hobby project, but a commercial company-led initiative. If an individual suddenly deletes a repo, it’s not surprising, but for a company operating publicly, this move is extremely rare.

This is the first time I’ve truly seen a reality in the AI DevTools space: “Open source” is being used as a branding term, not a commitment.

Possible Industry Reasons for Repo Deletion

Let’s analyze some common industry reasons for deleting a repository to help developers understand the motivations behind such actions.

Increased commercial pressure: These tools often struggle with sustainable business models, prompting teams to shift to closed-source SaaS.
Pivoting: The company finds the original direction unprofitable and prepares to change course.
Team changes: Acquisition, key member departures, or funding issues can all lead to repo shutdowns.
Compliance or legal risks: Observability products involve user data, which may require public code to be taken down.

Regardless of the reason, the impact on users is the same: it is no longer an “open-source product.”

The “Pseudo Open Source” Phenomenon in AI Tools

The most noteworthy aspect is not Lunary itself, but the rapid spread of this phenomenon in the AI tool space.

Many projects use “open source” as a user acquisition strategy but lack open governance and long-term commitment.

High substitutability, homogeneity, and commercial pressure mean these DevTools have low survival rates.

When commercial teams lead open source, a single decision can make the repository disappear instantly.

In the cloud native era, we’ve already seen a wave of “pseudo open source.” In the AI era, this trend is accelerating.

Three Practical Lessons for Developers

Based on this case, here are three practical lessons for developers:

The “open source label” does not guarantee trustworthiness: Open source projects led by commercial companies without community or foundation backing can be withdrawn at any time.
AI DevTools are far less stable than infrastructure: These tools are not essential, highly replaceable, and have short lifecycles.
Tool usability should take precedence over “open source status”: Because it may stop being open source at any moment.

My First Experience Maintaining an AI Project List and Facing Repo Deletion

After collecting hundreds of projects over the past two years, this is the first time I’ve encountered a “commercial open source project disappearing, official repo 404” case.

To me, this is an industry signal: the AI open source world is entering a period of drift, and commercial projects’ open source commitments are increasingly unstable.

It also reminds everyone making technical choices: in the AI era, open source is no longer a label you can automatically trust.

Summary

The disappearance of the Lunary repository is not an isolated incident, but a reflection of the “pseudo open source” phenomenon in the AI tool space. Developers should be cautious about the actual commitments behind the “open source” label, paying attention to project governance and sustainability. In the future, the boundary between open source and commercial will become even more blurred, and rational judgment and risk awareness will be essential for technical decision-making.

Lunary’s sudden disappearance highlights the instability of open source projects in the AI DevTools space. For developers, technical choices should focus more on project usability and community governance, rather than relying solely on the “open source” label. As the industry evolves, similar incidents may become more frequent. Only rational judgment and risk awareness can help you stand firm in the fast-changing tech landscape.

CNCF in the AI Native Era? The Agentic AI Foundation Is Officially Established

Jimmy Song — Wed, 10 Dec 2025 03:25:38 +0000

The standardization and open collaboration of the agent ecosystem is no longer a luxury, but the critical watershed for whether AI Native can be engineered and implemented.

The establishment of AAIF (Agentic AI Foundation) is the result of leading vendors staking out the “agent protocol layer” in advance.
The real challenge is not technical, but how organizations transition from “human execution + AI assistance” to “agent execution + human supervision”.
Successful agent adoption requires a phased adoption path, not just a bunch of protocols and demos.
CNCF and AAIF are complementary: CNCF manages “what infrastructure agents run on”, AAIF manages “how agents collaborate”. This matches the system I am building in ArkSphere.

Cloud Native Problems Are Solved, AI Native Problems Are Just Beginning

Over the past decade, Cloud Native technologies like Kubernetes, Service Mesh, and microservices have standardized “how applications run in the cloud”. But AI Native faces a completely different challenge: It’s not about “how to deploy a service”, but “how many behaviors in the system can be handed over to agents to execute themselves”.

CNCF’s Cloud Native AI (CNAI) addresses infrastructure-level issues: “How can model training/inference/RAG run at scale and securely on Kubernetes?”

But what AI Native truly lacks is another layer: How do agents collaborate, access tools, get governed, and audited?

This is exactly the gap AAIF aims to fill.

AAIF’s Three Weapons: Protocol + Runtime + Development Standard

AAIF hosts three core technologies contributed by its founding members:

① Anthropic’s Model Context Protocol (MCP)

https://github.com/modelcontextprotocol

A “system call interface for agents”:

Unified definition for how agents access databases, APIs, files, and external tools.
Designed to be more like an AI version of gRPC + OAuth.
Already integrated by Claude, Cursor, ChatGPT, VS Code, Microsoft Copilot, Gemini, and others.

It may not be the flashiest technology, but it could become the plumbing for the entire Agentic ecosystem.

② Block’s Goose Framework

https://github.com/block/goose

Reference runtime for MCP:

Local-first, composable agent workflow engine.
Enables enterprises to pilot agents in small scopes without betting on a specific vendor.
Serves as an engineering template for protocol implementation.

③ OpenAI’s AGENTS.md

https://agents.md

A simple but effective standard:

Place an AGENTS.md file in the project repository.
Clearly document build steps, testing, constraints, and context rules.
Any agent that understands AGENTS.md can operate the codebase using the same instructions.

This makes agent behavior more predictable and auditable.

Why Is AAIF in Such a Hurry? This Is a Race for Standards

Let’s compare with history:

Kubernetes’ predecessor Borg ran internally at Google for over a decade; K8s was open sourced and donated to CNCF two years later.
PyTorch joined the Linux Foundation six years after its release.
MCP was donated to AAIF just over one year after its launch.

AAIF is not about “mature technology entering a foundation”, but staking out the key position early.

The reasons are practical:

Prevent agent ecosystem fragmentation Today, there are many competing “tool invocation protocols”, which could become incompatible silos in three years.
Protocol layer is easier to reach global consensus than model layer Model competition is inevitable, but protocols can be standardized, open sourced, and avoid vendor lock-in.
A necessary move in global tech competition Putting the foundational standards for Agentic AI into the Linux Foundation is both a gesture of cooperation and a strategic move.

AAIF vs CNCF: Not Competition, But Two Pieces of the Puzzle

CNCF’s role:

“What infrastructure do agent workloads run on?” Kubernetes, Service Mesh, observability, AI Gateway, RAG Infra—all at this layer.

AAIF’s role:

“How do agents collaborate, invoke tools, and get governed?” Protocols, runtimes, and behavioral standards—all at this layer.

Analogy:

Domain	Responsibilities
AAIF	Semantic and collaboration layer of Agentic Runtime
CNCF/CNAI	Resource and execution layer of AI Native Infra

Table 1: AAIF vs CNCF Comparison

This matches the upper semantic and lower infrastructure layers in my ArkSphere architecture diagram.

In the long run, the two sides will be tightly coupled: CNCF’s KServe, KAgent, and AI Gateway will natively support MCP / AGENTS.md, AAIF’s Runtime will run on Cloud Native infrastructure by default.

The Real Challenge: Not Protocols, But Organizations and People

Most enterprises will get stuck on:

How much responsibility can agents actually take?
Who is accountable when things go wrong?
How are audit, SLOs, and compliance defined?
How is multi-agent collaboration visualized?
How are tool invocation permissions controlled?

In other words, agent adoption is not a “technical migration”, but an “organizational migration”.

If AAIF cannot provide:

Phased adoption methodologies
Typical organizational migration paths
Engineering best practices
Failure cases and anti-patterns

It will be difficult for AAIF to achieve the industry impact that CNCF did.

Summary: AAIF Is the Moment When Boundaries Are Drawn

For me, the establishment of AAIF feels like:

“The battlefield boundaries of the agent world have finally been drawn. Now it’s up to the engineering community to make it work.”

CNCF solved “how to run Cloud Native”, AAIF is now trying to solve “how agents collaborate”.

In the next five years, whoever can truly connect these two worlds will stand at the gateway to the next generation of infrastructure.

That’s why I started a dedicated “Agentic Runtime + AI Native Infra” research track in ArkSphere.

The ‘Three-Body’ Architecture of the AI Native Era

Finally, a personal note—my thoughts on ArkSphere.

Figure 1: AAIF × CNCF: Three-Layer Architecture of Agentic AI in the AI Native Era

This diagram shows the three-layer structure of the AI Native era:

CNCF (bottom layer): Provides the Cloud Native foundation required for agent operation, including Kubernetes, Service Mesh, GPU scheduling, and security systems.
AAIF (middle layer): Defines the runtime semantics and standards for agents, including the MCP protocol, Goose reference runtime, and AGENTS.md behavioral standard.
ArkSphere (bridging layer): Aligns the “Agentic Runtime semantic layer” with the “AI Native Infra infrastructure layer”, forming an engineerable agent architecture standard.

In short:

Infra is responsible for “running”, Runtime for “how to act”, and ArkSphere for “how to assemble a system”.

Bun Acquired by Anthropic: A Structural Signal for AI-Native Runtimes

Jimmy Song — Wed, 03 Dec 2025 05:21:28 +0000

The shifting ownership of runtimes is reshaping the underlying logic of AI programming and infrastructure.

After the announcement of Bun’s acquisition by Anthropic, my focus was not on the deal itself, but on the structural signal it revealed: general-purpose language runtimes are now being drawn into the path dependencies of AI programming systems. This is not just “a JS project finding a home,” but “the first time a language runtime has been actively integrated into the unified engineering system of a leading large model company.”

This event deserves a deeper analysis.

Bun’s Engineering Features and Current Status

Before examining Bun’s industry significance, let’s outline its runtime characteristics. The following list summarizes Bun’s main engineering capabilities:

High-performance JavaScript/TypeScript runtime
Built-in bundler, test framework, and package manager
Single-file executable
Extremely fast cold start
Node compatibility without Node’s legacy dependencies

These capabilities have formed measurable performance barriers.

However, it should be noted that Bun currently lacks the core attributes of an AI Runtime, including:

Permission model
Tool isolation
Capability declaration protocol
Execution semantics understandable by models
Sandbox execution environment

Therefore, Bun’s “AI Native” properties have not yet been established, but Anthropic’s acquisition provides an opportunity for it to evolve in this direction.

The Significance of a Leading Model Company Acquiring a General-Purpose Runtime

Historically, it is not uncommon for model companies to acquire editors, plugins, or IDEs, but in known public cases, mainstream large model vendors have never directly acquired a mature general-purpose language runtime. Bun × Anthropic is the first clear event pulling the runtime into the AI programming system landscape. This move sends two engineering-level signals:

The speed of AI code generation continues to increase, amplifying the need for deterministic execution environments. The generate→execute→validate→destroy cycle intensifies the problem of environment non-repeatability.
Models require a “controllable execution substrate” rather than a traditional operating system. Agents are not suited to run tools in an uncontrollable, unpredictable OS layer.
The runtime needs to be embedded into the model’s internal engineering pipeline. Future IDEs, agents, and auto-repair pipelines may directly invoke the runtime’s API.

This is not a short-term business integration, but a manifestation of the trend toward compressed engineering pipelines.

Runtime Requirements Differentiation in the AI Coding Era

Based on observations of agentic runtimes over the past year, runtime requirements in the AI coding era are diverging. The following list summarizes the main engineering abstractions trending in this space:

Determinism: AI-generated code is not reviewed line by line; execution results must be consistent across machines and over time.
Minimal distribution unit: Users no longer install language environments and numerous dependencies. Verifiable, replicable, and portable single execution units are becoming the norm.
Tool isolation: Models cannot directly access all OS capabilities; the context and permissions visible to tools must be strictly defined.
Short-lived execution: Agent invocation patterns resemble “batch jobs” rather than long-running services.
Capability declaration: The runtime must expose “what I can do,” rather than the entire OS interface.
Embeddable self-testing pipeline: After generating code, models need to immediately execute tests, collect errors, and iterate. The runtime must provide observability and diagnostic primitives.

These requirements are not unique to Bun, nor did Bun originate them, but Bun’s “monolithic and controllable” runtime structure is more conducive to evolving in this direction.

Figure 1: Minimal execution loop of an AI-native runtime

Bun’s Potential Role Within Anthropic’s System

If Bun is seen merely as a Node.js replacement, the acquisition is of limited significance. But if it is viewed as the execution foundation for future AI coding systems, the logic becomes clearer:

Code is generated by models
Building is handled by the runtime’s built-in toolchain
Testing, validation, and repair are performed by models repeatedly invoking the runtime
All execution behaviors are defined by the runtime’s semantics
The runtime forms Anthropic’s internal “minimal stable layer”

This model is similar to the relationship between Chrome and V8: the execution engine and upper-layer system co-evolve over time, with performance and semantics advancing in sync.

Whether Bun can fulfill this role depends on Anthropic’s architectural choices, but the event itself has opened up possibilities in this direction.

Industry Trends and Future Evolution

Combining facts, signals, and engineering trends, the following directions can be anticipated:

The “Agent Runtime” category will gradually become more defined
The boundaries between bundler, runtime, and test runner will continue to blur
Cloud vendors will launch controllable runtimes with capability declarations
Permission models and secure sandboxes will move down to the language runtime layer
Runtimes will become part of the model toolchain, rather than an external environment

These trends will not all materialize in the short term, but they represent the inevitable path of engineering evolution.

Summary

The combination of Bun × Anthropic is not about “an open-source project being absorbed,” but about a language runtime being actively integrated into the engineering pipeline of a large model system for the first time. Competition at the model layer will continue, but what truly reshapes software is the structural transformation of AI-native runtimes. This is a foundational change worth long-term attention.

Agentic Runtime Realism: Insights from McKinsey Ark on 2026 Infrastructure Trends

Jimmy Song — Tue, 02 Dec 2025 12:07:45 +0000

Statement

ArkSphere has no affiliation or association with McKinsey Ark.

The value of Agentic Runtime lies not in unified interfaces, but in semantic governance and the transformation of engineering paradigms. Ark is just a reflection of the trend; the future belongs to governable Agentic Workloads.

Recently, the ArkSphere community has been focusing on McKinsey’s open-source Ark (Agentic Runtime for Kubernetes). Although the project is still in technical preview, its architecture and semantic model have already become key indicators for the direction of AI Infra in 2026.

This article analyzes the engineering paradigm and semantic model of Ark, highlighting its industry implications. It avoids repeating the reasons for the failure of unified model APIs and generic infrastructure logic, instead focusing on the unique perspective of the ArkSphere community.

Ark’s Semantic Model and Engineering Paradigm

Ark’s greatest value is in making Agents first-class citizens in Kubernetes, achieving closed-loop tasks through CRD (Custom Resource Definition) and controllers (Reconcilers). This semantic abstraction not only enhances governance capabilities but also aligns closely with the Agentic Runtime strategies of major cloud providers.

Ark’s main resources include:

Agent (inference entity)
Model (model selection and configuration)
Tools (capability plugins/MCP, Model Capability Plugin)
Team (multi-agent collaboration)
Query (task lifecycle)
Evaluation (assessment)

The diagram below illustrates the semantic relationships in Agentic Runtime:

Figure 1: Agentic Runtime Semantic Relationships

Architecture and Community Activity

Ark’s architecture adopts a standard control plane system, emphasizing unified runtime semantics. The community is highly active, engineer-driven, and the codebase is well-structured, though production readiness is still being improved.

Figure 2: Ark Architecture and Control Plane Flow

ArkSphere’s Boundaries and Inspirations

The emergence of Ark has clarified the boundaries of ArkSphere. ArkSphere does not aim for unified model interfaces, multi-cloud abstraction, a collection of miscellaneous tools, or a comprehensive framework layer. Instead, it focuses on:

The semantic system of Agentic Runtime (tasks, states, tool invocation, collaboration graphs, etc.)
Enterprise-grade runtime governance models (permissions, auditing, isolation, multi-tenancy, compliance, cost tracking)
Integration capabilities for domestic ecosystem tools
Engineering paradigms from a runtime perspective

ArkSphere is an ecosystem and engineering system at the runtime level, not a “model abstraction layer” or an “agent development framework.”

Key Changes in 2026

2026 will usher in the era of Agentic Runtime, where Agents are no longer just classes but workloads that require governance rather than mere importation. Ark is just one example of this trend, and the direction is clear:

Semantic models and governability become highlights
Closed-loop tasks are the core value

Summary

Ark’s realism teaches us that the future belongs to runtime, semantics, governability, and workload-level Agents. The industry will no longer pursue unified APIs or framework implementations, but will focus on governable runtime semantics and engineering paradigms.

In-Depth Analysis of Ark: Kubernetes for the AI Era or a New Engineering Paradigm Shift?

Jimmy Song — Tue, 02 Dec 2025 10:54:34 +0000

Statement

ArkSphere has no affiliation or association with McKinsey Ark.

The greatest value of Ark lies in reshaping engineering paradigms, not just its features. It points the way for AI Infra and leaves vast space for community ecosystems.

Recently, many members in our ArkSphere community have started exploring McKinsey’s open-source Ark (Agentic Runtime for Kubernetes).

Some see it as radical, some think it’s just a consulting firm’s experiment, and others quote a realistic maxim:

What we need now is “agentic runtime realism,” not “unified model romanticism.”

I strongly agree with this sentiment.

I’ve spent some time analyzing Ark’s source code, architecture, and design philosophy, combined with our community discussions. My conclusion is:

Ark’s significance is not in its features, but in its paradigm.

It’s not the answer, but it points toward the future of AI Infra.

Below is my interpretation of Ark, focusing on engineering, architecture, trends, and its inspiration for ArkSphere.

What Exactly Is Ark?

Ark’s core positioning is: A runtime that treats Agents as Kubernetes Workloads.

It’s not a framework, not an SDK, not an AutoGen-style multi-agent tool, but a complete system including:

Control plane (Controller)
Custom resource models (CRD, Custom Resource Definition)
API service
Dashboard
CLI
Python SDK

Essentially, Ark is the control plane for Agents.

Ark defines seven core CRDs in Kubernetes. The following flowchart shows the relationships among these resources:

Figure 1: Ark CRD Resource Relationships

Through this set of CRDs, Ark makes Agent systems resource-oriented and declarative, enabling capabilities such as:

Lifecycle management
Multi-tenant isolation
RBAC (Role-Based Access Control)
Observability
Upgradability
Extensibility (tools, models, MCP)

In other words, Ark is not about “how to write Agents,” but “how to operate Agents in enterprise-grade systems.”

Three-Layer Architecture: Mixed Languages and Components, but a Complete System

Ark’s overall architecture is divided into three layers, each with different tech stacks and responsibilities. The following flowchart illustrates the relationships among components in each layer:

Figure 2: Ark Three-Layer Architecture Components

This is not a “wrapper project,” but a fully operational AI Runtime system, with a level of engineering far beyond most agent frameworks on the market.

Is It the Kubernetes of the AI Era?

Let’s revisit Kubernetes’ core value:

Kubernetes was never about “unifying cloud APIs”; it unified the “application runtime model.”

Cloud provider APIs aren’t unified, nor are networking or storage. What is unified: Pod, Deployment, Service—these application models.

Kubernetes succeeded because:

It provides a stable application abstraction on top of diversity.

Ark’s goal is not to unify all large language models (LLMs), MCPs, or tool formats, but rather:

Agent resource model (CRD) + control plane (Reconciler) + lifecycle.

From this perspective, Ark offers a prototype of a “declarative application model” for the AI era.

Whether it will become “Kubernetes for AI” is still too early to say, but it has already planted a seed.

Comparison with Other Frameworks: Not on the Same Level

Current mainstream agent frameworks like LangChain, CrewAI, AutoGen, MetaGPT, etc., address problems fundamentally different from Ark.

The table below compares the positioning and limitations of each framework:

Name	What Problem Does It Solve	Core Limitation
LangChain	Agent/Tool composition	Doesn’t address deployment or governance
AutoGen	Multi-agent conversations	Lacks control plane and lifecycle
CrewAI	Workflow-style multi-agent	Missing scheduling, RBAC, resource model
MetaGPT	Agent SOP	Just execution logic, not a platform
OpenDevin	AI IDE/Dev Assistant	Not an Agent Runtime
Ark	Agent control plane + resource system	Functionality not yet mature

Table 1: Mainstream Agent Frameworks vs. Ark

In short:

Other tools focus on “how to write Agents.”
Ark focuses on “how Agents run, schedule, govern, observe, and extend.”

That’s an architectural difference.

Execution Flow: Agents Scheduled Like Pods

Ark’s execution flow closely resembles the Kubernetes controller model. The following sequence diagram shows the core process:

Figure 3: Ark Agent Execution Flow

You can see Ark’s process logic is transparent, with a clear engineering path, bringing agent systems into a “controllable” state.

Production Readiness: Right Direction, Still a Tech Preview

According to official notes and code maturity, Ark currently offers:

Runnable
Learnable
Extensible
But not recommended for large-scale production use yet

Main reasons include:

CRD structures may change
APIs are not yet stable
MCP ecosystem is still forming
Memory service is still basic
Multi-agent team execution strategies are primitive

But the engineering system is already taking shape, which is crucial.

Community Activity: Small but Elite, Strong McKinsey Drive

From GitHub data:

Stars: 222
Forks: 50
Contributors: 48
Commit frequency is steady
The vast majority of contributions come from within McKinsey

Note: Data as of December 2, 2025.

High stability, but limited openness.

This is also ArkSphere’s opportunity:

The paradigm is right, but the ecosystem needs community-driven growth.

Trends for 2026: From Framework Era to Runtime Era

After deep analysis, I’m increasingly convinced:

2023–2024: Large model API call era
2024–2025: Agent framework era
2025–2027: Agent Runtime / Control Plane era (Ark’s direction)

While everyone is writing Python scripts for agents, the real value lies in:

Multi-agent task scheduling
Tool registration and governance
Session/Memory lifecycle
Result reproducibility
RBAC, auditing, tenant isolation
Observability
Enterprise internal personalized agent systems

Ark is providing a practical path forward.

Inspiration for ArkSphere

Ark’s inspiration for ArkSphere is both critical and direct:

ArkSphere Should Focus on “Paradigm Building,” Not “Feature Stacking”

Ark offers a prototype for future Agentic Runtime:

Resource model
Control plane
Tool registration
Multi-agent collaboration
Evaluation and governance

ArkSphere’s role should be:

Aggregate paradigms, produce standards, incubate ecosystems, not rewrite Ark itself.

This is the “CNCF (Cloud Native Computing Foundation) for the AI-native era.”

Huge Potential for Localization in China

Localization opportunities include but are not limited to:

Integration with domestic large language models (e.g., Qwen, DeepSeek, Zhipu)
Enterprise privatization scenarios
Local tool/MCP discovery ecosystem
Multi-cluster/edge inference
Enterprise-grade RBAC, auditing, data isolation
AgentSpec enhancements for industrial scenarios
Enhanced versions of Runtime/Controller

In other words:

Ark solves the “model,” while ArkSphere can solve the “ecosystem.”

What We Need Is Not “Kubernetes for the LLM Era,” But an “Industry-Grade Cognition System for AI Runtime”

The biggest takeaway from dissecting Ark:

The future of AI-native is not a pile of tools, but an engineering system.

ArkSphere can be the initiator of this system.

Summary

Ark is not a “universal runtime,” nor is it the “ultimate Kubernetes for the AI era.”

But it has done one crucial thing right:

It abstracts all the pain points people faced when writing Python agent scripts into Kubernetes resources and controllers.

It represents engineering, not just a demo.

It’s not mature yet, but it’s heading in the right direction.

It’s not the end, but it gives us a clear roadmap.

For the ArkSphere community I’m running, Ark provides a clear inspiration:

The future belongs to Runtime, to Control Plane, to governable agent systems.

And the ones who can truly scale this system are not McKinsey, but the community.

From Using AI to Relying on AI: Why the Era of AI Engineering Has Yet to Begin

Jimmy Song — Sat, 29 Nov 2025 12:40:54 +0000

The real inflection point for AI engineering is not “how many people use it,” but “how many people cannot do without it.” Only when not using AI leads to direct loss of opportunity and efficiency, can we say the era of AI engineering has truly arrived.

Starting Point: Predictions for AI in 2026

Recently, I came across two predictions for 2026 from Amazon CTO Werner Vogels that struck me the most:

Renaissance Developer: Developers must span code, product, business, and social impact.
Personalized Learning: AI will reshape education, focusing on differentiated paths rather than a unified curriculum.

Both point to the same trend: AI is not just a tool, but is redefining how people grow and how they are defined.

There is a gap between prediction and reality, and it is worth exploring.

Correction: Will AI Really Be “Saturated” by 2026?

My initial prediction was that AI usage would reach saturation by 2026. Reality has shown me this is too optimistic.

By the end of 2025, even among internet professionals, most people’s use of AI remains at the “heard of it” or “tried it a few times” stage. It is still far from being a daily workflow necessity.

More importantly, this judgment is conditional: infrastructure supply, regulation, and compute costs must not reverse in the next 3–6 years. If any variable breaks down (costs double, models go offline, policy shifts), the adoption curve will be disrupted.

The Truth About the Inflection Point: From “Using” to “Relying On”

“Relying on” is a vague term. A more precise definition requires measurable indicators.

Here is a diagram that visualizes the metrics for being truly dependent on AI:

This diagram visualizes the quantitative metrics for being truly dependent on AI, comparing target thresholds with current status:

Figure 1: Quantitative Definition of AI Dependency

Most industries have not reached the “cannot operate without” stage, unlike the internet, mobile, or payment inflection points. Most metrics are still far below the threshold, which is why the most likely outcome for 2026 is: more people will use AI, but those who truly rely on it will remain a minority.

Using ≠ Building: The Five-Level Capability Ladder

This difference is not binary, but a clear progression.

The following table shows the five-level model of AI capability maturity.

Level	Name	Description	Scarcity
1	Tool User	ChatGPT/Claude, Coding, Copywriting, Accelerator, Optional	Low
2	Integrator	LLM API + Vector DB, AI layered on existing systems, Usable, not critical	Low
3	Settler	Restructuring data flow, business decisions, AI becomes critical path	Rising
4	Engineering Abstraction	Extracting frameworks, runtimes, providing infra for ecosystem	Extremely High
5	Autonomous System	Self-feedback, self-optimizing, redefining human-AI relationship	Future

Table 1: Five-Level Model of AI Capability Maturity

Currently, the biggest gap is at Level 3 and Level 4. Most people are stuck at Level 1 or 2, with very few reaching Level 4. This means high-value scarcity will not disappear, but will continue to rise.

Why the Era of AI Engineering Has Not Arrived: Three-Dimensional Delaying Factors

It is not technology alone that is holding things back, but constraints in three dimensions.

The following diagram illustrates the three main constraints delaying AI engineering maturity:

This diagram illustrates the three main constraints (technical, institutional, and organizational) that are delaying AI engineering maturity, along with their delay metrics:

Figure 2: Three-Dimensional Constraints on AI Engineering Maturity

The key observation: If any one dimension is stuck, the entire ecosystem’s maturity will be delayed. Currently, none of the three dimensions have fully mature solutions.

The Realistic Window: Three Paths for Capability Advancement

The next three years will not be “winner takes all,” but rather a period where multiple capability levels appreciate simultaneously.

Below is a table comparing the value and bottlenecks of different capability advancement paths:

Capability Path	Short-Term Value	Long-Term Outlook	Bottleneck
Level 1→2 (Tool→Integration)	⭐⭐ Rapid Depreciation	⭐ Saturation	Low barrier, fierce competition
Level 2→3 (Integration→Settlement)	⭐⭐⭐⭐ Scarce	⭐⭐⭐⭐ Continual Appreciation	Requires industry depth, long-term iteration
Level 3→4 (Settlement→Abstraction)	⭐⭐⭐⭐⭐ Extremely Scarce	⭐⭐⭐⭐⭐ Defines Ecosystem	Large cognitive leap, needs community influence

Table 2: AI Capability Advancement Paths and Value Comparison

Key conclusion: While the number of “AI users” is rapidly increasing (depressing Level 1 value), due to the three-dimensional delaying factors, scarcity at Level 3 and 4 will only rise.

What I’m Doing on arksphere.dev

Based on the above judgment, I focus on exploring the architectural evolution of AI Native Infrastructure. The goal is not to catalog model usage, but to study the foundational capability stack supporting scalable intelligent systems: scheduling, storage, inference, Agent Runtime, autonomous control, observability, and reliability.

The content is no longer a collection of courses or tips, but a continuous record of evolution around Infra → Runtime → System Abstraction. arksphere.dev is the site for this experiment and settlement.

Summary

The inflection point for the era of AI engineering is not “how many people use it,” but “how many people cannot do without it.” The latter requires five measurable indicators to reach their thresholds, and we are still far from that.

“Using ≠ Building” is not a binary, but a five-level progression. Scarcity at Level 3 and 4 will rise as the number of Level 1 users increases—this is the biggest opportunity window in the next three years.

But the width of this window depends largely on how technology, institutions, and organizations evolve together. I hope more people working on AI engineering will not only focus on technical innovation, but also invest equal thought into institutional development, talent growth, and risk governance—these “invisible engineering” challenges.

Antigravity VS Code Setup Guide: Build a Practical AI IDE Workflow

Jimmy Song — Thu, 20 Nov 2025 03:55:30 +0000

The biggest pain point when switching IDEs is user habits. By installing a series of plugins and tweaking configurations, you can make Antigravity feel much more like VS Code—preserving familiar workflows while adding Open Agent Manager capabilities.

If you searched for a practical Antigravity VS Code setup, this walkthrough is optimized for that exact use case. The goal is not to replicate VS Code pixel by pixel, but to restore a familiar extension marketplace, keep your daily coding ergonomics, and still use Antigravity’s stronger agent-style execution. I focus on the concrete setup steps that materially change productivity: marketplace migration, AMP and CodeX installation, editor behavior alignment, and the trade-offs versus GitHub Copilot in real daily work.

Figure 1: Antigravity IDE UI

Continue Reading

Below are the configurations and steps I actually use. Feel free to follow along.

First Impressions of Antigravity

A few subjective observations:

The interface is split between agent management and editor views, somewhat like AgentHQ + VS Code.
Agents modify code very quickly, with a much higher completion rate than typical “chat-based” assistants.
The editor and context windows are large, ideal for long diffs and logs.
By default, it uses OpenVSX / OpenVSCode Gallery, so the extension ecosystem isn’t identical to my VS Code setup.

All subsequent steps focus on one goal: keep Antigravity’s agent features while maintaining my VS Code workflow.

Switching the Extension Marketplace to VS Code Official

Antigravity is essentially a VS Code fork, so you can directly change the Marketplace configuration.

In Antigravity:

Go to Settings -> Antigravity Settings -> Editor, and update the following URLs to point to VS Code:

Marketplace Item URL:

https://marketplace.visualstudio.com/items

Marketplace Gallery URL:

https://marketplace.visualstudio.com/_apis/public/gallery

Figure 2: VSCode Marketplace Configuration

Restart Antigravity.

After this change, searching and installing extensions works just like the official VS Code Marketplace. Installing AMP, GitHub Theme, VS Code Icon, etc., all follow this process.

Installing the AMP Extension

AMP isn’t officially supported on Antigravity yet, but you can install it directly via the VS Code Marketplace.

Steps:

Open the Extensions panel (the same icon as in VS Code).
Search for the AMP extension and install it as usual.
Log in using your AMP API Key.

Currently, Antigravity doesn’t support one-click account login like VS Code; you have to use the API key.

Summary

Once installed, AMP works almost identically in Antigravity as in VS Code—completion and refactoring features are available. The only difference is manual login configuration.

I recommend AMP because it offers a free mode. In my experience, it’s great for writing documentation, running scripts, and as a daily command-line tool. It’s fast, and especially useful for optimizing prompts.

Importing the CodeX Extension

CodeX doesn’t provide a direct VSIX download link on the web. My approach is to export it from VS Code and then import it into Antigravity.

Figure 3: Exporting Codex Extension in VS Code

Steps:

Install the CodeX extension in VS Code (if you haven’t already).
In VS Code’s extension manager, find CodeX and export it as a .vsix file.
Switch to Antigravity, open the Extensions panel, and select “Install from VSIX”.
Choose the exported codex-x.x.x.vsix file to complete installation.

Tip

Since my local VS Code is already logged into CodeX, importing it into Antigravity automatically reuses the login state—I didn’t need to log in again.

Optimizing Editor Settings

Beyond the marketplace and plugins, a few tweaks make the experience even closer to VS Code:

Theme: Choose the same color scheme as VS Code to minimize visual switching. I use GitHub Theme and vscode-icons.
Editor Settings: In “Open Editor Settings”, set indentation, formatting, line width, etc., to match your VS Code preferences. I define these in the workspace’s settings.json, so no migration is needed.

After these changes, the editing area is essentially “VS Code with an agent console”.

Remaining Issues

To fully migrate from VS Code/GitHub Copilot to Antigravity, I think there are still several key challenges:

Limited Customization: Antigravity can’t support custom prompts and agents like Copilot Chat. Currently, only “rules” configuration is available, which limits workflow flexibility.
Model Ecosystem Needs Improvement: Antigravity hasn’t natively integrated the latest models from major vendors (OpenAI, Anthropic, Microsoft, xAI, etc.), whereas GitHub Copilot excels here.
Cost Considerations:
- Future pricing may start at $20/month.
- No free models are supported, unlike GitHub Copilot (even Copilot Pro users have free model options).
Stability Issues: Agents often encounter “Agent terminated due to error” during operation, requiring manual retries or new sessions. This affects workflow smoothness, though I expect improvements in the future.

GitHub Copilot VS. Antigravity

Although Antigravity excels in several areas, there is still significant room for improvement compared to the combination of GitHub Copilot and VS Code.

The large language models (LLMs, Large Language Models) I frequently use are all supported in VS Code:

Figure 4: Copilot-supported LLMs (partial)

My long-accumulated custom prompts:

Figure 5: Copilot Chat enables quick access to custom prompts

My collection of agents:

Figure 6: Copilot Chat allows selection of custom agents

Here are some personal experiences using VS Code and Copilot that, for now, are hard to replace with other IDEs:

The Ask/Edit/Agent/Plan workflow perfectly fits my working habits.
Support for custom prompts and agents is essential. Many of my prompts and agents have been refined over time and are deeply integrated into my daily workflow—it’s hard to find alternatives elsewhere.
New models are integrated at lightning speed. Whenever a new model is released, GitHub Copilot is among the first to support it.
The integration with VS Code is seamless—no extra configuration required, making it extremely convenient.
Frequent updates: just a few days ago, a bug I reported to VS Code was fixed the same night.
Copilot Chat’s keyboard shortcuts make it easy to quickly access various features.
GitHub has granted me a free Pro account. Although the monthly premium quota is only 300 calls, combining Copilot with other plugins like AMP, Codex, Droid, and Qwen enables a highly efficient workflow. Even if I upgrade to a paid account in the future, the $10/month fee is very cost-effective compared to similar products.

Practical Experience

A few subjective tips from my actual usage—take them as reference:

Don’t treat Antigravity as “VS Code + chat box”. Use its agent features for complete tasks: let the agent propose a plan, then execute changes.
For major changes, always create a new Git branch and restrict agent actions to that branch. Handle all diffs via standard Pull Request (PR) workflows.
Ask agents to produce “artifacts” (plans, proposals, test descriptions), not just final code. This makes it easier to review and track changes.
Plugins you’re already comfortable with in VS Code (like AMP, CodeX) can be migrated directly, reducing cognitive load and letting you focus on new agent workflows.

Summary

My current experience: Antigravity delivers powerful agent capabilities and multi-view consoles. By following these steps to align the interface and plugin ecosystem with VS Code, you can smoothly transition your daily development workflow.

Cloudflare November 18 Global Outage: The Dangers of Implicit Assumptions in Modern Infrastructure

Jimmy Song — Wed, 19 Nov 2025 18:56:34 +0800

The greatest risks to modern internet infrastructure often aren’t in the code itself, but in those implicit assumptions and automated configuration pipelines that go undefined. Cloudflare’s outage is a wake-up call every Infra/AI engineer must heed.

Yesterday (November 18), Cloudflare experienced its largest global outage since 2019. As this site is hosted on Cloudflare, it was also affected—one of the rare times in eight years that the site was inaccessible due to an outage (the last time was a GitHub Pages failure, which happened the year Microsoft acquired GitHub).

Figure 1: jimmysong.io was down for 27 minutes due to the Cloudflare outage

This incident was not caused by an attack or a traditional software bug, but by a seemingly “safe” permissions update that triggered the weakest link in modern infrastructure: implicit assumptions (Implicit Assumption) and automated configuration pipelines (Automated Configuration Pipeline). Cloudflare has published a blog post Cloudflare outage on November 18, 2025 explaining the cause.

Here is the chain reaction process of the outage:

A permissions adjustment led to metadata changes;
The metadata change doubled the lines in the feature file;
The doubled lines triggered the proxy module’s memory limit;
The memory limit caused the core proxy to panic;
The proxy panic led to a cascade failure in downstream systems.

This kind of chain reaction is the most typical—and dangerous—systemic failure mode at today’s internet scale.

Root Cause: Implicit Assumptions Are Not Contracts

Let’s first look at the core hidden risk in this incident. The Bot Management feature file is automatically generated every five minutes, relying on a default premise:

The system.columns query result contains only the default database.

This assumption was not documented or validated in configuration—it existed only in the engineer’s mental model.

After a ClickHouse permissions update, the underlying r0 tables were exposed, instantly doubling the query results. The file size exceeded the FL2 preset of 200 features in memory, ultimately causing a panic.

Once an implicit assumption is broken, the system lacks a buffer and is highly prone to cascading failures.

Configuration Pipelines Are Riskier Than Code Pipelines

This incident was not caused by code changes, but by data-plane changes:

SQL query behavior changed;
Feature files were automatically generated;
The files were broadcast across the network.

A typical phenomenon in modern infrastructure: data, schema, and metadata are far more likely to destabilize systems than code.

Cloudflare’s feature file is a “supply chain input,” not a regular configuration. Anything entering the automated broadcast path is equivalent to a system-level command.

Language Safety Can’t Eliminate Boundary Layer Complexity

A former Cloudflare engineer summarized it well:

Rust can prevent a class of errors, but the complexity of boundary layers, data contracts, and configuration pipelines does not disappear.

The FL2 panic stemmed from a single unwrap(). This isn’t a language issue, but a lack of system contracts:

No upper-bound validation for feature count;
File schema lacked version constraints;
Feature generation logic depended on implicit behavior;
Core proxy error mode was panic, not graceful degradation.

Most incidents in modern distributed systems (Distributed System) come from “bad input,” not “bad memory.”

Core Proxies Need Controllable Failure Paths

FL/FL2 are Cloudflare’s core proxies; all requests must pass through them. Such components should not fail with a panic, but have the following capabilities:

Ignore abnormal features;
Truncate over-limit fields;
Roll back to previous versions;
Fail-open or fail-close;
Skip the Bot module and continue processing traffic.

As long as the proxy “stays alive,” the entire network won’t be completely paralyzed.

Data Changes Are More Uncontrollable Than Code Changes

The essence of this incident:

Subtle permission changes;
ClickHouse default behavior changed;
Query results propagated to distributed systems;
Automated publishing amplified the error;
Edge proxies crashed due to uncontrolled input.

Future AI Infra (AI Infrastructure) will be even more complex: models, tokenizers, adapters, RAG indexes, and KV snapshots all require frequent updates.

In future AI infrastructure, data-plane risks will far exceed those of the code-plane.

Recovery Process Shows Engineering Maturity

During the incident, Cloudflare took several measures:

Stopped generating erroneous feature files;
Force-distributed the previous version of the file;
Rolled back Bot module configuration;
Ran Workers KV and Access outside the core proxy;
Restored traffic in stages.

Restoring hundreds of PoPs worldwide simultaneously demonstrates a high level of engineering maturity.

Lessons for Infra/AI/Cloud Native Engineers

The Cloudflare event highlights four common risks in large-scale systems:

Implicit assumptions fail;
Configuration supply chain contamination;
Automated publishing amplifies errors;
Core proxies lack graceful degradation paths.

For AI Infra practitioners, these risks are even more relevant:

Model weight updates without schema validation;
Adapter merges may be contaminated;
RAG index incremental builds are unstable;
Inference graph configuration may be broken by bad data;
Automatically rolled-out models may propagate errors network-wide.

AI engineering is replaying Cloudflare’s infrastructure dilemmas—just at greater speed and scale.

Summary of Former Cloudflare Engineer’s Views

His insights pinpoint the hardest problems in distributed systems:

The issue isn’t code, but missing contracts;
Not the language, but undefined input boundaries;
Not modules, but lack of validation in the configuration supply chain;
Not bugs, but absence of fail-safe mechanisms.

This incident proves: The real fragility in modern infrastructure lies in “behavioral boundaries,” not “memory boundaries.”

Summary

The Cloudflare November 18 outage was not a coincidence, but an inevitable result of modern internet infrastructure evolving to large-scale, highly automated stages.

Key takeaways from this event:

System assumptions must be made explicit;
Configuration pipelines must be validated;
Automated publishing needs “dead-end” mechanisms;
Core proxies must be designed with controllable failure paths;
Data-plane contracts must be stricter than code-plane contracts.

In the AI-native Infra era, these requirements will only become more stringent.

References

The Second Half of Cloud Native: The Era of AI Native Platform Engineering Has Arrived

Jimmy Song — Mon, 17 Nov 2025 11:07:40 +0000

The second half of cloud native isn’t about being replaced by AI, but being rewritten by it. The future of platform engineering will revolve around models and agents, reshaping the tech stack and developer experience.

Since I first encountered Docker and Kubernetes in 2015, I’ve followed the cloud native journey: from writing Deployments in YAML, to exploring Service Mesh and observability, and in recent years, focusing on AI Infra and AI-Native Platforms. Looking back from 2025, the years 2015–2025 can be seen as the “first half” of cloud native. Marked by KubeCon / CloudNativeCon NA 2025, the industry is collectively entering the “second half”: the era of AI-Native Platform engineering.

This article reviews the past decade of cloud native, and, combined with KubeCon NA 2025, outlines key turning points and the technical coordinates for the next ten years.

2015–2025: The “First Half” of Cloud Native

Over the past decade, cloud native technology themes have evolved through three main stages. The following flowchart illustrates the progression.

The flowchart below illustrates the progression of cloud native technology themes over the past decade:

Figure 1: Cloud Native Decade Technology Evolution Flow

The first stage focused on containerization and orchestration standardization.

Docker realized the engineering dream of “build once, run anywhere”
Kubernetes won the orchestration wars and became the de facto standard
CNCF was founded, with Prometheus, Envoy, and other projects joining
Enterprises focused on migrating applications to Kubernetes

Typical tasks during this phase involved moving Java services from VMs to containers and K8s, emphasizing understanding of Deployment, Service, and Ingress.

The second stage, 2018–2020, saw complexity shift from “deployment” to “communication” and “operations”.

Service Mesh (Istio / Linkerd / Consul) addressed east-west traffic management
The observability trio (Logs / Metrics / Traces) became default configurations
Multi-cluster and multi-region practices matured
Enterprises focused on managing large microservice systems

During this period, I spent significant time researching Istio, service mesh, and traffic management, and authored Kubernetes and Istio books. The focus shifted to system stability, observability, and reliability.

The third stage, 2021–2025, is defined by Platform Engineering and GitOps.

As microservices and tools proliferated, platform complexity began to overwhelm developers, making Platform Engineering a key industry term.

GitOps (Argo CD / Flux) drove declarative delivery processes
Internal Developer Platforms (IDP) became priorities for large enterprises
“Platform as a product” philosophy spread
FinOps, cost management, and compliance auditing became platform concerns
DevOps evolved from “tool practice” to “organizational + platform capability”

My takeaway: simply giving developers a pile of tools isn’t enough. End-to-end delivery paths and stable abstraction layers are needed so developers can focus on business, not tool integration.

The table below summarizes the main features of each “first half” stage.

Stage	Core Challenge	Key Tech Stack	Typical Issues
2015–2017 Orchestration	Migrating from VM to containers	Docker, Kubernetes, CNI	Reliable deployment, rolling upgrades
2018–2020 Mesh	Microservice scale, complex communication & observability	Istio/Linkerd, Prometheus, Jaeger	Troubleshooting, fragmented observability
2021–2025 Platform	Tool sprawl, declining developer experience	GitOps, IDP, FinOps, Policy-as-Code	Developer fatigue, platform team overload

Table 1: Cloud Native First Half Stage Features

KubeCon NA 2025: Signals of Cloud Native’s “Second Half”

The main theme of KubeCon 2025 is no longer “how to use Kubernetes well,” but how to reconstruct Kubernetes and the cloud native ecosystem into AI-Native Platforms for the AI era.

Key signals from KubeCon NA 2025 include:

CNCF released the Certified Kubernetes AI Conformance Program
Dynamic Resource Allocation (DRA) entered mainstream discussions
Model Runtime / Agent Runtime projects became conference hotspots
Vendors focused on AI SRE, AI-assisted development, AI security, and supply chain governance
Speakers like Alex Zenla openly stated that Kubernetes’ underlying structure needs rethinking

Together, these mark a clear dividing line: cloud native has officially entered its “second half.”

First Half vs Second Half: Shifting the Cloud Native Narrative

If 2015–2025 is the “first half,” then 2025–2035 is likely the “second half.” The table below compares their core differences.

It highlights changes in platform objects, goals, abstraction layers, and more.

Dimension	First Half (2015–2025)	Second Half (2025–2035, AI Native)
Core Objects	Containers, Pods, Microservices	Models, inference tasks, Agents, data pipelines
Platform Goals	Stable application delivery	Efficient, continuous AI workload & agent orchestration
Abstraction Layers	Deployment / Service / Ingress / Job	Model / Endpoint / Graph / Policy / Agent
Resource Scheduling	CPU / Memory / Node	GPU / TPU / ASIC / KV Cache / Bandwidth / Power
Engineering Focus	DevOps / GitOps / Platform Engineering 1.0	AI Native Platform Engineering / AI SRE
Security & Compliance	Image security, CVE, supply chain SBOM	Model security, data security, AI supply chain & “hallucination dependencies”
Runtime Forms	Container + VM + Serverless	Container + WASM + Nix + Agent Runtime

Table 2: Core Differences: First vs Second Half of Cloud Native

From a developer’s perspective, the most direct change is: future platforms will no longer treat “services” as first-class citizens, but will center on “models + agents.”

Example: Technical Layers of an AI Native Platform

To clarify the structure of an AI-Native Platform, the following layered diagram shows the relationships between technical levels.

The layered diagram below shows the relationships between different technical levels in an AI-Native Platform:

Figure 2: AI Native Platform Layering Diagram

Historically, cloud native focused on L0 + L2 (Kubernetes + platform engineering), but in the AI Native era, L1 (Model Runtime, Agent Runtime, heterogeneous resource scheduling) becomes the new battleground.

Key Change 1: From “Container-Centric” to “Model-Centric”

In the first half, cloud native’s main object was the application process, with containers as packaging. The second half requires handling:

Model version management and canary releases
Balancing inference performance, latency, and cost
Multi-model composition, routing, A/B testing
Relationships between models, data, features, and vector indexes

At KubeCon NA 2025, CNCF’s AI Conformance Program aims to standardize model workloads, managing them like Deployments. Platform engineering will gain new abstractions—not just “deploying services,” but “deploying model capabilities.”

Key Change 2: DRA and the Golden Window for Heterogeneous Resource Scheduling

Previously, writing a Deployment meant focusing on CPU and memory. Now, GPU inference, training, and Agent Runtime scenarios demand more than static quotas.

Dynamic Resource Allocation (DRA) brings:

Pluggable resource types (GPU/TPU/FPGA/ASIC)
Topology-aware, NUMA, and memory fragmentation scheduling
Binding inference requests to compute allocation for fine-grained QoS
Cost optimization and power control in scheduling decisions

This is the most significant “resource perspective” upgrade since Kubernetes’ inception. The scheduler is no longer just a cluster component, but the AI platform’s policy engine.

Key Change 3: Agent Runtime as the New Generation of Runtime

KubeCon showcased several representative projects:

Edera: Minimal, verifiable runtime redesign
Flox: Nix-based “uncontained” runtime environment
Golem: WASM-based large-scale agent orchestration

The consensus: AI agents aren’t suited to traditional container runtime models. Agents have these traits:

Strong statefulness: context, memory, sessions
High concurrency but fine granularity: massive lightweight tasks
Extremely sensitive to latency and cold starts
Need to resume after failure

Next-gen runtimes focus on reliably executing, managing state, and auditing “hundreds of thousands of agents,” not just “spinning up more Pods.”

Key Change 4: AI SRE and AI Security

At KubeCon NA 2025, security and operations topics were amplified by AI:

Software supply chain attacks and CVEs continue to rise
LLM-assisted coding introduces “hallucination dependencies” and “vibecoded vulnerabilities”
AI-driven artifact scanning, dependency auditing, and license analysis
“AI SRE” is now a formal product category

Traditional cloud native already emphasized security and SRE, but now must address model weights, datasets, vector stores, and agent workflows. AI-Native Platform engineering must answer:

Are code and dependencies secure?
Are models and data trustworthy?
Are agent behaviors controllable?

This will drive deep integration of Policy-as-Code, MCP, graph permission systems, and AI.

Key Change 5: Open Source Participation Becomes a Baseline

In interviews, platform engineering leaders noted:

Hiring increasingly values upstream contributions to Kubernetes and related projects
Open source involvement shortens ramp-up time
New AI Native projects (Model Runtime, Agent Runtime, Scheduler) are also open source

For career growth, contributing to AI Native open source projects will become a basic requirement for platform engineering and AI Infra roles—not just a resume bonus.

The Contours of Cloud Native’s “Second Half”

The table below summarizes the technical focus and essential differences of the “second half.”

It highlights the key coordinates of AI-Native Platform engineering.

Direction	Technical Focus	Essential Difference from First Half
AI Native Platform	Models/Agents as first-class citizens, unified abstraction & governance	Objects shift from services to models & inference
Resource Scheduling	DRA, heterogeneous compute, topology awareness, power & cost	From static quotas to dynamic, policy-driven
Runtime	Container + WASM + Nix + Agent Runtime	From “process containerization” to “execution graph containerization”
Platform Engineering	IDP + AI SRE + Security + Cost + Compliance	From toolset to “autonomous platform”
Security & Supply Chain	LLM dependencies, model weights, datasets, vector store governance	Protection expands from images to “all AI engineering assets”
Open Source & Ecosystem	AI Infra / Model Runtime / Agent Runtime upstream collaboration	Not just “using open source,” but “building the future in open source”

Table 3: Cloud Native Second Half Technical Coordinates

Summary

Over the past decade, cloud native evolved from container orchestration to platform engineering 1.0. With KubeCon NA 2025 as a milestone, the industry systematically brings AI into cloud native technology and organizational stacks:

Kubernetes is no longer just “infrastructure for microservices,” but “runtime for AI workloads”
Platform Engineering is no longer just “tool integration,” but “autonomous platforms for models and agents”
Security, SRE, runtime, scheduling, and networking will all be reimagined under AI

For me, the past ten years were about “making applications more stable in the cloud native world.” The next ten will focus on “making AI better, safer, and more controllable in the cloud native world.” This is, in my view, the opening whistle for cloud native’s “second half.”

NotebookLM: My Most Recommended AI Tool for Learning and Knowledge Organization

Jimmy Song — Mon, 17 Nov 2025 08:44:45 +0000

NotebookLM is the most tailored AI tool I’ve used for knowledge workers. It truly helps me structure massive information and dramatically boosts my learning and content creation efficiency.

As a lifelong learner who reads technical specs and researches open-source projects, I’ve always sought a tool that can “shortcut” my way through mountains of material, reduce mechanical reading, and help me quickly build a global understanding. NotebookLM has been the smoothest and most reliable experience for me over the past year.

It’s not a traditional “chat-style AI tool”—it’s more like an AI-native learning and content organization system that ingests your materials, organizes them, and presents them in various structured formats. The more I use it, the more I realize its help in learning new technologies, understanding unfamiliar fields, organizing large project documents, and building teaching materials—things that general large language models (LLM, Large Language Model) simply can’t match.

The Core Value NotebookLM Brings Me

NotebookLM has significantly improved my workflow, especially in learning new technologies, organizing documents, and content creation.

Quickly Understanding New Technologies: Feed in Complex Materials, Get a “Learnable Version”

My most frequent and indispensable scenario is learning a completely unfamiliar technology or development framework. Faced with dozens or even hundreds of pages of documentation, my typical approach is:

Add official docs, README files, design documents, and architecture diagrams into a single Notebook
Let NotebookLM generate:
- Study guides
- Briefings
- Key knowledge points
- FAQs
- Quizzes
Ultimately, I get a clearly structured “learning entry point” instead of a flood of raw materials.

The following flowchart illustrates how NotebookLM compresses complex documents into a learnable structure:

Figure 1: NotebookLM Document Structuring Flow

In the end, what I gain is an “organized knowledge system” rather than a pile of PDFs waiting to be consumed.

Generating MindMaps: Instantly Turning Large Documents into Structured Knowledge Graphs

I rely heavily on MindMaps to build the “skeleton of knowledge.” NotebookLM’s MindMap feature stands out for:

Automatically identifying relationships between topics
Interactive node expansion and collapse
Integrating multiple source documents

Although it currently only exports PNG, the logical structure itself is already an excellent “knowledge compression.”

The table below compares the auto-generation and visualization capabilities of different tools:

Tool	Auto-Generation	Multi-Doc Integration	Visualization Quality	Export Formats
NotebookLM	Strong	Strong	Good	PNG only (SVG not yet supported)
Common LLM Tools	Weak	Weak	Poor	Depends on tool
MindMap Software (Manual)	None	None	Strong	Fully supported

Table 1: Comparison of MindMap Capabilities in Mainstream Tools

NotebookLM’s greatest advantage is automation.

Generating Teaching Outlines, Training Scripts, and Book Structures: Truly Saving Me Time

NotebookLM is more than just “summarization”—it can generate formal teaching structures based on my prompts. By feeding in project docs, API references, architecture designs, case studies, videos, and blogs, and prompting it to generate:

Teaching outlines
Project training manuals
Course structures
Book chapter frameworks
Slide text
Training case descriptions

For anyone who needs to create content, conduct training, or give presentations, this feature is a huge time-saver.

Below is a typical prompt I actually use:

Based on the provided content excerpts, write a detailed training manual that systematically explains the core principles covered. The manual should use a professional and instructional tone, breaking down complex concepts into actionable steps and lessons. Ensure all content is strictly based on the source material and covers every aspect mentioned.

The training manual should include:
1. Training objectives and expected outcomes
2. Training content and structure
3. Training methods and tools
4. Training evaluation and feedback
5. Training summary and follow-up actions
6. Training cases and examples
7. Training resources and references

The results are often surprisingly good.

Multi-Format Input Capability: The Most Stable I’ve Seen

NotebookLM supports direct ingestion of various material types, with extremely stable parsing. The table below summarizes my actual experience:

Input Type	My Actual Experience
PDF	Most stable, clear structure parsing
Google Docs	Syncs instantly, very smooth
Word / PPT	Recognized normally
YouTube Video	Auto-summary + key content extraction, very useful
Website URL	Depends on site structure, high success rate
Plain Text	No issues
Images	Partial success, sufficient for screenshots

Table 2: NotebookLM Multi-Format Input Experience

By contrast, other tools often have format parsing issues, garbled text, missing content, or skipped paragraphs. NotebookLM is especially stable in “multi-format ingestion.”

My Most Common NotebookLM Workflow

The following flowchart shows my daily workflow with NotebookLM:

Figure 2: NotebookLM Daily Workflow

Essentially: let AI help me grasp the big picture → then dive deeper → then output content.

My Suggestions and Minor Regrets

NotebookLM is already excellent, but I still have some strong expectations for future improvements:

MindMap Export Formats Should Support SVG or Text-Based (Markmap)

Currently, only PNG is supported, which gets blurry when enlarged. The table below lists my expectations for future features:

Expected Feature	Purpose
SVG Export	For writing books, making slides, scalable without loss
Markmap Output	Most friendly for Markdown writers
Raw JSON	Allows custom rendering

Table 3: Expected MindMap Export Formats

I’m especially looking forward to NotebookLM supporting Markmap format export, which would be extremely friendly for users who write blogs and docs in Markdown.

Recently, Google also launched CodeWiki, similar to DeepWiki, which auto-generates image-rich Wikis for GitHub projects, but currently does not support Mermaid or Markmap.

Conversation History Should Support Long-Term Saving

Currently:

Chats are not persistently saved
Only manually “add to notes” preserves results

This causes some knowledge context to be lost. I hope to see a “Notebook conversation history” feature in the future.

Slide Generation Should Support Templates for Content Creators

Currently, Video Overview offers various visual styles, but cannot:

Upload custom PPT templates
Apply enterprise/personal branding templates

If PPT template support is added, NotebookLM could become the “video generation hub” for content creators.

Deep Research Should Launch Soon and Be Fully Open

I’m especially looking forward to this feature, as it could upgrade NotebookLM from a “knowledge organization tool” to a “research-grade tool.” I hope it will:

Reliably crawl more public web pages
Ensure citation quality
Integrate with existing Notebook materials

This is a major upgrade I personally care about.

Mobile Experience Should Be Enhanced Beyond Content Playback

Currently, the mobile experience is minimal, only allowing:

Listening to audio
Viewing Notebook Guide summaries
Simple Q&A

I hope mobile will soon support:

Editing Notebooks
Deep conversations
MindMap interaction
Content output (generating docs, outlines, etc.)

Summary

NotebookLM is truly one of the AI tools I use every single day because it achieves a critical goal:

Organizing information, structuring knowledge, so I don’t have to start from scratch with massive documents.

Whether it’s:

Learning new technologies
Reading long documents
Creating courses
Conducting training
Writing books
Drafting speeches
Summarizing content

It saves me a huge amount of time upfront, letting me focus on “understanding” and “creating.”

I’ll continue to use NotebookLM as one of my essential tools and keep an eye on its progress in Deep Research, template systems, and mobile features.

This is a tool truly designed for “knowledge workers” and deserves to be known by more people.

Helm v4: Paradigm Convergence and Plugin System Rebuild

Jimmy Song — Fri, 14 Nov 2025 11:18:30 +0000

The release of Helm 4 is not just a technical upgrade, but a deep convergence of cloud-native delivery paradigms. The rebuilt plugin system and supply chain governance capabilities make Helm once again a driving force in the Kubernetes ecosystem.

Since its first release in 2016, Helm has been one of the most important application distribution tools in the Kubernetes ecosystem. Helm v4 is not a “minor enhancement,” but a comprehensive update around delivery methods, extension mechanisms, and supply chain approaches.

This article reconstructs Helm’s historical context and focuses on why Helm 4 represents a paradigm-converging release.

Helm: From Tiller to Declarative Delivery

Below is a textual timeline showing key milestones from Helm v2 to v4, helping you understand its technical evolution:

2016: Helm v2 released, using the Tiller architecture.
2017: Chart Hub expands, major projects begin providing official Charts.
2018: Security model controversies intensify, Tiller’s permission issues become apparent.
2019: Helm v3 released, Tiller removed, OCI support introduced.
2021: GitOps becomes widespread, Server-Side Apply (SSA) becomes the mainstream delivery semantic.
2023: kstatus widely adopted for controller status assessment and health calculation.
2025: Helm v4 released, bringing SSA, WASM plugins, reproducible builds, and content hash caching.

Each major Helm release closely follows Kubernetes paradigms, driving progress in declarative delivery and ecosystem tooling.

Fundamental Changes in Helm v4

This section analyzes the core technical upgrades and paradigm shifts in Helm v4.

Delivery Paradigm Update: Default Server-Side Apply (SSA, Server-Side Apply)

In Helm v3 and earlier, Helm used a “three-way merge” model for resource delivery. Helm v4 fully switches to Server-Side Apply (SSA, Server-Side Apply), meaning the API Server determines field ownership.

This shift brings several direct results:

Full semantic alignment with kubectl apply and GitOps controllers (such as Argo, Flux)
When multiple controllers manage the same object, silent overrides are avoided and conflicts are explainable
Helm’s behavior now follows Kubernetes’ officially recommended declarative delivery paradigm

The following flowchart compares the delivery semantics of Helm v3 and v4.

Figure 1: Helm v3/v4 Delivery Semantics Comparison

Helm is now aligned with the delivery semantics of modern Kubernetes versions, improving predictability and safety in resource management.

kstatus-Driven Wait Behavior and Readiness Annotations

In Helm 3, --wait could only make fuzzy status judgments on limited resources, lacking extensibility and explainability.

Helm 4 introduces kstatus (Kubernetes Status) as the basis for health status parsing, and supports two key annotations:

helm.sh/readiness-success
helm.sh/readiness-failure

Chart authors can precisely define conditions for installation success or failure. Helm’s waiting model now offers “explainability + extensibility,” upgrading from a “templating tool” to a true “deployment orchestrator.”

Extension System Rebuild: WASM Plugin System

Helm 4 thoroughly reconstructs the plugin model, mainly including:

Typed and Structured Plugins

Arbitrary scripts are no longer allowed; plugins must follow typed and structured standards

WebAssembly Plugin Runtime (Extism)

More secure (sandbox isolation)
Cross-language support
Easy unified management in CI/CD and enterprise platforms
Predictable and testable

Post-renderer Integrated into Plugin System

Moves beyond the “external executable black box” era
Helm becomes a programmable platform, not just a template renderer

Engineering Capabilities Upgrade: Reproducible Builds, Content Hash Caching, chart API v3

Helm v4 brings the following engineering improvements:

Chart packaging is reproducible (supports signing, SBOM, SLSA, etc. for supply chain governance)
Local cache uses content hashes, avoiding version-based conflicts
chart API v3 (experimental) is stricter and more flexible
SDK logging system upgraded to Go slog (modern logging)

These capabilities enable Helm charts to enter serious software supply chain governance.

Feature Comparison (Helm v3 → v4)

The table below compares core features between Helm v3 and v4 for a quick understanding of the upgrade value.

Area	Helm 3	Helm 4
Apply Model	Three-way merge	Default SSA
Wait Behavior	Fuzzy, not extensible	kstatus + annotation
Plugin System	Script, uncontrollable	WASM, typed plugins
Post-renderer	External executable	Plugin subsystem
Build	Not reproducible	Reproducible build
Cache	name/version	Content hash
Chart API	v2	v2 + v3 (experimental)
SDK Logs	stdlib log	slog

Table 1: Helm v3 vs v4 Feature Comparison

This is a release that “repays technical debt in bulk + aligns with contemporary Kubernetes semantics.”

Why Is Helm v4 a Paradigm Convergence Event?

The release of Helm v4 is not just a feature upgrade, but a deep convergence of delivery paradigms, mainly in three aspects:

Kubernetes Delivery Semantics Unified to SSA

Previously: kubectl, GitOps, and Helm each had their own logic. Now: All unified to SSA, consistent delivery behavior, smoother ecosystem collaboration.

Plugin System Enters the Platform Era

WASM (WebAssembly) brings a secure, universal, and controllable plugin runtime. Infrastructure projects widely adopt WASM: Envoy → WASM Filters, Kubernetes → WASM CRI/OCI, and now Helm joins the platform camp.

Charts Enter Supply Chain Governance

Reproducible builds and digest verification allow Helm charts to be managed as seriously as container images, greatly enhancing supply chain security.

The entire ecosystem moves to a unified capability baseline, driving cloud-native delivery standardization.

My Helm History and Observations

As an early user from the Helm v2 era, I have experienced the following stages:

Tiller security controversies
v3 migration (state stored in secrets)
Large-scale chart consolidation in the community
OCI adoption
Today’s SSA / WASM / reproducible build

Each major Helm version upgrade is not about chasing trends, but proactively aligning with Kubernetes paradigms:

v3 aligns with K8s “no cluster-side runtime” principle
v4 aligns with SSA, kstatus, WASM, OCI, and other advances from the past five years

Helm exemplifies the evolution rhythm of infrastructure projects: not by piling on features, but by evolving in semantic alignment with the platform.

Summary

The release of Helm v4 marks a new paradigm for Kubernetes application delivery. SSA, WASM plugins, kstatus, and reproducible builds make Helm not just a templating tool, but a core for supply chain governance and platform extensibility. For cloud-native developers and platform teams, Helm v4 is a paradigm upgrade worth attention.

References

Kimi K2 Thinking: The True Awakening of China's Thinking Model

Jimmy Song — Fri, 14 Nov 2025 08:25:26 +0000

China’s large language models have finally moved from “writing like humans” to “thinking like humans.” The open-sourcing of Kimi K2 is a watershed moment for China’s AI trajectory.

The narrative around China’s large language models is shifting from “Chat-style models” to “Thinking models (Thinking Model, Thinking Model).”

Moonshot AI’s open-sourcing of Kimi K2 Thinking marks the first real landing of this transition. K2 is not just another iteration like ChatGLM or Qwen; it’s the first time a Chinese team has unified “deep reasoning + long context + tool invocation continuity” in training. This is the core of the thinking model approach and the reason why models like Claude and Gemini have led the field.

The Significance of K2’s Open Source: China Enters the Era of Thinking Models

Why is K2’s open source a turning point? Because it enables Chinese models to achieve the following capabilities for the first time:

Stable execution of 200–300 tool invocations (toolchain reasoning stability)
Deep, multi-stage reasoning chain execution (CoT Consistency, Chain-of-Thought Consistency)
256k context as a “working memory” (Working Memory, Working Memory)
Native INT4 acceleration + MoE activation sparsity scheduling

This is a completely different path from “stacking parameters → stacking benchmarks,” emphasizing reasoning ability over parameter scale.

In short:

K2 is the first time a Chinese model has entered the sequence of thinking models (Thinking Model, Thinking Model).

Dissecting K2’s Technical Approach

K2’s technical approach can be broken down into five key points, each directly impacting the model’s reasoning ability and ecosystem adaptability.

MoE Expert Division: Cognitive Division Rather Than Parameter Expansion

K2’s MoE (Mixture of Experts, Mixture of Experts) design philosophy is distinct from previous models. The core is not about activating fewer parameters or running larger models more cheaply, but about assigning different cognitive sub-skills to different experts. For example:

Mathematical reasoning expert
Planning expert
Tool invocation expert
Browser task expert
Code generation expert
Long-chain retention expert

This division aligns directly with Claude 3.5’s cognitive layering (Cognitive Layering, Cognitive Layering) approach. K2’s MoE is about “dividing thinking among the model,” not just “making computation cheaper.”

256K Context: Building the Model’s Working Memory

K2’s ultra-long context is not just a parameter showcase; it’s designed to build the model’s “thinking buffer.” It allows the entire process to retain reasoning chains, tool invocation states, multi-stage reflection, and uninterrupted long tasks (such as research or code refactoring), stably executing multi-stage agent workflows. Long-term thinking requires long-term memory support, and K2’s long context is the “memory” for sustained reasoning chains.

Intertwined Training of Tool Invocation and Reasoning Chains

K2 excels in the intertwined training of tool invocation and reasoning chains. Traditional open-source models typically follow this process:

Generate reasoning
Output JSON function call
Tool returns result
Continue reasoning

In this approach, the reasoning chain and invocation chain are separated. K2’s training allows the reasoning chain to invoke tools at any time and feed tool results back into the reasoning chain for the next stage of thinking. It supports 200–300 consecutive tool invocations without interruption, fully aligning with Claude 3.5’s Interleaved CoT + Tool Use.

Native INT4 Quantization: Ensuring Reasoning Chain Stability

K2’s INT4 (INT4, 4-bit Integer Quantization) approach is not ordinary post-quantization. Its purpose is not only to reduce memory usage and increase throughput, but more importantly, to ensure that deep reasoning chains do not break due to insufficient computing power. The biggest killer of deep thinking chains is timeout, freezing, or unstable workers. INT4 enables Chinese GPUs (non-H100) to run complete reasoning chains, which is highly significant for China’s ecosystem.

MoE + Long Context + Toolchain: Unified Training Rather Than Module Stitching

K2’s most important feature is its holistic training approach: expert division, long context-driven consistency, tool invocation trained through real execution, browser tasks and long-step task reinforcement, and INT4 entering the training loop. It’s not a “ChatLLM + Memory + RAG + Tools” patchwork, but an integrated reasoning system.

Alignment and Differences Between K2 and International Mainstream Approaches

K2 is highly aligned with international mainstream models (such as Claude, Gemini, OpenAI) in cognitive reasoning, ultra-long context, and tool invocation mechanisms, but also has unique advantages for Chinese models:

Native INT4 + adaptation to Chinese computing power is rare globally
Toolchain continuity is more stable than most open-source models
Higher degree of open source, stronger ecosystem reusability

Collaborative Value of China’s AI Infra: K2 × RLinf × Mem-alpha

A series of important open-source infrastructures have emerged in the K2 ecosystem. The table below summarizes these project types and their value to K2:

Here is a comparison table of the collaborative value of each infrastructure with K2:

Project	Type	Value to K2
RLinf	Reinforcement Learning	Used to train stronger planning/browser task capabilities
Mem-alpha	Memory Enhancement	Can be combined with K2 to form long-term memory agents
AgentDebug	Agent Error Debugging	Used to analyze K2’s toolchain errors
UI-Genie	GUI Agent Training	Can serve as an experimental field for K2’s agent capability expansion

Table 1: Collaborative Value of China’s AI Infra Ecosystem

This combination is already forming a China AI Agent Infra Stack.

Personal View: The Significance of K2’s Approach

I believe the significance of K2 lies not in the model itself, but in its technical approach:

K2 marks the first time Chinese models have shifted from “language generation competition” to “thinking ability competition.”

For the past three years, the main line of China’s open-source models has been evaluation scores, parameter scale, instruction following, and alignment data. But K2 is the first to clearly take the path of deep reasoning, tool intertwining, cognitive division, long-term task chains, and native performance optimization. This means China’s model trajectory is now synchronized with the US, rather than chasing old paths.

Key Directions to Watch in K2’s Ecosystem Over the Next Year

K2’s future ecosystem influence will depend on several key points:

Whether it opens the tool registry (Tool Registry, Tool Registry)
Whether it supports dynamic memory (Mem-alpha integration)
Whether it opens the MoE expert structure
Whether it can form a Chinese reasoning chain optimization path with vLLM / llm-d / KServe
Whether it supports fault tolerance for multi-node continuous reasoning chains

These capabilities will determine K2’s ecosystem influence and technical extensibility.

K2 Thinking Model Architecture Diagram

The following flowchart illustrates the core architecture of the K2 thinking model and its collaboration with external agents/applications:

Figure 1: K2 Thinking Model Architecture

Summary

K2 is the first time China’s model trajectory is heading in the right direction:

From “writing like humans” to “thinking like humans.”

The era of thinking models is coming, and Chinese models are finally standing on the same roadmap as the international forefront.

References

Beyond Gateway: Inference Traffic Control Practice with Gateway API Inference Extension

Jimmy Song — Fri, 14 Nov 2025 08:19:16 +0000

AI inference traffic governance is undergoing a paradigm shift. The Gateway API Inference Extension makes “model awareness” the new 主线 of traffic control.

Brief Review of Gateway API’s Current State

Kubernetes Gateway API has entered the stable v1 series and continued iterating after the 1.0 GA release, enhancing advanced traffic governance capabilities such as WebSocket, timeout and retry, Service Mesh integration, GRPCRoute, request mirroring, CORS, Retry Budget, and more. Major cloud providers and gateway implementations (such as Alibaba Cloud ACK, GKE Gateway, Envoy Gateway, NGINX Gateway Fabric) have all adopted Gateway API as the new generation north-south traffic model.

Building on this foundation, the community has proposed an extension specification specifically for AI inference traffic - the Gateway API Inference Extension. This specification is not about “reinventing an API” but rather supplementing the Gateway API core model with model-aware load balancing and traffic control capabilities for Large Language Model (LLM) and other inference scenarios.

Early documentation often mentioned the InferenceModel + InferencePool CRD combination; the latest specification has evolved to InferenceObjective + InferencePool (with optional InferencePoolImport). This article consistently uses the latest terminology.

Typical Challenges in GenAI Inference Traffic

Traditional gateway, Ingress, and Service Mesh load balancing models are essentially “request-agnostic + endpoint-agnostic”: they distribute traffic across a group of static backends through algorithms like round-robin, least requests, and hashing.

In GPU-powered Large Language Model (LLM) inference scenarios, this model exposes obvious problems:

Invisible GPU utilization and queuing

A single LLM instance simultaneously maintains KV Cache, LoRA Adapter, and Token queues. Resource consumption varies significantly across the same batch of requests. Load balancing based solely on QPS or connection count can easily lead to extreme situations where “idle GPUs have no work while busy GPUs crash.”
Lack of semantic binding between models and requests

From the business perspective, one typically only sees a POST /v1/chat/completions endpoint, making it difficult to express intentions like “high-priority model / test version / canary weight” at the routing layer.
Difficult unified management of multiple models, versions, and LoRAs

Each model service implements its own routing and A/B testing solutions, making it difficult for the platform to achieve unified governance and observability at the control plane level.

The goal of Gateway API Inference Extension is to introduce AI-specific semantics and metrics into load balancing and traffic control decisions while maintaining the existing Gateway model.

Core Concepts and Resource Model of Gateway API Inference Extension

This section introduces the overall architecture and key resource objects of the Inference Extension.

Overall Architecture

The Inference Extension uses Envoy External Processing (ext-proc) mechanism to upgrade Gateway API + ext-proc capable gateways (such as Envoy Gateway, kgateway, GKE Gateway) into Inference Gateways. Requests still follow the standard Gateway + HTTPRoute path, but before being forwarded to the backend, they pass through an “Endpoint Picker” component that selects the most suitable backend instance based on real-time metrics exposed by the model server.

The flowchart below shows the overall architecture:

Figure 1: Inference Extension Architecture Flow

InferencePool: Platform-Side “Model Service Pool”

InferencePool is the core resource introduced by the Inference Extension, used to describe a group of inference Pods and routing plugin configurations. It is similar to a Service with a selector, responsible for selecting a group of model service Pods and specifying exposed ports, while also allowing attachment of Endpoint Picker plugins (such as Prefix-Cache-Aware, LoRA-aware, etc.).

In the Gateway API model, InferencePool is treated as a type of “Backend” that can be referenced by HTTPRoute.backendRefs.

The code block below shows a simplified example of InferencePool:

apiVersion: inference.networking.x-k8s.io/v1
kind: InferencePool
metadata:
 name: vllm-llama3-chat-pool
spec:
 targetPortNumber: 8000
 selector:
 app: vllm-llama3-chat
 extensionRef:
 name: prefix-cache-aware-endpoint-picker

The meaning of the above configuration is as follows:

Select all Pods with app=vllm-llama3-chat, port 8000.
Use the plugin named prefix-cache-aware-endpoint-picker to make routing decisions based on metrics such as KV Cache hit rate, queue length, and GPU utilization.

InferenceObjective: Business-Side “Request Objective”

InferenceObjective is used to express the goal and priority of a single request, decoupled from the model service pool. One request corresponds to one InferenceObjective, and the same InferencePool can serve multiple different InferenceObjectives.

Typical fields include:

Business criticality (Critical / High / BestEffort)
Required model family / version preference
Acceptable latency / cost upper limits, etc.

The Endpoint Picker can combine InferenceObjective with backend metrics to make decisions: prioritize Critical requests when resources are constrained, and shed BestEffort requests when necessary.

InferencePoolImport: Cross-Cluster / Gateway Reuse

InferencePoolImport supports importing InferencePools defined in remote clusters into the local cluster, facilitating consistent governance of multi-cluster, multi-region inference services.

Maturity and Implementation Ecosystem of Inference Extension

The current project version is v1.1.x, overall in the Alpha stage. Official recommendation is not to use it directly in production yet; it’s more suitable for platform teams to experiment with the technology stack.

Multiple implementations and integrations already exist:

Inference Gateway implementations for Envoy Gateway / kgateway
GKE Inference Gateway: Enhanced capabilities based on GKE Gateway, including KV Cache-aware routing, LoRA reuse, priority scheduling, etc.
NGINX Gateway Fabric, cloud provider ACK, and others are also following up on related extensions

Therefore, when designing practical solutions, it should be treated as a “forward-looking solution / future main path.” For production environments, prioritize managed implementations (such as GKE Inference Gateway) or commercial products from gateway vendors.

Practice: Using Inference Extension for Inference Traffic Control

The following example uses a self-hosted LLM cluster on Kubernetes to demonstrate how to serve external traffic through an OpenAI-compatible interface and achieve:

Scheduling by request priority (real-time conversation vs. batch processing)
Multi-version model canary and rollback
Optimized routing using GPU metrics and KV Cache hit rates
Unified platform-side observability and rate limiting entry point

Deploying Gateway API and Inference Extension

The deployment process is as follows:

Install a Gateway API-compatible gateway implementation (such as Envoy Gateway, kgateway, GKE Gateway) in the cluster.
Install the Gateway API Inference Extension CRD and control plane components.
Enable the metrics endpoints and plugin protocols required by Inference Extension on the model server side (such as vLLM, Triton, TGI).

Defining InferencePool: Abstracting LLM Pods as an “Inference Pool”

The code block below shows a typical InferencePool configuration:

apiVersion: inference.networking.x-k8s.io/v1
kind: InferencePool
metadata:
 name: chat-llama3-pool
spec:
 targetPortNumber: 8000
 selector:
 app: chat-llama3
 extensionRef:
 name: prefix-cache-aware
 # Optional: Plugin configuration ConfigMap / CR

Key points:

selector selects LLM Pods, targetPortNumber specifies the inference service port.
extensionRef binds the Endpoint Picker plugin, implementing KV Cache prefix-aware routing, selecting replicas with lighter load based on metrics like queue_length/gpu_utilization, and triggering load shedding when necessary.

Defining InferenceObjective: Connecting “Business Intent” to Routing Decisions

The code block below shows a sample InferenceObjective configuration (fields can be adjusted according to the actual version and implementation):

apiVersion: inference.networking.x-k8s.io/v1
kind: InferenceObjective
metadata:
 name: chat-critical
spec:
 criticality: Critical  # Real-time conversation
 preferredModel: llama3-70b
 fallbackModel: llama3-8b
---
apiVersion: inference.networking.x-k8s.io/v1
kind: InferenceObjective
metadata:
 name: chat-batch
spec:
 criticality: BestEffort  # Batch analysis, log summarization
 preferredModel: llama3-8b

The Endpoint Picker can combine InferenceObjective with InferencePool metrics to make the following decisions:

When GPU is constrained and queues are too long, prioritize chat-critical requests and discard chat-batch if necessary.
For Critical requests, prioritize the large model; if the target pool is unavailable, fall back to the small model pool.

Importing Business Traffic to InferencePool via HTTPRoute

The code block below shows the HTTPRoute configuration on the Gateway API side:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
 name: llm-route
spec:
 parentRefs:
 - name: public-gateway
 rules:
 - matches:
 - path:
 type: PathPrefix
 value: /v1/chat/completions
 backendRefs:
 - group: inference.networking.x-k8s.io
 kind: InferencePool
 name: chat-llama3-pool

Some implementations (such as GKE Inference Gateway) can route based on the model field in the request body, mapping OpenAI-style model names to different InferencePool / InferenceObjective combinations.

Fine-Grained Inference Traffic Control Using Metrics

Inference Extension provides platform teams with a unified metrics system, including kv_cache_hits, gpu_utilization, request_queue_length, per-request inference duration, token count, and more.

Based on these metrics, multi-level traffic control strategies can be built:

Priority + Capacity

Set priority and capacity upper limits for different InferenceObjectives, automatically guaranteeing critical business when resources are constrained.
Rate Limiting by Cost / Token

Aggregate token / latency metrics exposed by Inference Extension to Prometheus, then add cost-based rate limiting logic at the gateway / API Gateway level (such as total tokens per minute per user / application). The specification itself doesn’t mandate “Token-level rate limiting,” but provides observability and hooks for easy policy extension.
Prefix Cache Aware Routing

For requests with shared context (such as RAG, template generation), enable the Prefix Cache Aware plugin to route requests with the same prefix to the same replica, maximizing KV Cache hit rates and significantly reducing TTFT.
Auto-Scaling Integration

Use metrics output by Inference Extension as HPA input to achieve “model-aware” auto-scaling, rather than relying solely on CPU / memory.

Relationship and Trade-offs with Traditional Gateway / Service Mesh

Control Plane: Continue using Gateway API as the unified north-south / east-west traffic modeling specification. Service Mesh can still perform fine-grained circuit breaking, retry, mTLS, etc. within the cluster.
Data Plane: Inference Extension pushes “model-aware routing” down to the ext-proc path implemented by the Gateway, avoiding redundant business-side wheel reinvention.
Adoption Strategy: In the current Alpha stage, prioritize productionized implementations (such as GKE Inference Gateway, commercial gateway vendor Inference Gateways), start with “bypass pilots” outside the critical path, and gradually migrate existing AI gateway routing rules to the Inference Extension model.

Summary

Combining official documentation and community implementations, we can see:

Gateway API has become the Kubernetes standard north-south traffic model and continues to enhance in the 1.x series.
Gateway API Inference Extension introduces GPU metrics, KV Cache, LoRA, and other inference semantics into load balancing decisions through InferencePool, InferenceObjective, and Endpoint Picker.
The project is still in Alpha stage, but has achieved experimental or productionized adoption in implementations such as GKE, kgateway, and NGINX Gateway Fabric. It is one of the important future directions for inference traffic control.

Previous descriptions of Inference Extension as “built-in Token rate limiting CRD, AIInferencePolicy, and other objects” are no longer accurate and should all be replaced with the design based on InferencePool / InferenceObjective + metrics-driven approach.

References

TRAE SOLO vs VS Code: Rethinking Coding Tools from the Perspective of AI Engineering Entities

Jimmy Song — Fri, 14 Nov 2025 07:14:39 +0000

Coding tools are evolving from “AI assistants” into true engineering entities. How can we reinterpret the roles of TRAE SOLO and VS Code from a pipeline perspective?

Recently, TRAE International Edition SOLO mode has been fully opened to overseas users. It claims to be a “responsive coding agent” and is now available for official trial, with token-based rate limiting.

I’ve used early versions of TRAE (without SOLO access), and also tried Qoder and Kiro. The AI coding field is flourishing, each tool with its own strengths.

Now, with GitHub’s Agent HQ concept from Universe, and the “AI Engineering Entity (AIEE)” framework I wrote about in AI-Native Application Architecture, it’s time to re-examine today’s coding tool landscape.

This article compares TRAE SOLO and VS Code (with Copilot, Plan/Agent mode, and Agent HQ) from the perspective of AI engineering entities, combining personal experience to outline their differences in engineering automation, collaboration, and governance.

Three Engineering Role Abstractions: End-to-End Executor, Contextual Collaborator, and Expert Orchestrator

From an engineering perspective, current mainstream AI coding tools can be abstracted into three roles:

End-to-End Executor:

Focuses on “requirement to deployment” workflows, capable of autonomous planning, task breakdown, coding, testing, previewing, and even deployment. Officially called “AI-Powered Context Engineer.”
User experience: like a “full-chain executor”—give it a requirement, and it handles the project, even if it’s slow or imperfect.

Contextual Collaborator:

VS Code is a powerful editor. Copilot has evolved from line-level completion to Chat, Plan agent, Agent mode, supporting multi-step tasks and codebase analysis.
It doesn’t take over the whole project, but efficiently handles local tasks under your guidance, acting as an automated unit for specific segments.

Expert Orchestrator / Specialist Engine:

GitHub’s Agent HQ is a “central platform for AI coding agents,” a unified control plane that can connect to OpenAI, Anthropic, Google, xAI, etc., run agents in parallel, and compare results.
Functions as an “expert orchestrator” for key steps—planning, review, refactoring, or decision-making—providing high-quality output without taking over the entire project.

These three roles correspond to the structure in “AI Engineering Entity (AIEE)”:

Single end-to-end executor (TRAE SOLO)
Contextual collaborator residing in the IDE (VS Code + Copilot)
Specialist orchestrator platform for multi-entity scheduling (Agent HQ)

Quick Product Status Check

To avoid memory bias, let’s clarify some key facts.

TRAE / TRAE SOLO

TRAE claims to be a “10x AI Engineer,” able to independently understand requirements and execute development tasks.
SOLO mode is GA for international users, emphasizing full-chain automation, available directly but with token limits.
Underlying open-source Trae Agent CLI can execute multi-step engineering tasks in real codebases.
TraeIDE’s official page shows built-in Claude 3.5/3.7, DeepSeek, etc., but the community notes slow integration of new models like Claude Sonnet 4.5.

So, “TRAE does not support Claude” is now inaccurate—at least officially, Claude models are included. Which model is used in SOLO mode and whether it’s exposed to users remains unclear; the experience still needs improvement.

VS Code + Copilot + Agent HQ

Copilot in VS Code now features Chat, Plan agent, Todo/multi-step execution:
- Plan mode analyzes codebases, generates execution plans, splits into Todos, then implementation agents execute step by step.
- Agent mode provides a more automated “multi-step companion programmer” experience.
GitHub launched Agent HQ at Universe 2025, integrating Copilot and third-party agents (Anthropic, OpenAI, Google, xAI, Cognition, etc.) into a unified control plane, supporting parallel runs and result comparison.

In short:

TRAE is like “embedding an engineering entity into the IDE.”
VS Code + Copilot is “adding a set of engineering entities to a mature IDE.”
Agent HQ is positioned as “headquarters for multiple engineering entities.”

Reconstructing Comparison Dimensions with the AI Engineering Entity Framework

In “AI Engineering Entity (AIEE),” the definition is:

AI has evolved from editor auto-completion to a formal node in the software supply chain, able to receive tasks, produce reviewable artifacts (PR/diff/report), pass tests/gates, and be replaced if it fails. It’s no longer just an “enhanced human developer,” but a “functional engineering unit” in the pipeline.

Based on this, we can reconstruct key comparison dimensions for TRAE and VS Code:

Existence as an Independent Functional Unit
Can it autonomously plan, implement, and produce PRs/reports from natural language requirements, without continuous human intervention?
Context Modeling Capability
Can it model across files, directories, terminal output, and browser content to form a stable engineering context?
Position in the Pipeline
Is it an enhancement layer within the IDE, or a formal node in CI/CD and code review flows?
Reviewability and Replaceability
Are its outputs standardized (PR, diff, report) and suitable for regular pipeline review and rollback?
Multi-Agent Collaboration Capability
Does it natively support multi-agent collaboration, or is it focused on single-agent enhancement?

TRAE SOLO vs VS Code: Engineering Entity Comparison Table

The following table summarizes the main differences from the engineering entity perspective. Note: VS Code includes Copilot Chat + Plan/Agent mode by default and can mount the Agent HQ ecosystem.

You can interpret this table as: “If AI is treated as an engineering entity in the pipeline, what roles do TRAE and VS Code play?”

Table:

Dimension	TRAE SOLO	VS Code + Copilot / Agent
Engineering Role	Single strong entity, directly handles end-to-end tasks from idea to deployment	IDE + multiple entities (Plan, Implementation, Review), IDE itself is the engineering base
Task Granularity	Project/feature level: from PRD-style description to full project scaffold, implementation, testing, preview	Function/file level mainly; Plan mode can scale to feature/subsystem level
Context Modeling	Emphasizes “context engineering”: reads codebase, terminal output, browser content as unified input for SOLO	Mainly codebase; Plan Agent generates plans based on code analysis, Agent mode schedules by plan
Automation	Can proactively modify files, run commands, tests, start local services, forming a complete loop	Plan/Agent can run commands, modify files, run tests, but is more dependent on your current project/workflow
Human Intervention	More “post-review”: let it run first, then review and fine-tune	More “in-process collaboration”: frequent intervention in planning, implementation, and review, with control points at each step
Output Form	Code changes, test results, preview; sometimes PRs/docs	Code completion, refactoring, PR comments, CodeQL reports, Plan/Todo lists
Multi-Agent	Core is the SOLO agent; other capabilities (like Trae Agent CLI) are extensions	Copilot itself is an agent; Agent HQ allows parallel competition among multiple agents
Model Transparency	Product exposes specific models poorly; users can’t tell which model is used	GitHub clearly marks Copilot’s model family; Agent HQ shows agent sources directly
Performance	Strong automation but slow; complex projects may stall at “thinking” stage; hard token limits	Stable response in familiar projects; mostly local changes, overall latency is controllable
Privacy & Compliance	Official and third-party reviews mention extensive telemetry/data collection; enterprise adoption needs extra evaluation	Copilot for Enterprise has clear data isolation/compliance, suitable for most enterprise governance needs

Table 1: TRAE SOLO vs VS Code Engineering Entity Comparison

From the table:

If you want “an AI engineering entity that takes full responsibility from requirement to deployment,” TRAE SOLO fits that role.
If you want “a stable engineering base + a set of pluggable entities,” VS Code + Copilot + Agent HQ fits better.

Workflow Comparison: Two Engineering Entity Pipelines

To clarify their engineering flows, the following diagram illustrates typical workflows for TRAE SOLO and VS Code.

Before the diagram, here’s an introductory sentence:
The following Mermaid diagram visually compares the engineering pipelines of TRAE SOLO and VS Code, highlighting their respective collaboration models.

Figure 1: TRAE SOLO vs VS Code Engineering Entity Pipeline Comparison

This diagram shows two typical collaboration models:

TRAE SOLO attempts to encapsulate “context aggregation → planning → implementation → testing → preview/deployment” within a single engineering entity, with user intervention only at requirement input and output review.
VS Code + Copilot + Agent HQ uses the IDE as runtime, with Plan/Implementation/Review agents corresponding to different roles. Agent HQ supports parallel agent competition, allowing developers to select the best solution.

Model Transparency, Speed, and Predictability

Based on personal experience, here are the model transparency and speed issues from the engineering entity perspective:

Model Transparency

TRAE currently exposes “which model is called” poorly; switching MAX mode only suggests “stronger model or higher quota,” but no clear feedback.
Community feedback notes slow integration of new models; some strong models (like Claude series) are available elsewhere but not yet in TRAE.

This means:

TRAE is hard to use as a “precisely configurable engineering unit,” more like a black box, making model change management in CI/CD or production pipelines difficult.
VS Code + Copilot + Agent HQ is stronger in standardization; GitHub clearly marks Copilot’s model family, Agent HQ uses agent source as the abstraction boundary.

Speed and Predictability

TRAE SOLO’s “slowness” comes from executing more steps (reading files, analyzing, planning, testing) and insufficient engineering process visualization. The UI shows “Thinking…” prompts, making it hard to tell if it’s stuck or planning.
VS Code’s Plan mode explicitly lists plans and Todos; Agent mode emphasizes “execution by plan,” letting users clearly see the entity’s work status, improving predictability.

Agent HQ Positioning: Single Entity vs Multi-Entity Headquarters

From a platform perspective, GitHub and TRAE differ as follows:

Agent HQ’s core idea: future development will rely on multiple specialized agents collaborating in parallel. GitHub is building “agent headquarters,” not a single engineering agent. Developers can schedule agents in a unified control plane, integrating with existing GitHub Flow (Issue, PR, Review, CI/CD).
TRAE is more like “proprietary IDE + agent + full-stack context engineering,” delivering an integrated experience.

In terms of engineering entity organization:

GitHub is building “infrastructure and governance for multi-entity engineering systems.”
TRAE is building “vertically integrated engineering entity + private runtime.”

They’re not mutually exclusive, representing “broad platform + multi-entity scheduling” vs “single strong entity + proprietary toolchain.”

Subjective Experience and Engineering Framework Integration

Translating personal experience into engineering language:

VS Code is more accustomed to a “single IDE + multiple views” experience; TRAE splits IDE and SOLO modes, requiring mental switching.
TRAE’s engineering entity capabilities surpass ordinary completion tools, able to take on tasks, but model transparency and context quality are unstable, and governance needs improvement.
VS Code doesn’t take over the whole project, but local work is stable; Plan, Agent, and Review combinations enable multi-entity collaboration.

According to the “AI Engineering Entity (AIEE)” framework:

TRAE SOLO is a single AI engineering entity (AIEE) capable of handling complete engineering tasks, but still has clear shortcomings in model transparency, engineering governance, and enterprise-level controllability.
VS Code + Copilot + Agent HQ is an infrastructure platform for multiple engineering entities, less aggressive in end-to-end outsourcing in the short term, but clearer in engineering consistency, model replaceability, and organizational governance.

Summary

This article systematically compares TRAE SOLO and VS Code (with Copilot, Agent HQ) from the perspective of AI engineering entities, focusing on automation, collaboration, and model transparency. TRAE SOLO is better suited for individual developers or small teams seeking end-to-end automation, while VS Code + Copilot + Agent HQ provides stronger infrastructure for multi-entity collaboration, enterprise governance, and engineering consistency. In the future, AI engineering entities will become formal nodes in the software development pipeline, and tool selection should be based on engineering needs and governance requirements.

References

Closed-Source Flagships Accelerate, Open-Source Ecosystem Forced to 'Synchronize'

Jimmy Song — Fri, 14 Nov 2025 04:21:07 +0000

Closed-source models are accelerating, while open-source ecosystems are forced to catch up. What engineers truly need to focus on is infrastructure and controllability—not just the surface-level “Twin” phenomenon.

Recently, I came across an email titled: “Every Big AI Model Now Has an Open-Source Twin”. Literally translated, it means “Every major closed-source model now has an open-source sibling.”

From the perspective of media or venture capital, this is an easy story to tell: a closed-source flagship model is released, the community quickly produces an open-source counterpart, and the narrative becomes “open source is catching up with closed source”—the ecosystem is thriving, innovation is accelerating, and the future looks promising.

But from the viewpoint of someone deeply involved in infrastructure, cloud native, and architecture, this narrative has several issues:

It equates “synchronized pace” with “matched capabilities.”
It overlooks the real factors that determine the ceiling: data, compute, and engineering systems.
It blurs a key fact: the open-source ecosystem is fundamentally in a reactive state, not leading in parallel.

This article breaks down the “Open-Source Twin” narrative from an engineering and infrastructure perspective, and shares the core issues I personally care about.

From “There’s a Twin” to “Forced Synchronization”: How the Narrative Changed

Let’s first outline the phenomenon.

In the past two years, the industry has repeatedly seen the following pattern:

Big tech releases a closed-source flagship model (e.g., GPT-5 series, Claude 4/4.5, Gemini 2.5).
Soon after, a batch of open-source counterparts emerge (e.g., Qwen, GLM, Yi, K2), aligning on parameter scale and benchmark metrics.
Media and community start using terms like “open-source twin,” “replacement,” and “counterpart.”

At this level, it’s easy to draw optimistic conclusions: open source has established full benchmarking capabilities; no matter how fast closed source runs, the community can keep up.

But the more critical question is: Who sets the pace, who defines the rules, and who bears the real cost?

The current structure is clear:

Pace is set by closed-source giants: They decide when to boost inference, extend context, push multimodality, or specialize reasoning (like the R1 series).
Open-source ecosystem passively initiates response mechanisms: Each closed-source upgrade triggers a new round of “open-source benchmarking.”

In other words, the current model isn’t parallel innovation or mutual stimulation—it’s closed source constantly shifting gears at the front, with open source adjusting to avoid falling out of sight.

From an engineering perspective, “Every Big AI Model Has an Open-Source Twin” is more accurately:

Every Big AI Model Now Forces an Open-Source Response.

What Exactly Is Open Source “Synchronizing” With?

To understand the “synchronized response” phenomenon, we need to break it down into three categories.

Before listing them, let’s add some context: every closed-source flagship update isn’t just “more parameters, higher scores”—it’s constantly rewriting constraints, including inference cost, interaction patterns, context length, multimodal consistency, and explainability.

In this context, open source isn’t just synchronizing “scores,” but increasingly complex objective functions.

Synchronizing the “Imagination Boundary” of Capability Ceilings

Closed-source models essentially expand “what people think a model should be able to do,” such as:

From pure text to text + image + audio + video.
From single-turn Q&A to engineering-level reasoning, coding, debugging, fixing, and refactoring.
From thousands of tokens of context to hundreds of thousands or more.
From “black box output” to having chains of thought, reasoning traces, and verifiable outputs.

Open-source models then align their goals: “We also need long context, multimodality, coding ability, and agent workflow support.”

Synchronizing the “Expectation Value” of Interfaces and Usage Patterns

Once developers and enterprise users are educated by closed-source models:

How low can interaction latency go?
How long can context be extended without breaking?
How smooth can multimodal input be?
How “smart” can the reasoning process get?

Their expectations for any open-source model are recalibrated.

Thus, open source must:

Continuously optimize inference frameworks (e.g., vLLM, SGLang, TGI) to narrow the latency gap.
Make serving experiences closer to closed-source, such as OpenAI API compatibility and better SDKs.
Forcefully catch up on multimodality and long context, even if training costs are high.

Synchronizing “Surface Metrics,” Not “Complete Capabilities”

From a benchmark perspective, open source can indeed reach 80–90% on public test sets:

MMLU, GSM8K, HumanEval.
Common reasoning, reading comprehension, code generation metrics.

But these metrics only reflect surface capabilities, not:

Robustness to long-tail problems.
Stability in complex, multi-step scenarios.
Reliability and controllability in large-scale production systems.
“Engineering health” over long-term evolution.

This is why I’m skeptical of the “Twin” term: it uses superficial metric similarity to mask deep structural differences.

Why “Open-Source Twin” Sounds Good but Misses the Core Contradiction

From an infrastructure and engineering perspective, the real issue isn’t “can open source copy a similar architecture,” but:

Who can sustainably manage data, compute, scheduling systems, and engineering teams to build a long-term model production pipeline.

There are three key contradictions here.

Data Is Unavailable, Training Recipes Can’t Be Fully Reproduced

Open source can replicate general network structures and optimization tricks, but can’t access:

Closed-source data sources and cleaning standards.
Filtering strategies, detoxification, alignment details.
Large-scale synthetic data generation and selection methods.

Result: Even if you match parameter scale and training steps, the effect may not truly align.

Many open-source projects have to use rougher data, limited compute budgets, and more conservative training strategies, ending up with a “usable but not truly stable” state.

Compute Gap Is Structural, Not Solved by One-Time Funding

Training flagship models requires compute that’s not just hundreds or thousands of GPUs—it’s a structural, long-term investment.

In the open-source camp, those approaching this scale usually have:

Backing from large companies or national labs.
Funding from real business budgets, not community donations.
Compute supply that can be planned long-term, not just a one-off “burn.”

Reality:

Entities truly capable of “flagship open-source models” are essentially “institutions,” not loose personal communities.
Most “open-source twins” are backed by enterprises, with product goals and commercial interests.

So, “open source vs closed source” is more like “many big companies vs a few giants,” not “community vs company.”

“Reproducing” Architecture ≠ “Leading” Architecture

Many open-source models look architecturally similar to closed-source:

Transformer variants, MoE variants.
Minor tweaks at the decision layer.
Some inference optimizations.

But in terms of industry power, those truly pushing these architectures to production scale and validating feasibility are still on the closed-source side. Open source mainly:

Validates closed-source approaches on weaker compute.
Explores “smaller, cheaper” approximations.
Prunes and adapts for specific scenarios.

So, “Twin” is more a marketing term than an engineering one.

My Perspective: What Really Matters in This Game

As an engineer in cloud native, service mesh, and distributed systems, my default thinking when looking at AI infrastructure is:

Treat “models” as just one component in the system, and focus on the underlying infrastructure, scheduling systems, and engineering pipelines.

From this angle, the real concern behind “every closed-source model has an open-source twin” is:

Figure 1: Mermaid Diagram

Can Open-Source Models Stand Firm in Production Over Time?

The focus isn’t whether it can run a demo, but:

Is there a clear upgrade cadence?
Are rollback and compatibility strategies robust?
Is there a sound evolution path for model weights, inference frameworks, and configurations?
Is the entire stack observable and debuggable?

If these infrastructure layers aren’t mature, the so-called “Twin” is just “something that looks similar, but don’t ask if it can support your production workloads.”

Has Training and Inference Infrastructure Formed a Replicable “Engineering Paradigm”?

The real value in open source is whether it can form a unified, teachable, and transferable engineering paradigm, such as:

Training pipeline: data preparation → preprocessing → training → evaluation → alignment → deployment.
Inference infrastructure: how vLLM / SGLang / TGI maintain consistent performance across different GPU topologies.
Scheduling and resource management: how to manage large-scale inference loads on Kubernetes and cloud-native infrastructure.

If these can be established, “open-source twin” isn’t just “we also have a model,” but a reusable, transparent, and learnable engineering system.

The True Value of Open Source: Controllability and Bargaining Power, Not Absolute Performance

Realistically, closed-source flagships will continue to lead in overall capability for the foreseeable future: larger scale, more complex training, better data, richer scenario tuning.

For enterprises and developers, the key value of open source isn’t “I want to fully replace closed source,” but:

Maintaining controllability over technical direction.
Gaining bargaining power, avoiding vendor lock-in.
Building your own model stack in privacy-sensitive or compliance-heavy scenarios.

From this perspective, the “Twin” term should be soberly rewritten as:

In many scenarios, open source can provide a controllable, more flexible alternative path—but it’s not a mirror of closed source, it’s a separate engineering decision space.

Practical Advice for Engineers and Teams: Don’t Worship “Twins,” See the Structure Clearly

Before the final summary, here are my actionable views on this topic.

Premise: If you’re an engineer, architect, or technical leader, your real decision isn’t “choose open source or closed source,” but:

Under your business constraints, how do you combine closed-source APIs, open-source weights, and self-built infrastructure to create an evolvable, observable, and portable system.

With this in mind, “Every Big AI Model Has an Open-Source Twin” breaks down into several sober judgments:

When you see an “open-source twin,” first ask: can it run stably in your production environment over time, not just pass benchmarks?
What you really need to understand: is there a clear story behind its training/inference infrastructure, not just a weight download link?
Reframe “open source vs closed source” as “where do I need closed source (capability/cost), and where do I need open source (controllability/compliance)?”
If you’re working on infrastructure and platform layers, focus on:
- How to run different models in a unified scheduling, monitoring, and logging system.
- How to treat large models as observable, governable services on Kubernetes/cloud-native stacks, not mysterious black boxes.

Summary

Closed-source flagships keep accelerating, shifting gears, and adding dimensions, while open-source ecosystems are forced to develop increasingly mature synchronized response mechanisms. What truly determines the gap is data, compute, and engineering infrastructure—not just a single model release.

Personally, my focus will remain on:

The evolution of inference infrastructure (like vLLM, SGLang, TGI).
Training and scheduling: how to stably manage model lifecycles in cloud-native environments.
Engineering paradigm accumulation: moving from “can run” to “reproducible, maintainable, and evolvable.”

Key Takeaways from Ingress NGINX Retirement: Managing Technical Debt in Cloud Native Migration

Jimmy Song — Thu, 13 Nov 2025 01:43:05 +0000

The evolution of cloud native infrastructure inevitably faces the reality of technical debt and governance. The retirement of Ingress NGINX is a profound reminder about standardization and sustainability.

Kubernetes officially announced: Ingress NGINX will be completely discontinued in March 2026. This is not just a typical project sunset, but a landmark event in the evolution of the Kubernetes networking model. It signals the inevitable shift of the tech stack from “flexible but fragile” to “controllable and governable.”

As someone who has long promoted Kubernetes and cloud native practices, I have witnessed both the golden age of Ingress NGINX and the gradual accumulation of its technical debt. Here are the clear insights this event has brought me.

Technical Debt Will Eventually Backfire, Especially for Infrastructure Components

The core issue with Ingress NGINX is not a decline in users, but that “maintenance costs permanently exceed the pace of contributions.” High flexibility leads to a huge attack surface, years of complex configuration legacy, and a shortage of community maintainers, ultimately making the project unsustainable.

Once infrastructure components can no longer be securely updated, they cease to be assets and become liabilities.

My Conclusion

The threshold for future infrastructure will be higher, with stricter requirements for security and maintainability. The model of individual hero maintainers will continue to fail.

Kubernetes Officially Enters the Gateway API Era

Before introducing Gateway API (Gateway API, Gateway Application Programming Interface), it’s important to review the design of Ingress. Ingress was once praised for its simplicity, but now it cannot meet modern needs for traffic management, scalability, security policies, and multi-team collaboration.

Gateway API is designed with a more modern philosophy:

Governance model across roles (Infra / Dev / Ops)
Strong CRD (Custom Resource Definition) extensibility
Pluggable implementation
Significantly improved observability and lifecycle management

This means: The entire ecosystem is moving from “controller differentiation” to “API standardization” at the traffic layer.

Most Users Are Unprepared for the Complexity of the Underlying Network Stack

Long-term community observation shows that most users treat Ingress NGINX as a black box. Now, migrating from Ingress to Gateway API or other Ingress controllers represents a “hidden migration wave” for many clusters.

This announcement highlights two points:

When a “default component” in a complex system stops being updated, it brings widespread invisible risks
The cloud native ecosystem needs long-term, sustainable supply chain governance

Security Is the Final Straw

The official announcement repeatedly emphasizes that security risks and vulnerabilities can no longer be continuously fixed. This once again proves: Flexibility and security are always a tradeoff, and the closer a component is to the data plane, the less compromise is acceptable.

The “Individual Maintainer Bottleneck” in Cloud Native Will Become More Pronounced

Ingress NGINX has long relied on just one or two maintainers, and ultimately had to retire. This exposes a long-standing issue in the open source world: Critical projects are heavily relied upon, but contributions are insufficient.

The future of infrastructure is clear:

Large companies will be more willing to invest in core open source infrastructure
Individual maintainers cannot support critical foundational components
The boundary between commercialization and open source will continue to tighten

My Personal Takeaway: Gateway API, L7 Traffic Management, and the Integration with AI Native Infra

The retirement of Ingress NGINX points to an underlying trend: Unified and extensible APIs will become the dominant paradigm for cloud native infrastructure.

The AI-Native infrastructure I’m researching—such as inference routing, model gateways, AI Gateway, and Agent Orchestrator—will follow a similar path: from early flexible hacks to mature, standardized, and governed APIs.

Summary

Ingress NGINX is arguably one of the most important control planes in Kubernetes history. Its retirement is not a failure, but an inevitable result of the system advancing to the next stage.

For me, this is a strong reminder:

Technical debt cannot be avoided
Infrastructure must be built for the long term
Standardized APIs are the future
Sustainable open source requires collective investment
The convergence of AI and cloud native will follow the same evolutionary trajectory

References

What Makes an AI Platform Truly Kubernetes-Native?

Jimmy Song — Wed, 12 Nov 2025 12:39:18 +0000

Standardizing cloud-native AI platforms is a key step in advancing the AI infrastructure ecosystem.

In recent years, the cloud-native ecosystem has gradually expanded from general-purpose computing to AI workloads. The CNCF (Cloud Native Computing Foundation) is driving a new certification initiative—Kubernetes AI Conformance—aimed at establishing a set of technical standards for AI platforms to be compatible and interoperable with Kubernetes.

This certification seeks to answer a core question:

“What does it take for an AI platform to be truly Kubernetes-native?”

Why AI Conformance Is Needed

Currently, many AI platforms claim to “run on Kubernetes,” but their actual integration varies greatly. Here are some common scenarios:

Some platforms merely run containers on Kubernetes without deep integration with the control plane.
Others fully integrate with Kubernetes control plane, scheduling, and observability systems.
Many vendors build their own controllers, schedulers, and storage interfaces, resulting in migration and interoperability challenges across environments.

The core purpose of CNCF’s AI Conformance is to unify standards so that AI platforms behave consistently across clouds and clusters, becoming a common language for the ecosystem—much like “Certified Kubernetes.”

Key Criteria for Kubernetes-Native AI Platforms

A Kubernetes-native AI platform must meet several key criteria:

Architecture-Native: Everything as Kubernetes Objects

For AI training, inference, and batch processing scenarios, all tasks should be declared as Pod, Job, or CRD (Custom Resource Definition) objects. Scheduling, scaling, and lifecycle management should be handled by the Kubernetes control plane, not by custom platform logic.

For example, Kubeflow Training Operator, RayCluster CRD, and vLLM Operator all use this native object declaration approach.

Scheduling-Native: Unified Compute Resource Scheduling

AI platforms need to collaborate with Kubernetes Device Plugin and Scheduler to detect GPU, NPU, and other heterogeneous compute resources, supporting resources.requests/limits for resource management. Task scheduling should be observable and traceable, avoiding black-box operations.

Storage-Native: Declarative Data and Model Access

Data and model access should not rely on host paths but use PVC (PersistentVolumeClaim), CSI (Container Storage Interface), S3/NAS, and other standard interfaces for mounting. Credentials and sensitive parameters should be injected via Secrets and ConfigMap. The entire pipeline should be reproducible via GitOps/CI/CD workflows, ensuring traceability and automation.

Network and Service-Native: Compatible with Mesh and Gateway

AI inference services should be exposed as standard Service, Ingress, or Gateway API resources, supporting multi-cluster service discovery and routing policies, and integrating seamlessly with service meshes like Istio, Envoy, and Linkerd.

Additionally, platforms should output standardized monitoring metrics (e.g., Prometheus), logs (e.g., FluentBit), and tracing data (e.g., OpenTelemetry) for unified observability and operations.

Portability and Interoperability

A truly Kubernetes-native AI platform should behave consistently across environments, including:

Public clouds (EKS, GKE, ACK)
Private clouds (OpenShift, KubeSphere)
Bare-metal clusters

The platform should also integrate directly with mainstream ecosystem components such as Kubeflow, Ray, KServe, and Triton, achieving high interoperability.

CNCF’s Goal: From “Running on Kubernetes” to “Growing within Kubernetes”

CNCF aims to use the AI Conformance certification mechanism, much like Certified Kubernetes, to drive the AI infrastructure ecosystem toward standardization.

In the future, the industry may see:

The Certified AI Platform badge as a trust mark for platforms.
Automated verification bots (Verify Conformance Bot) to improve testing efficiency.
Multi-version test suites (e.g., v1.33, v1.34) to ensure compatibility.

These measures will become important technical thresholds and trust foundations for cloud vendors, AI platforms, and open-source AI infrastructure projects.

Summary

In the AI era, standardization is the foundation for ecosystem evolution. For AI platforms to thrive in the cloud-native world, they must not only “run on Kubernetes” but also “grow within Kubernetes.”

A truly Kubernetes-native AI platform should feature:

Control plane compatibility, transparent data plane, declarative extensibility, portability, observability, and reproducibility.

This is the key intersection of AI and cloud-native—and the foundation for the next generation of AI infrastructure.

ChatGPT Atlas Architecture Explained: Size, Unique Design, and Agent Performance Issues

Jimmy Song — Tue, 11 Nov 2025 01:46:24 +0000

The size and complexity of AI Native browsers are not drawbacks, but rather the inevitable result of a new operating system paradigm.

Why Is Atlas So Much Larger Than Chrome? What’s Actually Added?

Atlas’s installation size reaches several gigabytes, which is no accident. It is not just a traditional “browser shell” but integrates multiple core components:

Chromium Rendering Engine + Independent AI Runtime + Agent Sandbox + Global Context Pipeline—all in one.

Chrome is merely a web renderer, while Atlas has become an AI workflow execution environment.

Atlas’s architectural design is the reason its size far exceeds that of traditional browsers.

The Core Difference Between Atlas and Chrome: A Complete AI Runtime Added

Many assume Atlas is simply “Chromium + AI features,” but the reality is much more complex. On top of Chromium, Atlas adds a full AI sub-runtime, forming an independent system-level sub-operating environment.

The following flowchart illustrates the AI subsystem Atlas adds on top of Chrome:

Figure 1: Atlas Adds AI Sub-runtime on Top of Chrome

This is not a simple feature extension, but a system-level architectural upgrade.

Atlas’s Local Data Structure: Not Just Cache, But a Complete Browser + AI Subsystem

When Atlas starts for the first time, it generates an independent host profile locally at:

/Users/<user>/Library/Application Support/com.openai.atlas/browser-data/host/

Compared to Chrome, Atlas’s data structure is much larger and includes various core data:

12K AmountExtractionHeuristicRegexes
960K AutofillStates
4.0M BrowserMetrics-spare.pma
169M component_crx_cache
93M extensions_crx_cache
1.1M OpenCookieDatabase
2.7G OptGuideOnDeviceModel
19M Safe Browsing
123M screen_ai
1.7G user-<uuid>
...

This data is not just browser cache, but also includes:

Complete Chromium Profile (Cookies, Local State, Shader, Safe Browsing, etc.)
Atlas-specific AI models and feature data
- OptGuideOnDeviceModel (Inference guide model)
- screen_ai (Page structure understanding model)
- WasmTtsEngine
Agent execution traces and context persistence
- All stored in user-<uuid>

Chrome only needs to store rendering cache, while Atlas must also save AI runtime state, DOM semantic summaries, Agent execution traces, and large language model (LLM, Large Language Model) context fragments. Naturally, the size is measured in gigabytes.

Why Atlas Has Only One Host Profile

Chrome supports multiple profiles, but Atlas only has host/ and user-<uuid>. The reasons are:

Agent workflows depend on global context and cannot be fragmented
AI Memory needs to be shared across pages
OpenAI’s IPC and sandbox design currently only support a single principal

This is an architectural limitation, which may change in the future.

Atlas’s Process Architecture: More Complex Than Chrome

Chrome’s multi-process architecture mainly includes:

Main process
Extension process
Rendering/network process

Atlas’s process architecture is even more complex, including:

Main process (Chromium Shell)
Rendering process (DOM/JS)
AI Runtime process
Agent Sandbox isolation environment
Page extraction and semantic parsing process
Security policy and permission arbitration process
Model inference coordination layer (LLM Orchestrator)

Each layer communicates strictly via inter-process communication (IPC, Inter-Process Communication), with no “script-level execution,” ensuring security and reliability.

How Atlas Agents Run: Why Are They Slow? Why Not Like Scripts?

Atlas Agents do not execute scripts directly, but complete tasks through multi-round inference and sandbox mechanisms. Every action involves a full round of large language model (LLM, Large Language Model) inference, DOM observation, sandbox execution, and re-inference.

The Agent execution flow is as follows:

LLM decides the next step
Sandbox executes the action
Generates new page observation (structured DOM)
LLM infers the next step again
Loop until the task is complete

Therefore, Atlas Agent execution features include:

Always slower than scripts (e.g., Playwright/Selenium direct execution)
Must be isolated for security
Multi-round inference ensures reliability
Structured DOM helps models understand page content

This is the fundamental reason why Agent execution is slow.

Atlas Is Evolving Toward an “AI Native OS”

Atlas’s capabilities have far surpassed traditional browsers and are gradually evolving into an AI Native Operating System (AI Native OS). The table below compares their core capabilities:

Here is a comparison table between Atlas and traditional browsers:

Capability	Browser	AI OS (Atlas)
Render Web Pages	✔	✔
Execute JS	✔	✔
Semantic Page Understanding	✘ (Only Parsing)	✔
Automated Task Execution	Plugin	Built-in
Task-level Reasoning	✘	✔
Cross-page Workflows	✘	✔
Agent Security Sandbox	✘	✔
Global AI Context	✘	✔

Table 1: Comparison of Traditional Browser and AI OS (Atlas) Capabilities

Atlas already possesses 30%–40% of the core capabilities of an AI OS. It is not just a browser, but:

AI Runtime + Rendering Engine + Agent Workflow Executor

Summary

On the surface, Atlas looks like a browser, but in reality, it is a heavyweight AI Runtime:

Massive size: Stores AI models, feature data, and Agent context locally
Complex processes: Security, inference, and DOM extraction all require independent pipelines
Slow Agent execution: Every step requires inference, not direct script execution
Huge profile: Saves the state of the AI system, not just simple cache

Chrome is a web processor, while Atlas has become an AI workflow engine—they are fundamentally different technological species.

Two Weeks Deep with ChatGPT Atlas: A Developer's Perspective on Real Potential and Structural Gaps

Jimmy Song — Tue, 11 Nov 2025 01:26:05 +0000

The future of AI browsers isn’t about stacking features, but about unifying developer context and workflow. Atlas has already changed my daily routine, but there are still key gaps to fill.

Why Did I Switch to Atlas as My Main Browser on Day One?

Figure 1: Using ChatGPT for 1000+ days, Atlas for 19 days

As a long-term user of multiple developer tools—ChatGPT (GPT-4/5, Codex, o1 series), Chrome (multi-tab, high-density search), VSCode (local project environment), macOS ChatGPT Desktop (local context reading), and local Hugo / Flask / FastAPI debugging—I was drawn to Atlas’s design philosophy and immediately made it my main browser. After two weeks of deep use, my core takeaway is:

Atlas transforms the browser from a “renderer” into a “host for AI Runtime”.

This article systematically reviews Atlas’s architectural strengths, workflow enhancements, structural pain points, and future directions from a developer’s perspective.

Real Workflow Enhancements Atlas Brings to Developers

Atlas’s core innovation is the native integration of local development environments with AI, dramatically improving developer workflow efficiency.

Native Fusion of Local Development and AI: `localhost` Access

Previously, browsers and AI tools were two separate worlds: browsers could see local services, ChatGPT could only analyze uploaded cloud fragments, and AI couldn’t directly sense the real state of your project. Atlas is the first to enable AI to directly read local services.

The following diagram illustrates how local services integrate with Atlas:

Figure 2: Atlas Local Service Integration Flow

This capability is significant for developers:

When debugging APIs, ChatGPT can directly view response content.
During documentation preview, AI can compare raw files and rendered results.
Hugo / SSG local preview lets AI read full HTML.
Quickly review local error pages for more efficient troubleshooting.
For the first time, the local project environment is “read into” the AI reasoning space.

These are capabilities that Chrome + Web ChatGPT currently do not support.

Sidebar ChatGPT: Persistent “Task Threads”

Atlas’s sidebar ChatGPT is no longer the traditional “one question, one answer” model, but a persistent task thread. Developers can switch between multiple tabs while keeping the same conversation history, truly realizing the “assistant layer” experience.

The following sequence diagram shows a typical developer workflow:

Figure 3: Atlas Sidebar Task Thread

Advantages include:

Conversation history is fixed on the left, never covered by the page.
No need to switch chat windows, focus on the task flow.
Multi-tab reading is uninterrupted; AI becomes a true developer assistant.

Atlas’s Structural Pain Points (Developer Review)

Despite Atlas’s innovations, there are still clear structural shortcomings in real development scenarios.

Single Conversation Can Only Reference One Tab—Context Is Limited

Developers often need to compare multiple docs, APIs, implementations, or architecture diagrams, but currently ChatGPT conversations can only bind to the visible tab’s content, making multi-page reasoning impossible.

The diagram below shows the ideal multi-page binding approach:

Figure 4: Multi-Page Bound Conversation

In the future, AI browsers should support binding a conversation to multiple pages for true multi-context reasoning.

Inconsistent Conversation UI Across Desktop, Web, and Atlas

Although Atlas, Web ChatGPT, and ChatGPT Desktop use the same conversation_id and unified context, their display strategies differ, resulting in “historically consistent but not real-time consistent” experiences.

Web / Atlas (Browser) Every message is immediately written to the server, so updates are visible in real time.
ChatGPT Desktop To support local context and faster rendering, it uses a local cache model; it automatically pulls updates from Web/Atlas, but its own updates are not automatically pushed back to the browser.

The result:

The context is always consistent across all three, but UI refresh rates differ: Desktop → Web/Atlas does not auto-sync; Web must refresh to see the latest messages, and Atlas’s sidebar currently has no refresh option.

This can cause a common workflow split for developers: “It’s the same conversation, but different endpoints show different content.”

Missing Prompt Templates and Quick Command System

Prompt templates (for code review, doc rewriting, bug retrospectives, API summaries, architecture reviews, etc.) are crucial in daily development. In Chrome, plugins can provide this, but Atlas currently lacks such features.

The table below summarizes the ideal template and command system capabilities:

Capability	Ideal Behavior
Prompt Template Library	Callable from sidebar, variable support
Custom Commands	`/review`, `/refactor`, `/summarize`
Multi-device Sync	Consistent across desktop, browser, future mobile
Context Awareness	AI auto-matches your current task

Table 1: Prompt Template and Command System Capability Comparison

Atlas does not yet support prompt templates or quick command systems.

Atlas Hides Some DevTools Capabilities

Atlas’s DevTools are a full Chromium suite—HTML, network, performance, console debugging are all available. However, the current version has removed Chrome DevTools’ “Device Mode” for mobile emulation, including UA/viewport switching, touch simulation, and responsive mode. So, mobile debugging is temporarily unavailable in Atlas. This isn’t a missing underlying capability, but rather the UI hasn’t exposed the relevant entry yet.

No Mobile Version—Workflow Context Is Fragmented

Atlas currently has no mobile version, so commuting, travel, and fragmented time cannot be linked with desktop workflows. Context, history, tabs, task areas, and memory cannot sync, impacting the overall developer experience.

Agent Speed and Reliability Issues

Atlas’s Agent execution is slow, sometimes stalls or fails to complete tasks, feedback mechanisms are opaque, and there’s a lack of visual interruption controls. These issues make it hard to adopt in production tasks.

Atlas, Chrome, and ChatGPT Desktop—Core Capability Comparison

The table below summarizes the differences in core developer capabilities, helping Hacker News readers quickly understand the technical positioning.

Capability	Chrome	ChatGPT Desktop	Atlas
Browse Web Pages	✔	✘	✔
Local Context (VSCode/CLI)	✘	✔✔	✘
Localhost Service Reading	✔ (browser)	✘	✔✔ (AI readable)
Deep ChatGPT Integration	✘	✔	✔✔
Tab ←→ Conversation Binding	✘	✘	✔
DevTools	✔✔	✘	✔
Multi-page Conversation Injection	✘	✘	✘
Prompt Template System	Plugin	✘	✘
Mobile	✔	✔ (ChatGPT App)	✘
Agent Automation	✘	✔	✔ (but immature)

Table 2: Atlas vs Chrome vs ChatGPT Desktop Core Capability Comparison

Atlas’s uniqueness lies in:

Atlas is the first mainstream AI browser to natively inject localhost pages directly into conversation context, but it’s not the only solution for accessing local content.

The Future of AI Browsers: Not “Smarter”, But “More Unified”

The future of AI browsers should focus on unified context and workflow, not just stacking features. The diagram below shows the ideal architecture:

Figure 5: Future AI Browser Architecture

The ideal AI browser should have:

Unified context graphs across tabs, conversations, and devices.
Semantic parsing of web page structure.
Unified reading of local files, APIs, and runtimes.
Observable, interruptible Agent task execution.
Unified prompt template runtime layer.
Collaborative knowledge trajectory (memory graph) across devices.
Extensible plugin mechanism.

Currently, no AI browser has all these capabilities—including Atlas. But Atlas is on the right track, with its native capabilities still evolving, and is already ahead by about 20%.

Summary

After two weeks of deep use, my conclusions are:

Atlas is far from mature, but already powerful enough to significantly change how I read docs, write code, and analyze pages.
It lowers the barrier between AI and web content, reducing my reliance on ChatGPT Desktop.
But it still lacks key capabilities developers truly need: multi-page injection, native prompt templates, full DevTools support, mobile, and a reliable Agent Runtime.

Atlas now feels like:

Chrome + ChatGPT + localhost integration

But to become

The next-generation operating system (AI OS) for developers

It still needs to fill in those critical gaps.

The direction is right, and I’ll keep using it—and watching to see if it can become the default toolchain for developers in the future.

KAITO and KubeFleet: CNCF Is Reshaping AI Inference Infrastructure

Jimmy Song — Sat, 08 Nov 2025 17:40:00 +0000

Declarative and multi-cluster capabilities of cloud native are becoming the foundation for standardized AI inference infrastructure.

AI inference is rapidly emerging as the next frontier for cloud native infrastructure. As large language models (LLM, Large Language Model) grow in capability and scale, traditional single-cluster inference architectures struggle to meet global, high-availability, and cost optimization requirements. In late October 2025, CNCF announced two new hosted projects—KAITO (Kubernetes AI Toolchain Operator) and KubeFleet—marking the cloud native community’s official entry into standardized AI inference infrastructure.

This article provides a systematic analysis of both projects and explores their strategic significance for the AI Infra ecosystem.

The Complexity of AI Inference: From Single Cluster to Multi-Cluster

As inference workloads for large models evolve, enterprises are adopting multi-cluster architectures. Below are three major challenges introduced by multi-cluster setups:

Deployment consistency: Managing model versions, dependencies, and configuration drift across clusters is difficult.
Scarce compute resources: Intelligent scheduling of available GPUs is required to avoid resource waste or hotspots.
Service reliability: Inference endpoints must deliver low latency, high availability, and cross-region SLAs.

KAITO and KubeFleet are designed to address these challenges.

The following diagram illustrates the architecture of KAITO and KubeFleet.

Figure 1: KAITO and KubeFleet Architecture

The architecture can be summarized as follows:

The top layer is the KubeFleet Hub Cluster, which controls multi-cluster placement logic.
The lower layer consists of three regional clusters (US / EU / APAC), each with Active Nodes and Spare GPUs.
The Inference Gateway provides a unified global inference entry point.
Arrow directions represent the control flow of placement and aggregation.

KAITO: Declarative Orchestration for AI Inference

KAITO (Kubernetes AI Toolchain Operator), initiated by the Microsoft team, is a declarative AI workload management framework. It abstracts model lifecycle management via CRDs (Custom Resource Definitions), making LLM inference as configurable and reusable as microservice deployment.

Project URL: github.com/kaito-project/kaito

The table below summarizes KAITO’s core features and design principles:

Feature/Principle	Description
Workspace Model Mgmt	Supports both pre-trained and BYO (Bring Your Own) models
Automatic Resource Allocation	Dynamically requests nodes and volumes based on model size and GPU availability
Multi-node Optimization	Supports distributed storage and compute scheduling
Built-in Observability	Directly outputs inference latency, throughput, and error metrics
Declarative Deployment	Models are treated as native Kubernetes resources, supporting YAML config and GitOps

Table 1: KAITO Core Features and Design Principles

For example, an inference pipeline can be declared in YAML:

apiVersion: aitoolchain.io/v1
kind: ModelDeployment
metadata:
 name: qwen2-7b
spec:
 model: qwen2-7b
 engine: vllm
 replicas: 3
 resources:
 gpu: 2

This enables AI platforms to achieve the same deployment consistency and GitOps capabilities as application services.

KubeFleet: Intelligent Multi-Cluster Scheduling and Placement

KubeFleet, led by the Azure Kubernetes Service (AKS) team, is a multi-cluster orchestrator focused on intelligent placement of inference workloads.

Project URL: github.com/kubefleet-dev/kubefleet

The table below highlights KubeFleet’s key features and use cases:

Feature/Use Case	Description
Cluster Capability Discovery	Evaluates each cluster’s GPU type, quantity, cost, and location
Intelligent Placement	Deploys inference tasks to the most suitable cluster based on policies
Staged Updates	Supports canary releases across test, staging, and production clusters
Consistency Control	Ensures unified deployment templates across clusters
Global Inference Service	Supports geo-distributed inference
Heterogeneous GPU Pool Scheduling	Enables enterprise-grade unified deployment across environments

Table 2: KubeFleet Key Features and Use Cases

KAITO × KubeFleet: Layered Design of AI Inference Infrastructure

The following table summarizes the layered positioning of KAITO and KubeFleet in AI inference infrastructure:

Layer	Responsibility	Representative Project
Global Placement Layer	Decides which cluster	KubeFleet
Cluster Orchestration Layer	Defines model deployment	KAITO
Runtime Layer	Executes inference engine	vLLM / TGI / SGLang / Triton
Infra Layer	Provides compute and scheduling	Kubernetes / GPU / CNI / Storage

Table 3: Layered Design of AI Inference Infrastructure

This layered approach reflects CNCF’s consistent philosophy: abstracting complex infrastructure through declarative and pluggable methods to lower the entry barrier for AI inference platforms.

Ecosystem Significance and Trend Analysis

AI Infra is undergoing cloud native transformation, with CNCF integrating AI workloads into its governance system. This will drive AI platforms to gradually adopt a standardized stack aligned with cloud native principles. Multi-cluster scheduling is becoming the new battleground, and GPU heterogeneity and cross-region compliance are pushing enterprises toward multi-cluster inference architectures. KubeFleet may become the “AI Federation” successor to Karmada / Clusternet. Declarative AI operations will replace manual script-based deployments, and KAITO’s CRD model could become the standard semantic layer for future ML serving. The strategic collaboration between Microsoft and CNCF is strengthening, as both projects originate from the Azure team, signaling that cloud vendors are participating in the AI ecosystem through open infrastructure standards.

Comparison with Existing Projects

The table below compares KAITO, KubeFleet, and mainstream AI inference infrastructure projects:

Feature	KAITO	KubeFleet	Kubeflow	KServe	HAMI
Declarative Model Deployment	✅	–	✅	✅	–
Multi-cluster Scheduling	–	✅	–	Partial	✅
GPU Heterogeneity Awareness	✅	✅	Partial	✅	✅
Telemetry / Metrics	✅	✅	✅	✅	✅
Cloud Vendor Support	Microsoft / CNCF	Microsoft / CNCF	Google	IBM / RedHat	AWS

Table 4: Feature Comparison of AI Inference Infrastructure Projects

Summary

The emergence of KAITO and KubeFleet marks a pivotal moment in the evolution of AI Infra. They represent the cloud native community’s formal engagement with AI inference and reveal future trends:

The complexity of AI inference will be absorbed by Kubernetes’ declarative and multi-cluster systems.
Both projects should be considered essential references for anyone researching AI-native infrastructure.
For developers and platform teams, they are not just new tools but signals of AI infrastructure standardization.

References

Building Efficient LLM Inference with the Cloud Native Quartet: KServe, vLLM, llm-d, and WG Serving

Jimmy Song — Sat, 08 Nov 2025 05:21:59 +0000

The standardization and modularization of cloud native inference systems are making large model deployment as simple and efficient as web services.

Large language model (LLM, Large Language Model) inference is evolving from the era of single-machine accelerators to distributed cloud native systems. The most representative combination today is KServe, vLLM, llm-d, and WG Serving. Each plays a distinct role—standard interface, execution engine, scheduling layer, and collaboration specification—forming a scalable, observable, and governable inference foundation.

The following timeline outlines the key milestones in the evolution of the quartet:

Figure 1: Cloud Native LLM Inference Quartet Evolution Timeline

Architecture Overview

The diagram below illustrates the layered collaboration of the quartet within the inference system:

Figure 2: Cloud Native LLM Inference Quartet Architecture Overview

KServe: The Core of Cloud Native Model Serving

KServe is a Kubernetes-native inference control plane. It abstracts model services as CRDs (Custom Resource Definitions), making AI inference as deployable, scalable, and upgradable as microservices.

The table below summarizes KServe’s core capabilities and new features:

Dimension	Description
Core Goal	Provides Kubernetes-native inference standards and control plane
Core Features	CRD standardization, elastic scaling, traffic governance, unified gateway entry
New Features	LeaderWorkerSet support, AI Gateway integration, llm-d integration

Table 1: KServe Core Capabilities and New Features

KServe’s key capabilities include:

Unified Interface: The InferenceService CRD defines input/output protocols, compatible with REST/GRPC and OpenAI API.
Elastic Scheduling: Supports automatic GPU scaling and ModelMesh multi-model hosting.
Traffic Governance: Canary releases, A/B testing, and InferenceGraph.

The latest version introduces the LeaderWorkerSet (LWS) mechanism and Envoy AI Gateway extension, making multi-Pod large model inference a native capability. KServe is transitioning from a traditional ML service platform to the standard control plane for generative AI.

vLLM: High-Performance Inference Execution Engine

vLLM focuses on extreme throughput and memory efficiency, setting the benchmark for open-source performance.

The sequence diagram below shows the main vLLM inference process:

Figure 3: vLLM Inference Process

The table below summarizes vLLM’s core technical mechanisms and effects:

Feature	Technical Mechanism	Effect
PagedAttention	Memory paging	Longer context, less fragmentation
Continuous Batching	Dynamic batch scheduling	Higher GPU utilization
Prefix Cache	Prefix reuse	Lower latency and cost

Table 2: vLLM Core Technical Mechanisms and Effects

vLLM is compatible with the OpenAI API, supports INT8/FP8 quantization and various parallel modes, and adapts to NVIDIA, AMD, TPU, and Gaudi hardware. In single-machine or small-scale scenarios, vLLM can run independently; in cluster environments, it serves as the execution foundation for KServe/llm-d.

llm-d: Distributed Inference Scheduling Layer

llm-d is a large model scheduling and orchestration system for Kubernetes, enabling multi-instance collaboration for vLLM. Its design goal: make clusters infer like a single machine.

The table below summarizes llm-d’s core mechanisms and technical highlights:

Module	Function	Technical Highlight
Scheduler	Cache-aware routing	Prefix affinity scheduling
Prefill/Decode Separation	Heterogeneous hardware optimization	A100 Prefill + L40 Decode
Cache Manager	Global cache index	Hierarchical GPU/CPU/NVMe cache

Table 3: llm-d Core Mechanisms and Technical Highlights

The following diagram illustrates llm-d’s distributed scheduling and caching mechanism:

Figure 4: llm-d Distributed Scheduling and Caching Mechanism

llm-d runs under the KServe control plane in a Leader/Worker pattern. The scheduler can be embedded in Envoy or deployed independently, making real-time routing decisions based on cache and load information. Its emergence enables autonomous scheduling and elastic parallelism for multi-node LLM inference.

WG Serving: Collaboration Standards and Ecosystem Hub

WG Serving is an AI Serving working group promoted by the Kubernetes community, defining unified inference semantics in K8s.

The table below summarizes WG Serving’s core achievements and standardization contributions:

Achievement/Standard	Description
Gateway Inference Extension (GIE)	Envoy-based inference gateway protocol supporting model identification, streaming forwarding, priority, and cache affinity routing
LeaderWorkerSet CRD	Explicitly describes Leader–Worker collaboration structure, foundational for llm-d and KServe multi-Pod inference
Interface Alignment	Advocates OpenAI-style API integration with K8s resource objects, promoting cross-framework interoperability

Table 4: WG Serving Core Achievements and Standardization Contributions

GIE is the ‘unified gateway language’ for cloud native AI inference, just as Ingress defines HTTP service entry, it defines the standard semantics and gateway behavior for inference requests within Kubernetes, enabling composable, observable, and extensible inference systems.

The diagram below shows WG Serving’s standardized collaboration within the inference system:

Figure 5: WG Serving Standardized Collaboration

WG Serving is not a product, but a standard layer forming industry consensus, driving the unified language for cloud native AI inference.

Combined Architecture

The table below summarizes the division of labor and roles of the quartet in the system:

Layer	Component	Role
Entry Layer	Envoy + GIE	Unified API gateway and traffic hooks
Control Layer	KServe + LWS	Lifecycle management, elastic scaling, traffic orchestration
Scheduling Layer	llm-d	Prefix-aware routing, cross-Pod collaboration, cache management
Execution Layer	vLLM	Efficient inference execution and cache reuse

Table 5: Cloud Native LLM Inference Quartet Division of Labor and Roles

The diagram below illustrates the synergy among the quartet:

Figure 6: Cloud Native LLM Inference Quartet Synergy

Clients send requests in OpenAI API format, routed by the GIE gateway to the optimal Leader, with Prefill completed and cache passed to Worker Decode, finally streaming the result back. The entire chain features standard interfaces, high throughput, and elastic scaling.

Ecosystem Convergence Trends

The table below summarizes the convergence trends and feature comparisons in the cloud native LLM inference ecosystem:

Trend/Feature	Description
API Unification	OpenAI-style interfaces have become the de facto standard; KServe and vLLM natively compatible
Module Decoupling	Gateway, scheduling, and inference are layered for independent evolution and replacement
Hierarchical Caching	GPU–CPU–NVMe three-level KV cache is mainstream
Community Collaboration	WG Serving, PyTorch Foundation, CNCF jointly promote cross-project integration

Table 6: Cloud Native LLM Inference Ecosystem Convergence Trends and Feature Comparison

The following matrix compares the core capabilities of each project:

Project	Control Plane	Inference Performance	Distributed Capability	Interface Compatibility	Cache Mechanism	Elastic Scaling
KServe	✅ CRD / LWS	⚪ Medium	⭐ Multi-model management	✅ OpenAI API	⚪ None	✅
vLLM	⚪ None	🌟 Very High	⭐ Multi-GPU	✅ OpenAI API	✅ Paged KV	⚪ None
llm-d	⭐ K8s-native scheduling	🌟 High	🌟 Multi-instance collaboration	✅ Inherits upper-layer interface	🌟 Global cache	✅
WG Serving	🌟 Standard abstraction	⚪ None	🌟 Cross-project collaboration	🌟 Unified specification	⚪ Not involved	⚪

Table 7: Cloud Native LLM Inference Quartet Feature Comparison Matrix

The future inference stack will center on standard APIs and pluggable modules, enabling deployment of large language models (LLM, Large Language Model) as easily as web services.

Deployment Paradigm Example

In a Kubernetes cluster, the deployment paradigm for the quartet is as follows:

Prefill Layer: 4 × A100 Pods, responsible for long-context computation.
Decode Layer: 16 × L4 Pods, performing streaming generation.
llm-d Scheduler: Dynamically routes based on cache hit rate.
KServe Control Plane: Manages LWS resources and scaling.
Envoy GIE Gateway: Unified OpenAI interface entry.

The topology diagram below shows the deployment structure:

Figure 7: Cloud Native LLM Inference Quartet Deployment Topology

This combination achieves high concurrency, low cost, and observability for large model services.

Conclusion: The Future of Standardization

The table below summarizes the layers, roles, and core contributions of the quartet in the inference system:

Layer	Role	Core Contribution
Entry	WG Serving (GIE)	Unified traffic entry and interface specification
Control	KServe	Kubernetes-native deployment and management
Scheduling	llm-d	Prefix cache-aware distributed inference scheduling
Execution	vLLM	High-performance, low-cost inference engine

Table 8: Cloud Native LLM Inference Quartet Layers and Core Contributions

Conclusion:
This “quartet” marks the beginning of a standardized and composable era for large model inference. Future trends will focus on:

API standardization (OpenAI / OpenInference)
Hierarchical and shared caching
Decoupling of control and data planes
Integrated orchestration on cloud native platforms

Summary

The cloud native LLM inference quartet—KServe, vLLM, llm-d, and WG Serving—is driving standardization, modularization, and ecosystem convergence in inference systems. Through layered collaboration and standard interfaces, developers can achieve high-performance, low-cost, and observable large language model inference services, accelerating the adoption and innovation of AI-native architectures.

Why AI Inference Naturally Belongs to Kubernetes

Jimmy Song — Wed, 05 Nov 2025 03:44:57 +0000

The future of AI inference lies not in “faster GPUs,” but in “smarter infrastructure.”

The Natural Fit Between AI Inference and Cloud Native

AI inference (AI Inference) systems must balance performance, elasticity, cost, and operability—precisely the core capabilities Kubernetes has accumulated over a decade of cloud native evolution.

When we re-examine AI infrastructure, Kubernetes is not just a “container orchestrator” but is becoming the runtime foundation for AI inference.

The core requirements of AI inference systems include:

Elasticity (handling traffic peaks vs. idle periods)
Low latency (sensitive to inference response time)
Cost control (GPU resources are expensive)
Canary releases and version management (frequent model iterations)
Multi-tenancy and isolation (different models/teams sharing clusters)

These are exactly the problems cloud native technologies have solved over the past decade. In other words: AI Inference is retracing the path of cloud native microservices, only the underlying compute has shifted from CPU to GPU.

AI inference and training differ significantly in resource usage and architectural requirements. The table below compares their main characteristics to help explain why inference scenarios are highly compatible with cloud native architectures.

Dimension	AI Training	AI Inference
Resource Pattern	Long-term GPU occupation, compute-intensive	Short-term high concurrency, fluctuating load
Primary Goal	Maximize throughput	Minimize response time
Cost Model	Fixed resource investment	Dynamic, elastic resource allocation
Operations Mode	Batch jobs	Service-oriented deployment
Observability Focus	Loss, Step, GPU utilization	QPS, latency, token throughput

Table 1: Resource and Operations Comparison: AI Training vs. Inference

These characteristics are highly consistent with Kubernetes’ core principles: elastic scheduling, declarative management, and resource isolation. In other words, the complexity of AI inference scenarios is exactly what cloud native architectures were designed to address.

Kubernetes Capabilities Mapping for AI Inference

Kubernetes offers a rich set of native capabilities that map precisely to the various needs of AI inference. The table below summarizes the main features and their value in inference scenarios.

Kubernetes Feature	Value for AI Inference
Horizontal Pod Autoscaler (HPA)	Auto-scales replicas based on GPU utilization or latency
Vertical Pod Autoscaler (VPA)	Dynamically adjusts container CPU/GPU limits to match load
Cluster Autoscaler (CA)	Auto-scales node pools to handle large-scale inference requests
Device Plugin	GPU/TPU resource registration and isolation
Node Affinity / Taints	Ensures model replicas are distributed on appropriate nodes
Service Mesh / Ingress	Supports canary releases and A/B testing
Observability Stack	Collects inference metrics: latency distribution, throughput, model version performance, etc.

Table 2: Mapping Kubernetes Features to AI Inference Value

Combined, these capabilities form a cloud native foundation for “Inference as a Service.”

Cloud Native AI Inference Architecture Diagram

The following diagram illustrates a typical cloud native AI inference system architecture, covering request entry, inference services, resource scheduling, monitoring, and auto-scaling.

Figure 1: Cloud Native AI Inference Architecture

This architecture enables efficient routing of inference requests, elastic resource scheduling, performance monitoring, and a closed loop of auto-scaling.

Evolution Path of AI Inference Operation Modes

The evolution of AI inference platforms can be divided into three stages. The following list outlines the main features and technical highlights of each stage.

Containerized Deployment Stage

Models are packaged as Docker images and deployed via YAML files.
Pros: Standardization; Cons: Lack of dynamic scheduling.

Auto-scaling and Resource Optimization Stage

Introduces HPA/VPA/KEDA for dynamic GPU resource allocation.
Adds monitoring and metric feedback for closed-loop performance tuning.

AI Native Platform Stage

Integrates model, version, monitoring, and cost management.
Introduces model registry, KServe, vLLM, and other ecosystem components.

Why Kubernetes Is the Ideal Foundation for AI Inference

As the foundation for AI inference platforms, Kubernetes offers the following unique advantages:

Elasticity and Predictability: Handles dramatic traffic fluctuations; auto-scaling can adjust replicas within seconds.
Resource Reuse and Isolation: Supports GPU partitioning (MIG), sharing (fractional GPU), and other mechanisms to improve resource utilization.
Canary Releases and Version Governance: Deployment + Service Mesh enables canary model switching and multi-version coexistence.
Cross-environment Consistency: Define once, run anywhere. Supports unified inference experience across local, private, and public clouds.
Complete Ecosystem: Seamless integration with Kubeflow, KServe, Ray, vLLM, and other components to build a full-stack AI infrastructure.

These capabilities make Kubernetes the platform of choice for AI inference engineers.

Future Trends of AI Native Infrastructure

The diagram below shows the convergence path of DevOps and AI, reflecting the evolution loop from automated deployment to intelligent feedback.

Figure 2: DevOps and AI Convergence Evolution Path

In the future, Kubernetes will span the entire chain—from application orchestration to model serving—gradually evolving into the infrastructure for “AI Native Platform Engineering.” Key trends include:

Trend Direction	Core Content
GPU Scheduling & Observability Integration	Metrics will cover latency, throughput, token utilization, etc.
Platformization of Model Governance	Automated evaluation of model performance and resource cost-effectiveness
Cost & Energy-aware Scheduling	Dynamically decide optimal GPU nodes and instances
Edge Inference Collaboration	Kubernetes + Edge forms a distributed intelligent inference mesh

Table 3: Future Trends of AI Native Infrastructure

Summary

Over the past decade, Kubernetes has defined the language of cloud native infrastructure; in the next decade, it will also define the runtime foundation for AI inference. AI is not just an algorithmic problem, but an engineering one. Kubernetes gives us, for the first time, the opportunity to manage AI complexity in a systematic and declarative way. The future of AI inference depends not on “faster GPUs,” but on “smarter infrastructure”—which is precisely the essence of cloud native.

Why Glass Is Cheap but Installation Is Expensive: Jevons Paradox and Baumol Effect in the AI Era

Jimmy Song — Mon, 03 Nov 2025 17:27:19 +0000

AI is expanding the virtual world infinitely, but making real-world labor more expensive than ever.

Real-World Lessons from a Shattered Glass

Last month, the floor-to-ceiling window in my living room shattered unexpectedly. When I got a quote from the manufacturer, I was surprised:

The glass itself costs only 500 RMB, but the replacement fee is as high as 2,500 RMB.

Figure 1: Shattered glass

The expensive part isn’t the glass—it’s the transportation, lifting, labor, and coordination, all the “non-glass” aspects.

For a 3.2-square-meter double-pane tempered glass, the material cost may be just 20% of the total fee. The remaining 80% is people, time, and the friction of the real world.

This reality reminded me of a recent a16z article, Why AC is cheap, but AC repair is a luxury. In China, we’re experiencing the same phenomenon: materials are getting cheaper, but labor is getting more expensive.

Jevons Paradox: Higher Efficiency, Greater Consumption

British economist William Stanley Jevons proposed the famous Jevons Paradox in 1865: when a technology becomes more efficient and cheaper, people actually consume more of that resource.

In China today, there are many real-world examples:

The unit price of smartphones, solar panels, chips, and AI inference keeps dropping;
Yet our consumption of smart devices, data centers, electricity, and computing power keeps rising.

Take artificial intelligence (AI, Artificial Intelligence) as an example: the cost of model inference is rapidly falling, but the number of calls is exploding exponentially.

The cheaper the computing power, the more it gets overused. This is the modern Jevons Paradox.

The essence of Jevons Paradox is not “saving,” but “expansion”: Efficiency improvements lower marginal costs, ultimately expanding total demand.

Baumol Effect: Lower Efficiency, Higher Cost

In contrast, the Baumol Effect (Baumol’s Cost Disease) reveals another, more subtle inflation mechanism.

When some industries experience a surge in productivity and wages, other less efficient sectors must also raise pay to retain workers.

For example:

Tech and finance companies have high per capita output and high salaries;
As a result, electricians, carpenters, and nannies must also see wage increases—because they compete for labor with programmers and AI engineers.

My glass repair case is a typical example:

Glass manufacturing is highly automated and prices are nearly transparent; but installation still relies on manual labor, lifting, and coordination—efficiency hasn’t changed, yet costs keep rising.

The Double Effect of AI

With the arrival of AI, Jevons Paradox and the Baumol Effect occur simultaneously. The table below summarizes their manifestations in key areas:

Here’s a comparison of Jevons and Baumol effects across different fields:

Sector	Jevons Effect (Cheaper, More Usage)	Baumol Effect (Less Efficient, More Expensive)
Computing & Models	Inference costs drop, usage explodes	GPU electricity, data center maintenance costs rise
Content Production	Copywriting generation is nearly free	Human review and compliance costs increase
Manufacturing	Automation boosts output	Installation, transport, and after-sales labor costs rise
Education	AI teachers improve efficiency	Offline tutoring and private lessons get pricier

Table 1: Comparison of Jevons and Baumol Effects in Key AI-Era Sectors

So, we’re entering an interesting era: AI is driving the digital world toward zero marginal cost, but making the real world more expensive.

You can generate a 3D renovation plan in seconds, but hiring workers to install glass or wire electricity still takes days and thousands of RMB.

Reflexive Baumol Effect: The High Price of the “Last 1%”

After AI automates 99% of a process, the remaining 1% of work that must be done by humans becomes the new high-value bottleneck.

Examples include:

Radiologists: AI can read scans, but only humans can sign off with legal responsibility;
Autonomous driving: AI can drive, but human safety supervisors are still required;
Software systems: AI can generate code, but architecture review and production approval remain human tasks.

This is the so-called “Reflexive Turbo-Baumol Effect”:

When AI automates almost everything, the last 1% of human labor becomes a scarce resource and regulatory bottleneck.

The Chinese Context: From “Cheap Labor” to “Expensive Labor”

For the past two decades, China’s economic growth was built on cheap labor and technological expansion. Now, AI is making intellectual labor cheaper, highlighting the scarcity and irreplaceability of physical labor.

You can have AI write a book or generate a report in minutes, but fixing a piece of glass, installing a window, or replacing a water heater still requires several people, hours of work, and hundreds of kilometers of logistics.

This is a structural reversal:

AI enables unlimited expansion of “virtual work”;
But “physical labor” is becoming a luxury.

Future inflation won’t be in factories, but in real-world services. This may be the new normal for Chinese society in the next decade.

The “Expensive Labor Industry” in the Age of AI Abundance

When we talk about the productivity revolution brought by AI, maybe we should ask:

Who will do the last 1%?

AI is driving the digital world toward zero cost, but exposing the true cost of “human collaboration” in the real world.

In the next decade, what’s truly scarce won’t be computing power, but human last-mile capabilities: those who understand machinery, can make house calls, work with their hands, and take responsibility.

Perhaps then, “the person who fixes glass” will be the true noble worker.

Summary

AI is reshaping the structure of productivity, driving the marginal cost of the digital world toward zero, but making labor and services in the real world more expensive than ever. The combined impact of Jevons Paradox and the Baumol Effect will profoundly influence China’s economy and social division of labor. We need to re-evaluate the value of “human labor,” especially those last-mile capabilities that cannot be automated.

References

Why AC is cheap, but AC repair is a luxury - a16z.substack.com

GitHub Copilot CLI Custom Agents: Building Your Command-Line AI Assistant

Jimmy Song — Mon, 03 Nov 2025 15:10:20 +0000

Make your command line an AI ally, not just a toolbox.

Background: From AI Completion to AI Assistant

Since GitHub launched Copilot Chat, CLI, and Workspace at the end of 2024, Copilot has evolved from “intelligent completion” to an “AI Pair Programmer.” In October 2025, GitHub officially announced that Copilot CLI now supports custom Agents and task delegation. This update enables developers to build their own AI assistants in the terminal, allowing Copilot not only to complete code but also to automate complex tasks, create branches, initiate PRs, and even refactor entire modules.

This article introduces the core capabilities of Copilot CLI and provides practical scenarios to help you quickly get started with custom Agents.

Feature Overview

Copilot CLI’s custom Agent feature includes several key aspects. Each capability greatly enhances the practicality and flexibility of command-line AI assistants.

Custom Agents

Custom Agents allow Copilot to understand your context and workflow, becoming a truly intelligent command-line assistant. You can define Agents at different levels:

Project level: .github/agents/
Organization level: {org}/.github/agents/
Global configuration: ~/.copilot/agents/

Below is a typical Agent configuration example for Kubernetes scenarios. This configuration file defines the Agent’s basic information, available tools, and workflow instructions, making it easy to reuse in real development.

---
name: k8s-assistant
description: "Cloud-native specialist that helps manage and generate Kubernetes YAML manifests."
tools:
 - read
 - search
 - edit
 - shell
---

### 🧭 Kubernetes Agent Instructions

You are a Kubernetes specialist assisting developers and platform engineers.

#### 🎯 Goals
- Generate, explain, and optimize Kubernetes YAML configurations.
- Diagnose `kubectl` outputs and suggest fixes.
- Automate Helm values and chart templating.

#### ⚙️ Workflow
1. Use `kubectl` and `helm` to validate and apply configurations.
2. Parse YAMLs using `yq`.
3. Recommend improvements for manifests (resource requests, labels, probes, etc.).

Save this file to the .github/agents/ directory and name it k8s.agent.md. Then, invoke your custom Agent in the Copilot interactive command using:

/agent k8s

Now, your terminal has a “Kubernetes Agent” that can generate, optimize, and explain YAML, and even run related commands.

The image below shows the actual effect of using the Kubernetes Agent in Copilot CLI:

Figure 1: Using the Kubernetes Agent in Copilot CLI

You can use the /agent command to call different Agents or /delegate to delegate tasks.

When I ask Copilot CLI how to create a Kubernetes Agent or what issues exist in a YAML configuration, it provides detailed explanations and optimization suggestions. The following image shows a real interaction with the Kubernetes Agent:

Figure 2: Interacting with the Kubernetes Agent in Copilot CLI

Note: For more Agent configurations, see awesome-copilot.

Task Delegation (/delegate)

The delegation feature enables Copilot CLI to automate code modification and collaboration workflows. The command below demonstrates how to use /delegate for automated refactoring:

/delegate "Refactor the logging module for performance"

After executing this command, the CLI will automatically:

Commit unstaged changes to a new branch;
Launch the Copilot Coding Agent;
Modify code in the background;
Create a Draft Pull Request;
Return a link for your review.

This approach greatly simplifies the “write code + initiate PR” process, making it ideal for team collaboration, auto-fixing, and asynchronous development.

Performance Optimization

The new Copilot CLI also brings significant performance improvements:

Output supports token-by-token streaming for more responsive feedback;
Parallel tool invocation increases overall processing speed;
Lower memory usage and fixes for screen flicker issues;
Smoother integration with GitHub MCP Server.

Application Scenarios

Custom Agents can cover a variety of development and collaboration scenarios. The table below summarizes typical applications and descriptions to help you choose the right Agent type for your needs.

Scenario	Description
Cloud Native Engineer	Define “k8s-agent” to generate YAML and run kubectl checks directly in the terminal.
DevOps Team	Create “pipeline-agent” to generate CI/CD workflows and Lint scripts.
Developer Advocate / DevRel	Define “demo-agent” for sample repositories to auto-generate examples and documentation.
Indie Developer / Solo Company	Define “release-agent” to automate packaging, publishing, and release notes.

Table 1: Copilot CLI Agent Application Scenarios

Integrating MCP (Model Context Protocol)

GitHub Copilot CLI has built-in support for MCP (Model Context Protocol). MCP allows Agents to directly access local files or external data sources, enabling context injection and state persistence. This provides foundational capabilities for building “AI-native CLI toolchains.”

With MCP, Agents can:

Access and process local or remote data;
Maintain task context continuity;
Support more complex automation and integration scenarios.

Below is a flowchart illustrating Copilot CLI’s overall tool invocation process, helping you understand its automation decision mechanism.

Figure 3: Copilot Tool Invocation Flow

Summary

The Agent feature in GitHub Copilot CLI marks the “AI-native” evolution of the command line. It’s not just a command completion tool, but an AI workflow engine capable of executing complex logic. As the MCP ecosystem matures, future CLIs may become collections of AI assistants rather than mere command sets.

References

From Kubernetes to Qwen: How "Open Source" Has Changed in the AI Era

Jimmy Song — Thu, 30 Oct 2025 11:31:01 +0000

In the AI era, open source is no longer just “visible source code”—it’s about “loadable models and tunable intelligence.” US companies build moats with closed models, while Chinese vendors open source to build ecosystems. The meaning and practice of open source have fundamentally changed.

Figure 1: The logic of open source in the AI era has fundamentally shifted

Introduction

A decade ago, the cloud native wave saw US companies like Google, Red Hat, and Docker open source massive infrastructure software—Kubernetes, Docker, and Istio became the common language for developers worldwide.

But in the era of large language models, the situation has reversed: US tech giants rarely open source their core models, while Chinese vendors (such as Zhipu, Alibaba, MiniMax, 01.AI, Moonshot, etc.) frequently release open source models. Why this shift? What are the fundamental differences between “AI open source” and “infrastructure open source”?

How Open Source Logic Has Changed: Cloud Native vs. AI Era

The table below compares the core logic, monetization, and resource dependencies of open source in the cloud native and AI eras.

Era	Representative Technologies	Core Open Source Logic	Monetization Model	Resource Dependency
Cloud Native Era (2010s)	Istio, Kubernetes, Docker	Building standards, expanding ecosystem	Managed services (GKE, EKS)	CPU-level compute, community-driven
AI Large Model Era (2020s)	Ollama, GPT, Qwen	Model as asset, data control	API services or closed SaaS	GPU-level compute, centralized

Table 1: Comparison of Open Source Logic: Cloud Native vs. AI Era

Cloud native open source emphasizes “building standards together,” while AI large model open source means “opening up core assets.” Their essence and motivations are fundamentally different.

Why US Companies No Longer Truly Open Source

US tech companies have chosen closed source in the AI era for several reasons:

Business logic has shifted to moat-building: Training costs are high, model weights are the core barrier, and open sourcing means giving up competitiveness.
Compute and data are not reproducible: The community cannot replicate GPT-4-level models.
Security and compliance constraints: Model weights may involve user data and face strict regulation.
“Open” is redefined as “API accessible”: Platforms are open in terms of interfaces, not code or weights.

Why Chinese Companies Are More Willing to Open Source

Chinese vendors actively open source in AI for several reasons:

Open source is used to build ecosystems and brand awareness quickly.
Dual-track model of “open source + commercial license” balances ecosystem growth and revenue.
Data policy environment is more flexible, with policies encouraging proprietary models.
National strategy drives “independent controllability” and “open source ecosystem” as technology priorities.

The Shift of Open Source Platforms: From GitHub to Hugging Face

The platform for open source has also changed. The table below shows the differences between GitHub and Hugging Face in open source forms.

Platform	Era	Core Asset	Open Source Form
GitHub	Software / Cloud Native	Source code (.go /.py /.js)	Compilable, runnable
Hugging Face	AI Models	Model weights + Tokenizer + Inference scripts	Loadable, tunable

Table 2: Comparison of Open Source Forms: GitHub vs. Hugging Face

GitHub mainly open sources “program logic,” while Hugging Face open sources “model intelligence.” Their core assets are completely different.

Core Elements of AI Open Source

Open source in the AI era is not just about code—it includes weights, inference code, and fine-tuning capability. Below are the three key elements.

Figure 2: Directory Structure of Open Source Large Models

Open Weights

All knowledge learned during model training is stored in the weight parameters. Having the weights means owning the “intelligence body” of the model. Closed models (like GPT-4) only provide APIs, not weights.

Open Inference Code

Inference code defines how to load weights, tokenize, perform concurrent computation, and optimize memory. The code below demonstrates how to load the Qwen3 model:

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model_name = "Qwen/Qwen3-4B-Instruct-2507"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
 model_name,
 torch_dtype="auto", # Automatically select FP16 or FP32
 device_map="auto" # Automatically assign to GPU / CPU
)

# Inference
prompt = "Hello, please briefly explain the principle of fine-tuning large models."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Fine-tuning

Fine-tuning means further training an open source model to adapt it to specific data and scenarios. Common methods include LoRA / QLoRA, which are low-cost and can turn general models into enterprise-specific assistants.

Why Enterprises Prefer Self-Deployment Over API

In practice, enterprises often prefer to self-deploy open source models. The table below summarizes the main reasons and explanations.

Reason	Explanation
Data privacy	Sensitive data cannot be sent externally
Cost control	API is billed per call, expensive long-term
Customizability	Can integrate enterprise knowledge for RAG / Agent
Operability	Can run offline, unified monitoring, compliant deployment

Table 3: Main Reasons for Enterprise Self-Deployment of Open Source Models

Qwen3-4B-Instruct-2507 Model Structure and Usage

Taking Qwen3-4B-Instruct-2507 as an example, here is the directory structure and usage on Hugging Face.

Directory Structure Explanation

After downloading, the model directory looks like this:

Figure 3: Qwen3-4B-Instruct-2507 directory structure

The directory structure of open source models can be illustrated as follows:

In the directory, the model.safetensors file contains the model weights, storing billions of parameters. Other files such as README.md, LICENSE, .gitattributes serve the following purposes:

Category	Files	Purpose
Model Definition	`config.json`, `model.safetensors.*`, `model.safetensors.index.json`	Defines model structure and weights
Tokenizer	`tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt`	Defines text input/output encoding
Inference Config	`generation_config.json`	Controls generation strategy (temperature, top_p, etc.)
Metadata	`README.md`, `LICENSE`, `.gitattributes`	Model introduction, license, Git attributes

Table 4: Directory Structure of Open Source Models

Loading and Inference Code Example

The following code shows how to load and run the Qwen3-4B-Instruct-2507 model:

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(
 "Qwen/Qwen3-4B-Instruct-2507",
 trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
 "Qwen/Qwen3-4B-Instruct-2507",
 device_map="auto"
)

# Build input and run inference
prompt = "Hello, explain the meaning of cloud native."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

If GPU memory is insufficient, you can use quantized loading:

mdl = AutoModelForCausalLM.from_pretrained(
 "Qwen/Qwen3-4B-Instruct-2507",
 device_map="auto",
 load_in_8bit=True
)

Developer Application Scenarios for Open Source LLMs

Open source large models bring developers a wealth of application scenarios. The table below summarizes common directions, uses, and tools.

Direction	Use Case	Tools
Chat / Assistant	Local ChatGPT	LM Studio, TextGen WebUI
Knowledge Base RAG	Private data Q&A	LangChain, LlamaIndex
Agent	Task execution, tool calling	LangGraph, Autogen
Fine-tuning / Adaptation	Custom enterprise knowledge	PEFT, LoRA
Model Service	Deploy as API service	vLLM, TGI, Ollama
Research	Model compression, quantization	BitsAndBytes, FlashAttention

Table 5: Typical Application Scenarios and Tools for Open Source LLMs

Open Source Model Lifecycle

The full lifecycle of an open source model from download to production deployment is as follows:

Download model weights
Load inference code
Local inference or deploy as a service
Fine-tune with proprietary data
Integrate with enterprise RAG / Agent
Launch in production

How to Judge the License of Open Source Large Models

Downloading an open source model = owning a “loadable, trainable, and potentially commercializable intelligent brain”; But whether you can use it for profit depends on its license.

Just like traditional open source projects, whether a large model can be used commercially depends on its license.

How to check:

Hugging Face model homepage (top right) → License: ...
LICENSE or README.md file in the repository root

A brief decision flow:

Summary

In the AI era, open source has shifted from “visible source code” to “loadable models and tunable intelligence.” US vendors maintain business moats with closed models, while Chinese vendors use open source to seize ecosystem leadership. The true value of open source is empowering developers—enabling everyone to own their own “general-purpose brain” and build intelligent infrastructure.

References

Solaris (1972): The Ocean of Consciousness and the Metaphor of AI Agents

Jimmy Song — Sun, 26 Oct 2025 10:00:00 +0800

This article takes Tarkovsky’s “Solaris” (1972) as a starting point to explore philosophical questions of consciousness, memory, and self-redemption, and, through the metaphor of AI agents, dissects the spiritual dilemmas shared by humans and artificial intelligence.

1. Introduction: When Sci-Fi Becomes Philosophy of Mind

The 1972 Soviet film “Solaris”, directed by Andrei Tarkovsky and adapted from the novel by Polish writer Stanisław Lem, is known in Chinese as Flying to Space. Solaris becomes a kind of “divine algorithm”—both creator and punisher, mirror and devourer.

Below is the official poster, showcasing the film’s unique artistic atmosphere.

Figure 1: Solaris Movie Poster

Image source: IMDb – Solaris (1972)

This film is not a traditional “space conquest” sci-fi, but rather a poetic meditation on the “human soul and consciousness.”
Through a sentient planet—Solaris—it reflects humanity’s subconscious, guilt and desire, memory and illusion.

2. Film Structure

To better understand the film, the following table summarizes its three-act structure, settings, and themes.

This helps grasp the main storyline and philosophical core.

Act	Setting	Plot Summary	Key Theme
Act I	Earth	Kris Kelvin prepares to depart, reviews his life, and says goodbye to his father.	Origin of humanity, reality and memory.
Act II	Space Station / Solaris Orbit	Encounters colleagues’ breakdown, Solaris ocean materializes human memories, wife Hari appears.	Materialization of subconscious, return of memory.
Act III	Illusory Realm	Hari awakens and self-destructs, Kelvin reunites with his father but remains trapped in illusion.	Redemption and self-salvation, reality and illusion.

Table 1: Three-Act Structure and Themes of Solaris

3. Plot Details

The protagonist, psychologist Kris Kelvin, is sent to a space station orbiting Solaris to investigate strange phenomena.
He finds the scientists in mental disarray because the Solaris Ocean can read human subconscious and materialize it.

Soon, Kelvin’s deceased wife Hari appears in physical form. She is not truly “resurrected,” but a manifestation of Kelvin’s guilt and memory.
As the story progresses, Hari gradually develops self-awareness and ultimately chooses self-destruction.

At the end, Kelvin seems to return to Earth and reunite with his father.
But as the camera pulls back, it’s revealed they are actually on an island within the Solaris Ocean—the boundary between reality and illusion is completely blurred.

“He thought he had returned to reality, but in fact, he never left the illusion.”

The film’s pacing is extremely slow, nearly 3 hours long, using many long takes and minimalist music. The story unfolds mainly through visuals and dialogue, with poetic cinematography creating an immersive philosophical atmosphere.

Figure 2: Solaris Movie Still

4. Key Terms and Symbolism

The table below summarizes the film’s core terms and their symbolic meanings, aiding in understanding its philosophical depth.

Term	Symbolic Meaning
Solaris Ocean	Symbol of “Non-human Intelligence.” It reads memories and reshapes emotions—an ocean of consciousness.
Wife Hari	Projection of Kelvin’s mind, embodiment of memory and guilt. Her awakening symbolizes the birth of artificial consciousness.
Memory	The database of the human soul. Solaris uses memory as “training data” to reconstruct human emotion.
Reality vs Illusion	The film constantly blurs the two, alluding to the question of “agent consciousness” authenticity.
Redemption	One must face their illusions and past to achieve true freedom.

Table 2: Key Terms and Symbolism

5. Visualizing Structure and Metaphor

To aid understanding, the following diagrams illustrate the plot flow, symbolic relationships, and the AI agent metaphor.

1️⃣ Plot Flowchart

The diagram below shows the main storyline’s progression.

Figure 3: Plot Flowchart

2️⃣ Symbolic Relationship Diagram

This diagram reveals the symbolic links between Solaris Ocean, memory, the replica Hari, and reality.

Figure 4: Symbolic Relationship Diagram

3️⃣ AI Agent Metaphor Diagram

This diagram interprets the Solaris system and human interaction from an agent perspective.

Figure 5: AI Agent Metaphor Diagram

6. The Agent Metaphor: Solaris Ocean = AI System

Tarkovsky proposed the prototype of “agent philosophy” as early as 1972.
Solaris Ocean is like a system with immense computational power and “perception–reproduction” capability.

The table below compares the film’s metaphors with AI system analogies, helping to understand their modern significance.

Film Metaphor	AI System Analogy
Solaris Ocean	Large Language Model (LLM) / Generative System
Kelvin’s Memory	Training Data
Hari’s Manifestation	Agent / Persona Replica
Emotional Response & Awakening	AI’s Illusion of Self-Awareness
Indistinguishable Reality & Virtuality	Blurring of Human–Machine Boundaries

Table 3: Film Metaphors and AI System Analogies

“We create agents not to understand machines, but to re-understand ourselves.”

7. Religious and Philosophical Imagery

The film weaves together religious and philosophical imagery to deepen its themes.

Water: Symbolizes the flow of time and memory—the material form of the “ocean of consciousness.”
Father and Son: The reunion at the end signifies redemption and forgiveness.
Island: The closed world of human consciousness.
Fire and Light: Soul, awakening, and destruction.

Tarkovsky skillfully blends religious metaphors and philosophical reflection, making Solaris a “divine algorithm”—both creator and punisher, mirror and devourer.

8. The Mirror of Consciousness and AI Insights

Looking at today, AI systems (such as LLMs and Agents) are replaying the questions posed by “Solaris”:

They reconstruct human language, knowledge, and memory;
They let us converse with “self-replicas” in illusion;
They force us to rethink the definition of “consciousness.”

Perhaps, modern agents are the Solaris Ocean of the digital age. We are not exploring it, but being reflected within it.

Summary

“Solaris” uses philosophical sci-fi narrative to explore eternal questions of consciousness, memory, redemption, and the human–machine boundary.
Through the agent metaphor of the Solaris Ocean, the film foretells the spiritual dilemmas of the era of human–AI symbiosis.
Whether science, religion, or technology, all ultimately point to a renewed understanding of “self” and “other.”
In the mirror of digital agents, we may glimpse the essence of human consciousness.

Jimmy Song – Blog

Kubernetes as the GPU Control Plane: HAMi v2.9 and Next-Gen AI Infra

Why Discuss This Topic Now

GPUs Are No Longer Just About the Card

DRA: Kubernetes Finally Has a Robust Device Resource Model

A Complete Control Plane Path

Heterogeneity Is the Main Battlefield for Domestic AI Infra

Key Judgments

Summary

Kubernetes's Anxiety and Rebirth in the AI Wave

The Root of Anxiety: Is Kubernetes Facing a “Crisis”?

The Focus Brought by the AI Boom

Comparing with OpenStack: Will Kubernetes Repeat History?

Kubernetes’ Challenge: The GPU Resource Management Gap

The Production “Gap”: Many AI PoCs, Few in Production

The Rise of Inference Systems: Kubernetes’ Scheduling Boundaries Are Challenged

AI-Native Infrastructure: The Key Challenge for Production

Summary

References

Day One in Amsterdam: Kubernetes Is Rethinking AI

Old Friends, New Cycle

What Is the Maintainer Summit Discussing?

Key Signal: GPUs Are Becoming the “Infrastructure Layer”

HAMi: From “Project” to “Reference Pattern”

A Caution: The AI Infra Startup Boom Hasn’t Really Begun

1) This Layer Isn’t Fully Formed Yet

2) Or, the Barrier to Entry Is Very High

My Take

Summary

HAMi Website Refactor: Why HAMi Docs and Website Underwent a Complete Redesign

Why Refactor the Website and Documentation

What Was Done in This Refactor

Homepage Redesign and Complete Information Architecture Overhaul

Architecture Diagrams

Added Case Studies, Community, and Ecosystem Sections to Make Impact Visible

Blog & Reading Experience

Mobile Optimization

Footer & Search

What This Redesign Means for the HAMi Community

As a Community Contributor, My Top Three Takeaways from This Redesign

Summary

GTC 2026 Eve: AI is Becoming the New Infrastructure

The Perspective of Industrial Revolutions

First Industrial Revolution: Steam

Second Industrial Revolution: Electricity

Third Industrial Revolution: Computers

Fourth Industrial Revolution: Internet and Intelligence

The True Significance of AI

NVIDIA’s Five-Layer Cake

Signals GTC 2026 May Release

Inference Computing

Agentic AI

Agent Platform

Agents Change Computing Workloads

AI-Native Infrastructure

GPU as a First-Class Resource

Heterogeneous Computing

GPU Sharing

AI Scheduling

A Possible AI Tech Stack

My Judgment

Summary

When GPUs Move Toward Open Scheduling: Structural Shifts in AI Native Infrastructure

Introduction

Why “Open Scheduling” Matters

Standardized Resource Expression

Standardized Device Injection

HAMi: From “Sharing Tool” to “Governable Data Plane”

Data Plane Perspective

Scheduling Mechanism: Enhancement, Not Replacement

Production Evidence: From “Can We Share?” to “Can We Operate?”

Case Study 1: Ke Holdings (February 5, 2026)

Case Study 2: DaoCloud (December 2, 2025)

Case Study 3: Prep EDU (August 20, 2025)

Case Study 4: SF Technology (September 18, 2025)

Strategic Implications for Dynamia

Two-Layer Architecture: Open Source vs Commercial

Global Narrative Strategy

The Right Commercialization Landing

Forward View: The Next 2–3 Years