How the Envoy Proxy Handles User Requests for Tracing

This is an original article by Jimmy Song. You may repost it, but please credit this source: https://jimmysong.io/en/blog/envoy-tracing/

Click to show the outline

In a cloud-native environment, improving application observability to better understand user experience is crucial. However, relying solely on metrics and logs does not provide specific details for individual cases. This is where tracing technology becomes essential.

Basic Principles of Tracing

Tracing provides developers with a complete context of the user experience by attaching an associated ID to each user request. This associated ID acts like a thread that connects traces across multiple services, enabling comprehensive observability.

The diagram below illustrates the flow of user requests processed by Envoy.

User Request and Envoy Proxy Processing Flow Diagram

Tracing enables developers to have complete context of the user experience by appending an associated ID to each user request. This associated ID connects traces across multiple services.

Although all requests pass through the Envoy proxy, it cannot independently provide complete tracing information. Envoy only sees the application as part of the network and lacks insights into internal processes. This limits Envoy’s ability to distinguish whether incoming and outgoing requests originate from the same user, thus it cannot automatically forward tracing context.

Request Context in the Service Mesh

Envoy can act as a Sidecar or Waypoint proxy in the Istio service mesh. The following diagram illustrates how Envoy handles request context within the service mesh.

1. The Start of the User Request

Tracing involves following the path across multiple services to understand the complete context of the user experience. Tracing begins with a user request assigned an associated ID.

2. Request Passing Through Envoy Proxy

Envoy sits beside the application, and all incoming requests go through Envoy.

3. Envoy Adds Extra Headers

Envoy can add extra headers to the request to gather information about what happens internally within the application.

4. Application Processes Request and Calls Backend Services

During request processing, the application may need to contact other systems, such as external authentication and authorization services.

5. Application Needs to Copy the Associated ID

The application knows which outbound request corresponds to which inbound request (e.g., the request with Trace ID 1234). However, Envoy does not know this. Thus, the application must copy the associated ID and context from the inbound request to the outbound request.

6. Concurrent Handling of Multiple Requests

In real scenarios, applications handle multiple user requests simultaneously, leading to concurrency. Since Envoy only sees network-level requests and responses, it cannot distinguish the causal relationships between these requests.

7. Limitations of Envoy

Since Envoy cannot see the internal processing logic of the application, it only observes a series of network requests and responses without knowing which outbound requests were triggered by which inbound requests.

The Role of the Application

Since Envoy cannot automatically forward tracing context, the application itself needs to be responsible for copying the headers from inbound requests to outbound requests to maintain the integrity of the tracing information.

Application Copies Headers

While processing inbound requests, the application needs to copy necessary headers (like associated ID and user identity) to any outbound requests.

Response Returns to User

Once the application completes processing the user request, it returns the response to the user.

Solutions and Recommendations

To ensure the integrity of tracing information, applications need to actively copy and transmit tracing-related headers. This can be achieved by integrating tools like Apache SkyWalking, which not only supports distributed tracing but also includes performance monitoring and log analysis features. By utilizing SkyWalking’s libraries and agents, the process of copying headers and transmitting tracing information can be simplified.

For details on using SkyWalking for distributed tracing in Istio, refer to this blog post.

Conclusion

Importance of Tracing: Tracing provides developers with complete context of user requests, aiding in better understanding and improving user experience.
Role of Envoy: Envoy serves as a proxy for network requests but cannot provide complete tracing information without assistance from the application.
Application Responsibility: Applications must actively copy and transmit tracing-related headers to maintain context.
Utilizing Tools: Leveraging distributed tracing tools like Apache SkyWalking can enhance observability and streamline the header copying process.