Advanced Request and Response Processing in Microservice Architecture with Envoy's External Processing Filter

In a microservice architecture, API gateways often need to perform high-level request and response processing tasks such as authentication, data transformation, and security checks. Envoy’s ext_proc external processing filter is a powerful tool that enables flexible request and response handling by interacting with a gRPC service. This article delves into its features, configuration, and performance optimization strategies to help developers and DevOps engineers effectively utilize this capability.

Relationship Between `ext_proc` and Other Filters

The ext_proc filter shares similarities with other gRPC-based filters in Envoy, such as ext_authz . However, ext_proc offers more extensive capabilities, supporting full request and response processing. This makes it particularly suitable for scenarios requiring deep content inspection and modification.

Refer to the following mind map for a quick overview of Envoy’s ext_proc external processing filter:

This mind map illustrates the core structure and functional modules of Envoy’s ext_proc filter. Using a bidirectional gRPC stream protocol, ext_proc communicates with external services, enabling flexible HTTP request and response handling, including synchronous and asynchronous processing.

How `ext_proc` Works and Its Configuration

Definition and Functionality

ext_proc is an Envoy HTTP filter that delegates request and response processing to a gRPC service, allowing complex logic implementation in external services to meet specific business requirements.

Working Mechanism

ext_proc uses a bidirectional gRPC stream to communicate with an external service, enabling real-time request and response processing interaction. Envoy can offload complex tasks such as authentication, data transformation, and custom header manipulation to an external service, improving flexibility and scalability.

Envoy sends ProcessingRequest messages, and the external service returns ProcessingResponse messages. Importantly, this gRPC stream is created per HTTP request stream; it is not shared among multiple requests. Each HTTP request handled by Envoy creates its own dedicated gRPC stream, ensuring isolation and precise request-response management. This design allows the external service to intervene at various stages of the request and response lifecycle, even generating entirely new responses.

Here’s a sequence diagram illustrating the process:

Key Features

Request and Response Handling: Read and modify HTTP request and response headers, bodies, and trailers.
Flexibility: Define custom logic based on business needs, extending built-in functionality.
Asynchronous Processing: Support for asynchronous processing to prevent request blocking.

Envoy Configuration Example

Below is a basic Envoy configuration example:

⚙️ Envoy Configuration Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 8080
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          access_log:
          - name: envoy.access_loggers.stdout
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
              log_format:
                text_format: "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% \"%RESP(X-EXTPROC-HELLO)%\" \"%RESP(CONTENT-TYPE)%\" \"%RESP(CONTENT-LENGTH)%\" %DURATION% ms\n"
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: [ "*" ]
              routes:
              - match:
                  prefix: "/"
                route:
                  host_rewrite_literal: www.envoyproxy.io
                  cluster: service_envoyproxy_io
          http_filters:
          - name: envoy.filters.http.ext_proc
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
              grpc_service:
                envoy_grpc:
                  cluster_name: ext_proc_cluster
              failure_mode_allow: true
              processing_mode:
                request_header_mode: SKIP
                response_header_mode: SEND
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
  - name: ext_proc_cluster
    connect_timeout: 0.25s
    type: LOGICAL_DNS
    lb_policy: ROUND_ROBIN
    http2_protocol_options: {}
    load_assignment:
      cluster_name: ext_proc_cluster
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.1
                port_value: 9000
  - name: service_envoyproxy_io
    type: LOGICAL_DNS
    dns_lookup_family: V4_ONLY
    load_assignment:
      cluster_name: service_envoyproxy_io
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: www.envoyproxy.io
                port_value: 443
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        sni: www.envoyproxy.io

To understand the connection between Envoy configuration and a gRPC service, consider how the following configuration items influence traffic processing:

grpc_service: Defines the target address and cluster name for communication with the gRPC service, corresponding to ext_proc_cluster in Envoy’s configuration.
processing_mode: Controls when to trigger processing stages such as request headers, request bodies, and response headers, determining when to invoke the gRPC service.
failure_mode_allow: Specifies whether to continue request processing when the gRPC service fails, ensuring high availability in partial failure scenarios.
listeners: Defines the network address and port where Envoy listens for incoming requests.
filter_chains: Configures the request processing chain, including the HTTP connection manager and external processing filters.
http_filters: Lists the enabled filters, such as ext_proc and router.
clusters: Defines upstream services and the location of the external processing gRPC service.

You can see the configuration instructions in the Envoy documentation .

Envoy uses these configuration options to offload request and response processing to the gRPC service. The results are returned through a bidirectional streaming protocol, influencing request-forwarding behavior.

gRPC Service Example

Below is a simple gRPC external processing server implementation demonstrating how to add custom response headers using ext_proc. This example highlights key method choices and design decisions, such as using the Process method to continuously receive requests and send responses, ensuring seamless processing. It also leverages the HeaderMutation configuration to modify HTTP response headers, illustrating the tight integration between gRPC messages and Envoy’s configuration for dynamic extension and flexible management.

🐹 gRPC External Processing Server Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
package main

import (
	"log"
	"net"

	"google.golang.org/grpc"
	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/status"

	configPb "github.com/envoyproxy/go-control-plane/envoy/config/core/v3"
	extProcPb "github.com/envoyproxy/go-control-plane/envoy/service/ext_proc/v3"
)

type extProcServer struct {
	extProcPb.UnimplementedExternalProcessorServer
}

// Process handles external processing requests from Envoy.
// It listens for incoming requests, modifies response headers,
// and sends the updated response back to Envoy.
//
// When a request with response headers is received, it adds a custom header
// "x-extproc-hello" with the value "Hello from ext_proc" and returns the modified headers.
//
// Note: The `RawValue` field is used instead of `Value` because it supports
// setting the header value as a byte slice, allowing precise handling of binary data.
//
// This function is called once per HTTP request to process gRPC messages from Envoy.
// It exits when an error occurs while receiving or sending messages.

func (s *extProcServer) Process(
	srv extProcPb.ExternalProcessor_ProcessServer,
) error {
	for {
		req, err := srv.Recv()
		if err != nil {
			return status.Errorf(codes.Unknown, "error receiving request: %v", err)
		}

		log.Printf("Received request: %+v\n", req)

		// Prepare the response to be returned to Envoy.
		resp := &extProcPb.ProcessingResponse{}

		// Only process response headers, not request headers.
		if respHeaders := req.GetResponseHeaders(); respHeaders != nil {
			log.Println("Processing Response Headers...")

			resp = &extProcPb.ProcessingResponse{
				Response: &extProcPb.ProcessingResponse_ResponseHeaders{
					ResponseHeaders: &extProcPb.HeadersResponse{
						Response: &extProcPb.CommonResponse{
							HeaderMutation: &extProcPb.HeaderMutation{
								SetHeaders: []*configPb.HeaderValueOption{
									{
										Header: &configPb.HeaderValue{
											Key:      "x-extproc-hello",
											RawValue: []byte("Hello from ext_proc"),
										},
									},
								},
							},
						},
					},
				},
			}
			log.Printf("Sending response: %+v\n", resp)
			// Send the response back to Envoy.
			if err := srv.Send(resp); err != nil {
				return status.Errorf(codes.Unknown, "error sending response: %v", err)
			}
		} else {
			// If it is not a callback in the response header stage, do not make any modifications and continue processing the next event.
			// For request_headers or other events, do not modify & ensure that Envoy will not be stuck.
			// An empty processing can be returned for request_headers, or it can be skipped in envoy.yaml.
			// Here, simply continue to wait for the next event.
			continue
		}
	}
}

func main() {
	lis, err := net.Listen("tcp", ":9000")
	if err != nil {
		log.Fatalf("Failed to listen: %v", err)
	}

	grpcServer := grpc.NewServer()
	// Register the ExternalProcessorServer with the gRPC server.
	extProcPb.RegisterExternalProcessorServer(grpcServer, &extProcServer{})

	log.Println("Starting gRPC server on :9000...")
	if err := grpcServer.Serve(lis); err != nil {
		log.Fatalf("Failed to serve: %v", err)
	}
}

Expected Result: The request returns a status code of 200, with a custom header x-extproc-hello: Hello from ext_proc in the response. If the header is missing, check the following:

Is the gRPC service running? Ensure the gRPC server has started and is listening on port :9000.
Is the Envoy configuration correct? Verify the Envoy configuration file to ensure the ext_proc filter is enabled and that the ext_proc_cluster configuration is correct.
Logs and Error Investigation: Check the logs of both Envoy and the gRPC server for potential errors.

Running Envoy and the gRPC service locally, you can test with:

envoy -c envoy.yaml
go run main.go
curl -v http://localhost:8080

You’ll see a response including the custom header x-extproc-hello: Hello from ext_proc.

Metrics and Monitoring

ext_proc generates statistics under the http.<stat_prefix>.ext_proc. namespace, where stat_prefix is the HTTP connection manager prefix. Common metrics include:

Metric Name	Type	Description
streams_started	Counter	Number of started gRPC streams
streams_msgs_sent	Counter	Number of messages sent
streams_msgs_received	Counter	Number of messages received
spurious_msgs_received	Counter	Number of unexpected messages received
streams_closed	Counter	Number of successfully closed streams
streams_failed	Counter	Number of streams with gRPC errors
failure_mode_allowed	Counter	Number of ignored failures (per config)
message_timeouts	Counter	Messages timed out without response
rejected_header_mutations	Counter	Rejected header modifications
clear_route_cache_ignored	Counter	Ignored clear route cache requests
clear_route_cache_disabled	Counter	Disabled clear route cache requests

For more details, refer to the Envoy documentation .

Use Cases and Optimization Strategies

Common Use Cases

Authentication: External authentication services check user credentials.
Data Auditing: Log request and response data for compliance purposes.
Traffic Management: Adjust traffic routing based on dynamic analysis.

Configuration Tips and Optimization Strategies

Failure Recovery and Load Balancing: Deploy multiple instances of the gRPC service with load balancing.
Message Timeouts and Retries:

http_filters:
  - name: envoy.filters.http.ext_proc
    config:
      grpc_service:
        envoy_grpc:
          cluster_name: ext_proc_server
      processing_mode:
        request_header_mode: SEND
        response_header_mode: SEND
      message_timeout: 500ms
      max_message_timeout: 1000ms
      failure_mode_allow: false

Metadata Options and Security Policies:

metadata_options:
  forwarding_namespaces:
    untyped: ["custom_namespace"]

Conclusion

Envoy’s ext_proc filter provides robust support for service governance, data transformation, and request inspection through flexible request and response processing capabilities. Correctly configuring and optimizing ext_proc can significantly enhance system flexibility and scalability to meet diverse business requirements.

Advanced Request and Response Processing in Microservice Architecture with Envoy's External Processing Filter

Relationship Between ext_proc and Other Filters

How ext_proc Works and Its Configuration