Envoy configuration in a nutshell: Listeners, Clusters and More

In the previous blog post, I briefly discussed what Envoy Proxy is and where it can be used. If you’re not familiar with Envoy I strongly suggest reading the previous piece first. This text is meant for developers or devops engineers who want to learn more about how to make the most of its functionality. We will discuss how Envoy Proxy actually works and how it should be configured.

Understanding Envoy Basics

Let’s start with a simple example.

Fig. 1 Envoy configuration diagram

This demonstrates the most common situation when the client initiates a connection with Envoy Proxy as it tries to reach the server.

Envoy receives a connection request from the client (downstream) and then opens a new connection with the server (upstream). Let’s take a look at Envoy in action.

Fig. 2 Envoy logic

Below, there are described three elements that handle the majority of Envoy logic.

Listener

As the name suggests, this element listens for connections.

The connections are being sent from DOWNSTREAM. The listener binds to a port and awaits the connection. When the request appears, the listener analyzes the connection and decides which filter chain fits the conditions. The most simple case means that each request will be sent through the same set of filters.

Filter

This is Envoy’s main operational unit.

Configuring Envoy is mostly a case of choosing the right filters. Each fitler is usually tied up with a single functionality that may have an impact on the request/data. To name a few of them:

• Echo filter - a simple filter designed mostly for testing purposes. It sends back the received bytes.
• TCP proxy filter - a filter which allows us to specify the target cluster for the traffic.
• HTTP connection manager filter - a special filter which allows us to specify a set of subfilters designed specifically for working with HTTP requests/responses - this is the most crucial filter for proxying REST API applications.
• Custom filter - as mentioned at the beginning, you can easily write your own logic, writing your own filters and applying them in the filter pipeline just like others. Custom filters will be described in detail in the next blog post.

Usually the last filter in the filter chain is a proxy filter or HTTP manager filter (as it contains routing options). An important thing to remember is that the response received from UPSTREAM will be processed in reverse order. Filters can be configured to work in only one direction.

Cluster

An abstraction for UPSTREAM. This describes the target location for the client’s connections. If the location consists of multiple addresses it also defines the load balancing algorithm, which will be used by Envoy to resolve where exactly the data should be sent. Apart from the location it also defines what IP version should be used, the timeout, etc.

Real use cases are usually not as straightforward as those shown in the figure above. Let’s take a look at a more complex scenario:

Fig. 3 Envoy configuration detailed diagram

Now, this is something! To simplify the figure above we skipped a few details and didn’t draw responses. Let’s agree that all four servers replied to TCP/HTTP traffic and the response data went back via the same path as it came in. Now let’s talk about what this figure actually tells us. As you can see we have here:

• two listeners (the first one binds itself to three different socket addresses and the second one binds to single address),
• three clusters (two of them point to a single server but the first one sends the traffic to one of two servers randomly),
• and plenty of filters.

So what is going on exactly here?

• The first listener sends requests through two different filter chains. Depending on the request criteria, it sends it via three network filters or two network filters, where the last one is an HTTP filter containing two more sub filters.
• The second listener sends the request to the filter, which gathers statistics about the request and then simply drops it.
• The first filter chain proxies the TCP traffic to the cluster, which chooses one of two servers. The load balancing algorithm is a round robin so the green request goes to the first server and following blue request goes to the second server.
• The second filter chain uses an HTTP filter to decide which cluster is the target. It allows us to choose the cluster by inspecting the HTTP request endpoint.

This gives us some understanding of Envoy and its core concepts. A few additional things we can cover in the situation above:

• Listener configuration allows us to handle the TLS protocol - by providing certificates we can ensure that the connection between DOWNSTREAM and the listener will be considered secure (please note that specifying the TLS doesn’t mean the connection is 100% secure - specifying allowed cipher suites, generating correct certificates, etc. is your responsibility so don’t lose your focus here).
• Cluster configuration can be extended by providing health check options. This is a place where we can define our rules for inspecting whether the server is reliable or not.

This is a good moment to learn how to write an Envoy configuration. Remember, unless you need complex functionality, after the next section you can download an Envoy binary and cover your requirements with your own configuration file.

Configuring Listeners & Clusters

The core

One of the most crucial features of Envoy is the option for dynamic configuration of the data plane. We don’t need to restart the Envoy process, risking that our application won’t be working for a while. However, before we dive in, let’s start with a simpler configuration - using a static configuration file.

To start the configuration, we create the following YAML file:

static_resources:
    listeners:
      [LISTENER CONFIGURATION]
    clusters:
      [CLUSTER CONFIGURATION]

This is our starting point. We need to define our listeners and clusters. Both of these fields expect lists. 

Clusters

Let’s start with the example cluster:

 clusters:
  - name: books-server
    connect_timeout: 8s
    type: STATIC
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: service
      endpoints:
      - lb_endpoints:
          - endpoint:
              address:
                socket_address:
                    address: 127.0.0.1
                    port_value: 10100
          - endpoint:
              address:
                socket_address:
                    address: 127.0.0.1
                    port_value: 10101

This is an example for a single backend application which runs locally on two different ports.

The configuration consists of:

• name - this is our identification for the following target and its configuration,

• connect_timeout - allows us to specify how long the upstream connection should be maintained before we cut it off,

• type - defines how the cluster information should be obtained/parsed - the static option means that we simply provide a list of IP addresses and ports, no DNS is used, no service discovery, etc.,

• lb_policy - the load balancing policy - here we specify ROUND_ROBIN, which means continuous iteration over our list of endpoints. The next request is sent toward the next endpoint. It is worth mentioning that Envoy keeps its own session of UDP clients sent by the same client, so all UDP packets are forwarded to the same upstream destination to simulate TCP connection,

• load_assignment - the field which defines our possible targets:

  • cluster_name - required but not used in the following configuration, this is used for discovery purposes,
  • endpoints - a list of locations where the traffic can be propagated. As we don’t use locality information here, we simply specify one endpoint which will contain an actual list of endpoints called lb_endpoints - a list of actual addresses.

This is a fairly simple configuration but it already brings us two functionalities - a dummy load balancing and a timeout. We could easily extend this configuration to add more endpoints and specify weights to these endpoints to focus more connections on a stronger machine and we could specify health-checking instructions to detect when the endpoint is down.

The entire cluster configuration schema can be found here.

Listeners

Having covered our targets, let’s now dig into a more complex part of this configuration - listeners section.

 listeners:
  - name: envoy_listener
    address:
      socket_address:
        address: 127.0.0.1
        port_value: 10000
    filter_chains:
      ...

Okay, it doesn’t seem that complex. We have specified here one listener called envoy_listener, which will listen to a certain address. The actual complex part is the filter chains, defining the filters that will be used in order to work with the traffic that was received by this listener.

filter_chains:
- filter_chain_match:
    application_protocols:
    - http/1.1
  filters:
    [LIST OF FILTERS]
- filters:
    [OTHER LIST OF FILTERS]

As the plural form suggests, there is a list of filter chains. As I mentioned earlier, each listener can have multiple filter chains. When the traffic begins, the listener verifies the criteria for each filter chain and decides which one matches the given traffic. The listener chooses the first match which applies, so the most common matches should be described at the bottom of the list.

But a simple filter chain configuration can consist of a single filter chain without any conditions. We mainly focus on providing certain filters and that’s what we will discuss now. The filters field is a list of Network Filters.

filter_chains:
  - filters:
      - name: envoy.filters.network.tcp_proxy
        typed_config:
          "@type": type.googleapis.com/envoy.[...].v3.TcpProxy
          stat_prefix: destination
          cluster: books-server

This is the simplest scenario. We have a single filter called tcp_proxy which will propagate the traffic to the cluster called books-server. This example will help us understand what is going on here.

We configure our filters by providing the typed_config field. This is a configuration that accepts