Using the Dynamic Modules extensions in Production

Built On Envoy already provides the extensions compiled as Envoy Dynamic Modules for linux/amd64 and linux/arm64. Extensions are packaged as OCI artifacts in the Built On Envoy registry.

Preparing the extensions to be used in Production

There are two main approaches to use the precompiled Dynamic Modules:

1. Download the extensions manually and make them available to Envoy. For example, in Kubernetes, extensions can be stored in a volume Envoy can mount to access the extension files.
2. Create a custom Envoy Docker image with the extensions.

Option 1: Download extensions manually

Extensions can be easily downloaded using the bow download command:

boe download ip-restriction --platform linux/amd64
boe download composer --platform linux/amd64

Once the extensions are downloaded, you need to make the .so files available to Envoy. This could be a shared Kubernetes volume, a custom Envoy image, or any approach that is convenient.

Option 2: Build a custom Envoy image

Building a custom image of Envoy is straightforward, as extensions are packaged in OCI artifacts that can be used directly in the Dockerfile. The following example creates a custom Envoy image with all the Go extensions and the Rust ip-restriction extension:

# Reference all extensions that will be added to the image
FROM ghcr.io/tetratelabs/built-on-envoy/composer:0.5.0-dev AS all_go_extensions
FROM ghcr.io/tetratelabs/built-on-envoy/extension-ip-restriction:0.2.0 AS ip_restriction

FROM envoyproxy/envoy:v1.38.0

# Copy the extension files
COPY --from=all_go_extensions /libcomposer.so /var/boe/extensions/
COPY --from=ip_restriction /libip-restriction.so /var/boe/extensions/

# Setup the env vars Envoy needs to find the extensions and safely run the Go Dynamic Modules
ENV ENVOY_DYNAMIC_MODULES_SEARCH_PATH=/var/boe/extensions
ENV GODEBUG=cgocheck=0

Configuring the Dynamic Modules

To configure the Dynamic Modules you can follow the Envoy documentation. You basically need to inject the Dynamic Module filter in your HTTP filter chain.

Rust Dynamic Module configuration example:

- name: ip-restriction
  typed_config:
    '@type': type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter
    dynamic_module_config:
      name: ip-restriction
    filter_name: ip-restriction
    filter_config:
      '@type': type.googleapis.com/google.protobuf.StringValue
      value: '{"deny_addresses": ["127.0.0.1"]}'

Go Dynamic Module configuration example:

- name: chat-completions-decoder
  typed_config:
    '@type': type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter
    dynamic_module_config:
      name: composer
    filter_name: chat-completions-decoder
    filter_config:
      '@type': type.googleapis.com/google.protobuf.StringValue
      value: '{"metadata_namespace":"llm-request"}'

Dynamic Modules and Go

Running Go-based Dynamic Modules in Envoy has some limitations. Envoy only supports loading one Go runtime in-process, and that means that using Go-based Dynamic Modules needs some careful considerations.

Built On Envoy makes it easier to overcome this limitation while providing maximum flexibility by providing:

• The composer Dynamic Module that bundles all Go extensions. This can be safely loaded in Envoy to leverage any of the provided Go extensions.
• A goplugin-loader special Dynamic Module that can load extensions compiled as Go Plugins.

Using the composer Dynamic Module (recommended)

Built On Envoy provides the composer Dynamic Module that already embeds all Go extensions. By pulling the composer Dynamic Module and loading it into Envoy, you can use any Go extension in the Built On Envoy extension catalog. At the time of writing, the composer Dynamic Module is ~60MB.

The configuration to use the composer Dynamic Module is the one showed in the previous section:

- name: chat-completions-decoder
  typed_config:
    '@type': type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter
    dynamic_module_config:
      name: composer
    filter_name: chat-completions-decoder
    filter_config:
      '@type': type.googleapis.com/google.protobuf.StringValue
      value: '{"metadata_namespace":"llm-request"}'

Dynamically loading Go Plugin extensions (alpha, not recommended for prod)

Apart from the composer Dynamic Module that bundles all extensions, Built On Envoy also provides precompiled packages for each individual Go extension compiled as a Go Plugin. These extensions can be dynamically loaded by the special goplugin-loader Dynamic Module.

To use this approach, you can pull the composer-lite extension, which is a Dynamic Module that only contains the plugin loading functionality. Using the composer-lite. At the time of writing, the composer Dynamic Module is ~19MB.

Option 1: Download extensions manually

Extensions can be easily downloaded using the bow download command:

boe download composer-lite --platform linux/amd64             # Download the `lite` version
boe download chat-completions-decoder --platform linux/amd64  # Download the Go Plugin artifact

Option 2: Build a custom Envoy image

The following example creates a custom Envoy image with all the Go extensions and the chat-completions-decoder extension:

# Reference all extensions that will be added to the image
FROM ghcr.io/tetratelabs/built-on-envoy/composer-lite:0.5.0-dev AS go_loader_dym
FROM ghcr.io/tetratelabs/built-on-envoy/extension-chat-completions-decoder:0.5.0-dev AS chat-completions-decoder

FROM envoyproxy/envoy:v1.38.0

# Copy the extension files
COPY --from=go_loader_dym /libcomposer.so /var/boe/dym/
COPY --from=chat-completions-decoder /plugin.so /var/boe/goplugins/

# Setup the env vars Envoy needs to find the extensions and safely run the Go Dynamic Modules
ENV ENVOY_DYNAMIC_MODULES_SEARCH_PATH=/var/boe/dym/
ENV GODEBUG=cgocheck=0

Configuring the goplugin-loader Dynamic Module

- name: chat-completions-decoder
  typed_config:
    '@type': type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter
    dynamic_module_config:
      load_globally: true            # It is recommended to load all symbols globally when using the `goplugin-loader`
      name: composer                 # This is the `composer-lite`
      filter_name: goplugin-loader   # We use the `goplugin-loader` to Dynamically load the Go plugin
    filter_config:
      '@type': type.googleapis.com/google.protobuf.StringValue
      value: '{"name":"chat-completions-decoder","strict_check":false,"url":"file:///var/boe/goplugins/chat-completions-decoder.so","config":{ ... }}'

When dynamically loading extensions compiled as Go plugins:

• Use the goplugin-loader as the name of the filter.
• The configuration consists of:
  • name: Name of the extension to load.
  • strict_check: Whether to check for dependency incompatibilities and fail early.
  • url: URL (or path) to the Go plugin file to load.
  • config: Configuration that will be passed “as-is” to the plugin.

Appendix: Building your own Dynamic Module with a subset of extensions

If you don’t plan to use all Go extensions but don’t want to use the alpha dynamic loading approach, you can build your own Dynamic Module with your desired set of extensions. All Built On Envoy extensions can be go-imported, so it is trivial to create your own extension bundles. The following code shows an example to create a custom Go Dynamic module with two extensions:

package main

import (
	// Importthe Dynamic module SDK
	_ "github.com/envoyproxy/envoy/source/extensions/dynamic_modules/sdk/go/abi"

	// Import the desired extensions
	_ "github.com/tetratelabs/built-on-envoy/extensions/composer/opa/embedded"
	_ "github.com/tetratelabs/built-on-envoy/extensions/composer/saml/embedded"
)

func main() {} // main is required to build as a C shared library.

If you have your own custom Go extensions (created with boe create, for example), you could import them here as well. The Dynamic Module can be compiled as follows:

CGO_ENABLED=1 go build -buildmode=c-shared -o libmyboe.so main.go

This will produce a Go Dynamic Module with all the configured extensions that can be safely loaded into Envoy without having to worry about the Envoy Go runtime limitations. Then you can use it the same way, but using myboe as the name of the Dynamic Module:

- name: my-boe-extensions
  typed_config:
    '@type': type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter
    dynamic_module_config:
      name: myboe
    filter_name: opa
    filter_config:
      '@type': type.googleapis.com/google.protobuf.StringValue
      value: '{ ... }'