Agent configuration reference

The default agent configuration can be found in the /etc/redirectionio/agent.yml file.

Minimal configuration

After installing the agent, you must edit this file in order to insert your redirection.io instance name. Here is the minimal configuration:

# name of this instance, mandatory for logging
instance_name: "A UNIQUE NAME"

Configuration reference

The agent however supports some more option keys, detailed in the sections below.

instance_name

• Type: string
• Default: -
• This directive is required

The instance_name configuration directive allows to give a unique name to this agent instance. You must use a unique name for this setting across all the agents running for your project. Else, the agent will appear as "misconfigured" on the project dashboard (its name will be the hostname of the server running the agent), and its logs will be ignored.

A good idea can be to use a combination of the hostname and deployment environment as the instance_name. Also, remember that you can use environment variables anywhere in the configuration file.

listen

• Type: string
• Default: 127.0.0.1:10301
• This directive is optional

The listen configuration directive determines on which interface the agent is listening. It defaults to 127.0.0.1:10301 (port 10301 on the localhost), but can take other values:

• /var/run/redirectionio.sock: if you prefer to use a file, please make sure that user running the agent (redirectionio, by default) has the right to create this file
• "": an empty string, if you want your agent to listen on all network interfaces, on port 10301 (not advised)

persist

• Type: boolean
• Default: true
• This directive is optional

The persist configuration directive allows to define whether or not to store the rules on the disk. This allows to restart the agent with existing rules without the need to call redirection.io's backend, and adds of course more resiliency to your setup.

datadir

• Type: string
• Default: /var/lib/redirectionio
• This directive is optional

If the persist directive is enabled, datadir defines the directory where the rules will be persisted on the local filesystem. Of course, the user running redirectionio-agent must have a write permission on this directory.

cache

• Type: integer
• Default: 10000
• This directive is optional

The cache configuration directive defines how many compiled versions of the rules will be stored in memory (RAM). This setting makes the agent much faster, but can consume more memory on very large rulesets if it is set high.

The default value is 10000, which should use about 150-200MB of memory for a very large ruleset. It is not required to set this value much more high, as the agent will cache in priority the hot paths.

You can completely disable the rules memory cache, in order to minimize memory usage, by setting cache: 0

logging

• Type: boolean
• Default: undefined
• This directive is optional

The logging setting allows to define the logging behavior for this instance. If it is set to "true", the instance will be marked as allowed to log traffic. If it is set to "false", the instance will not collect traffic logs. If this setting is not defined, or defined with any other value than "true" or "false", it will be managed by the value defined in the manager graphical interface. Else, this setting takes precedence over the logging behavior defined in the manager graphical interface.

test_mode

• Type: boolean
• Default: undefined
• This directive is optional

The test_mode setting allows to set this instance in "test" mode. If this setting is not defined, or defined with any other value than "true" or "false", it will be managed by the value defined in the manager graphical interface. Else, this setting takes precedence over the logging behavior defined in the manager graphical interface. If it is enabled, the value of the "logging" setting will be forced to "false".

project_keys

• Type: list of strings
• Default: []
• This directive is optional

The project_keys setting allows to preload the agent with some projects rulesets at startup time. If this directive is not set, the agent will start with an empty ruleset, and it will only load the rules for your project(s) when the first request arrives.

In order to avoid the first requests to fail (because the ruleset would not be loaded in time), you can use this configuration directive to set a list of project keys to load at agent startup time.

proxies

This optional section allows to run the agent as a reverse proxy instead of only running it as a server. If this section is missing in the configuration file, the agent will simply not run as a reverse proxy.

The proxies configuration node is an array that can contain one or more reverse proxy configuration. For each proxy that you need to define, use the configuration directives described in the sections below.

Here is an example proxy configuration:

proxies:
    -
        listen: "127.0.0.1:80"
        forward: "http://localhost:8080/"
        project_key: "SOME_PROJECT_KEY_HERE"
        preserve_host: true

In this setup, the proxy will listen for HTTP traffic, which means that the agent will inspect requests that come with the http scheme. Hence, a request on http://example.com/demo will be seen as a request for the scheme http, the hostname example.com and the path demo.

As the preserve_host flag is set to true, the request will be proxified to the backend service (at http://localhost:8080/) with a request header Host: example.com.

Please note that, if the redirection.io proxy is located behind a SSL termination software (HAProxy, Varnish, nginx, Envoy, or any other reverse proxy) and receives requests that are proxified by this proxy, these requests will be seen as http requests, not https ones. However, if you have correctly configured the trusted_proxies directive, the redirectionio-agent embedded proxy will trust the Forwarded header, if present, then fallback to the X-Forwarded-Proto header, then use the request URI.

In other words, this means that, in the case HTTPS requests are received in your infrastructure by a SSL reverse proxy, then forwarded to the redirectionio-agent proxy as http requests, they will still be seen as https requests by the redirection.io routing engine (and they will be logged as https requests), as long as the SSL reverse proxy adds the Forwarded or X-Forwarded-Proto header and the reverse proxy IP address is listed in the directive trusted_proxies.

listen

• Type: string
• Default: -
• This directive is required

The address of the interface to listen to, in the form address:port. Valid values are, for example, 0.0.0.0:80, 0.0.0.0:443 (for listening on all network interfaces), 127.0.0.1:8080, etc. The HTTP(s) requests received on this interface will be analyzed and possibly forwarded to the forward address, if necessary.

Listening on a privileged port can require additional permissions, so you may need to run the redirectionio-agent binary with a user having appropriate permissions.

When listening on port 443, you max expect SSL-encrypted requests. In that case, you have to setup the tls_cert_file and tls_key_file configuration directives (see below).

forward

• Type: string
• Default: -
• This directive is required

The address of the interface to forward the HTTP requests to, in the form scheme://address:port. Valid values are, for example, http://127.0.0.1:8080, https://192.168.0.30:443, etc.

project_key

• Type: string
• Default: -
• This directive is required

This is the project key to use for this HTTP proxy. The HTTP requests received by this proxy will be matched against the ruleset of this redirection.io project, and the transformations defined in rules (redirections, etc.) executed directly at the proxy level.

add_rule_ids_header

• Type: boolean
• Default: false
• This directive is optional
• Available since: 2.2.5

If this setting is enabled (value true), a response header named X-RedirectionIo-RuleIds will be added to the response. Its value will contain the list of the redirection.io rule ids applied to this response, separated by the ; character.

This option is available since the release 2.2.5.

allow_invalid_certificates

• Type: boolean
• Default: false
• This directive is optional
• Available since: 2.5.2

If this setting is enabled (value true), the agent will allow forwarding requests to SSL backends with invalid certificates. If the backend is a simple HTTP backend, this setting has no effect. If the allow_invalid_certificates setting is disabled (ie. not defined or set to false) and the SSL backend serves invalid certificates, then the proxy will return a 503 Service not available response.

compress

• Type: boolean
• Default: false
• This directive is optional
• Available since: 2.5.0

If this setting is enabled (value true), then the agent reverse proxy will send compressed responses to the client, based on the value of the Accept-Encoding request header.

If this setting is disabled, the agent always sends back uncompressed responses.

override_matching_request_host

• Type: string
• Default: -
• This directive is optional
• Available since: 2.3.0

This directive allows to force the hostname to use when matching the request. This is useful if you have defined rules using URL triggers with absolute URL values (containing a scheme, a domain name, etc.), while the proxy receives the requests using another hostname.

Please note that, most of the time, you should prefer the use of the trusted_proxies directive along with the correct X-Forwarded-Host request header.

override_matching_request_scheme

• Type: string
• Default: -
• Possible values: http or https
• This directive is optional
• Available since: 2.3.0

This directive allows to force the scheme to use when matching the request. This is useful if you have defined rules using a URL trigger with an absolute URL value (containing a scheme, a domain name, etc.), but you want to use the same rules in a proxy definition with another scheme.

Imagine, for example, a proxy definition listening https traffic only, using a project in which rules are defined using the http:// scheme: URLs would never be matched, as no rule uses the https:// scheme. In this case, setting the override_matching_request_scheme value to http will force the module to match with this scheme, not the real request scheme.

Please note that, most of the time, you should prefer the use of the trusted_proxies directive along with the correct X-Forwarded-Scheme request header.

preserve_host

• Type: boolean
• Default: false
• This directive is optional

This setting allows to pass the Host header to the forward destination, so your backend application can use the hostname value. Most of the time, you should want to enable this option.

If this option is disabled (false), then the request will be passed to the forward backend using the hostname defined in the forward address.

request_body_size_limit

• Type: string
• Default: 2MiB
• This directive is optional
• Available since: 2.4.2

This setting allows to define a limit for the size of the request body. Valid values are strings of the form <number><unit>, where <unit> can be one of B, KiB, MiB, GiB (binary system units) or KB, MB, GB, etc. (decimal system units). The <unit> can be lowercase.

tls_cert_file

• Type: string
• Default: -
• This directive is optional

If this proxy listens on port 443 or is expected to receive https traffic, please set in the tls_cert_file directive the absolute path to a SSL certificate file, in a location readable by the redirectionio-agent binary.

tls_key_file

• Type: string
• Default: -
• This directive is optional

If this proxy listens on port 443 or is expected to receive https traffic, please set in the tls_key_file directive the absolute path to a SSL key file, in a location readable by the redirectionio-agent binary.

trusted_proxies

• Type: list of strings
• Default: []
• This directive is optional
• Available since: 2.3.0

This directive is used by the agent to restrict which proxies are trusted before evaluating the X-Forwarded-* headers. This is in particular useful when using the IP Address trigger, in order to ensure that the evaluated IP address is valid and has not been forged.

The value to use for this directive is a comma-separated list of all the proxies IP addresses that should be trusted (it might also be a CIDR notation for subnetworks). For example:

trusted_proxies: ['192.168.10.7', '192.168.11.0/28']

This option is available since the release 2.3.0.

metrics_server

• Type: string
• Default: -
• This directive is optional
• Available since: 2.2.0
• Will be deprecated in: 3.0.0 (replaced with metrics)

The address of the http monitoring interface to listen to, in the form address:port. Valid values are, for example, 0.0.0.0:8080 (for listening on all network interfaces), 127.0.0.1:8080, etc. Un-commenting this option enables a HTTP endpoint which serves a JSON feed containing metrics.

Here is an example response:

[
    {
        "id": "5e11a6df-3a1a-4f30-ba91-3c79715857d2:60c680c1-3a5b-491f-a3e4-d701a82d64cd",
        "name": "example-com",
        "rules": 4242,
        "milliseconds_since_last_update": 14452,
        "last_update_duration": 325,
        "log_buffer_size": 1233,
        "total_requests": 18279
    },
    {
        "id": "fe75016c-1b78-4e6b-a769-3ecd38c4c4b1:2552637c-1080-4069-960a-07da9eb25bc3",
        "name": "other-example-com",
        "rules": 117392,
        "milliseconds_since_last_update": 13273,
        "last_update_duration": 1581,
        "log_buffer_size": 899,
        "total_requests": 21924
    }
]

The keys of the JSON array are the project keys for the redirection.io projects that this agent instance is aware of.

metrics

When running an infrastructure, ops often need to monitor how services work, and be able to detect if issues are about to happen.

The redirection.io agent provides an optional monitoring endpoint, which exposes several metrics intended to be loaded by your favorite monitoring dashboard.

Here is an example metrics configuration:

metrics:
    listen: 127.0.0.1:31401
    format: "prometheus" # "json" or "prometheus"

listen

• Type: string
• Default: -
• This directive is optional
• Available since: 2.4.1

The address of the http monitoring interface to listen to, in the form address:port. Valid values are, for example, 0.0.0.0:8080 (for listening on all network interfaces), 127.0.0.1:8080, etc.

Un-commenting this option enables a HTTP endpoint which serves a JSON feed containing metrics (see the format option below).

format

• Type: string
• Default: json
• Possible values: json or prometheus
• This directive is optional
• Available since: 2.4.1

This setting defines the format of the data exposed by the monitoring endpoint.

When the format is defined as json (which is the default value), a request on the metrics listen address generates a response of the form:

[
    {
        "id": "5e11a6df-3a1a-4f30-ba91-3c79715857d2:60c680c1-3a5b-491f-a3e4-d701a82d64cd",
        "name": "example-com",
        "rules": 4242,
        "milliseconds_since_last_update": 14452,
        "last_update_duration": 325,
        "log_buffer_size": 1233,
        "total_requests": 18279
    },
    {
        "id": "fe75016c-1b78-4e6b-a769-3ecd38c4c4b1:2552637c-1080-4069-960a-07da9eb25bc3",
        "name": "other-example-com",
        "rules": 117392,
        "milliseconds_since_last_update": 13273,
        "last_update_duration": 1581,
        "log_buffer_size": 899,
        "total_requests": 21924
    }
]

The keys of the JSON array are the project keys for the redirection.io projects that this agent instance is aware of.

The format can also be defined as prometheus, which then exposes a prometheus-compatible format, with more information available:

• global agent metrics:
  • process_cpu_seconds_total: Total user and system CPU time spent, in seconds.
  • process_max_fds: Maximum number of open file descriptors allowed for the agent process.
  • process_open_fds: Number of open file descriptors.
  • process_resident_memory_bytes: Resident memory size, in bytes.
  • process_start_time_seconds: Start time of the process since unix epoch in seconds.
  • process_threads: Number of OS threads in the process.
  • process_virtual_memory_bytes: Virtual memory size in bytes.
• per-project metrics:
  • redirectionio_matching_requests: Number of times a matching request was executed by the router
  • redirectionio_rules_count: Number of rules inside the router
  • redirectionio_last_update_duration:Duration, in milliseconds, of the last update
  • redirectionio_logs_buffer_size: Memory size of the logs buffer, in bytes
  • redirectionio_duration_since_last_update: Duration since last router update, in milliseconds
  • redirectionio_duration_since_last_matching_request: Duration since last router request, in milliseconds

log

This optional section contains directives to configure the agent logging. Each subsection is optional and independent of one another. If this section is missing in the configuration file, the agent will not log errors or warnings.

file

This subsection can contain directives for file logging.

level

• Type: string
• Default: info
• This directive is optional

This fixes the log verbosity. It can be one of debug, info, error.

path

• Type: string
• Default: -
• This directive is required

Path of the log file to write in. Please make sure that user running the agent (redirectionio, by default) has the right to create this file.

stderr

Use the stderr to output log messages to the standard error output.

level

• Type: string
• Default: info
• This directive is required

This fixes the log verbosity. It can be one of debug, info, error.

syslog

This subsection can contain directives for syslog logging.

network

• Type: string
• Default: ""
• This directive is required

The type of connection to use ("", udp or tcp). Use an empty string ("") to use a local syslog server.

address

• Type: string
• Default: ""
• This directive is required

The address of the logging server to use. Use an empty string ("") to use a local syslog server. In order to use a remote server, or a server running on a non-standard port, use a string of the form address:port, eg. 192.168.0.30:514

level

• Type: string
• Default: info
• This directive is required

This fixes the log verbosity. It can be one of debug, info, error.

tag

• Type: string
• This directive is optional

This allows to attach a tag to the logs sent to syslog.

Using environment variables in the configuration file

It is possible to use env vars in the configuration file, which may be helpful in dynamic environments like cloud-based deployments:

See this example:

instance_name: "${REDIRECTIONIO_INSTANCE_NAME}"
datadir: "${REDIRECTIONIO_APPLICATION_DIR}/rules"
log:
    file:
        level: info
        path: "${REDIRECTIONIO_APPLICATION_DIR}/log/agent.log"

In this example, the REDIRECTIONIO_INSTANCE_NAME and REDIRECTIONIO_APPLICATION_DIR environment variables must be defined set when the agent is started (of course, you can use any name you want for the environment variables). If they are not defined, the variables will be replaced with an empty string:

$ redirectionio-agent
level=error msg="[server] The agent is Misconfigured: The 'instance_name' is not defined."

When the required env vars are set, they are interpreted during the startup of the agent:

$ export REDIRECTIONIO_INSTANCE_NAME=example-agent
$ export REDIRECTIONIO_APPLICATION_DIR=/tmp/redirectionio
$ redirectionio-agent
level=info msg="redirectionio-agent (version: 2.2.6, build date: 2021-08-05T13:25:14Z)" instance_name=example-agent
...

Complete example

Here is a complete configuration example, for reference:

# Name of this instance, mandatory for logging
instance_name: "A UNIQUE NAME"

# Interface listening. "" for all interfaces (not advised)
# /run/redirectionio.sock or 127.0.0.1:10301
listen: 127.0.0.1:10301
#listen: /run/redirectionio.sock # If you use a file, please make sure that user running the agent has the right to create it

# Whether or not to store rules on the disk.
# This allows to restart the agent with existing rules without the need to call redirection.io's backend
persist: true

# Directory where the rules will be persisted
datadir: /var/lib/redirectionio
  
# This setting allows to define the logging behavior for this instance. If it is set to "true", the instance will
# be marked as allowed to log traffic. If it is set to "false", the instance will not collect traffic logs.
# If this setting is not defined, or defined with any other value than "true" or "false", it will be managed
# by the value defined in the manager graphical interface. Else, this setting takes precedence over the logging
# behavior defined in the manager graphical interface.
logging: true

# This setting allows to set this instance in "test" mode. If this setting is not defined, or defined
# with any other value than "true" or "false", it will be managed by the value defined in the
# manager graphical interface. Else, this setting takes precedence over the logging behavior defined in the
# manager graphical interface. If it is enabled, the value of the "logging" setting will be forced to "false".
test_mode: false

# Number of items to store in memory.
# This makes the agent much faster, but will consume more memory on very large rulesets if this setting is high
# With the default value, a very large ruleset should consume about 150-200 MB of memory
# Set to 0 to disable the cache and minimize memory usage
cache: 10000

# Keys of projects to preload
# This allows to preload the agent with some projects rulesets at startup time
project_keys:
    - "A VALID PROJECT KEY"

# Run the agent as a reverse proxy
# Leave it empty to disable the reverse proxy feature
proxies:
    -
        listen: "0.0.0.0:80"
        forward: "http://127.0.0.1:8080"
        project_key: "A PROJECT KEY"
        preserve_host: true # Preserve host header, defaults to false
        add_rule_ids_header: true # Add a X-RedirectionIo-RuleIds response header, defaults to false
        compress: true # compress the responses, depending on the Accept-encoding header
    -
        listen: "0.0.0.0:443"
        forward: "http://192.168.1.47:80"
        project_key: "A SECOND PROJECT KEY"
        preserve_host: true # Preserve host header
        cert_file: /path/to/ssl/cert.pem
        key_file: /path/to/ssl.key
    -
        listen: "0.0.0.0:9080"
        forward: "https://192.168.1.47:443"
        project_key: "A THIRD PROJECT KEY"
        preserve_host: true
        compress: true
        allow_invalid_certificates: true # allow proxifying requests to the SSL backend, served with invalid certificated
    -
        listen: "127.0.0.1:8080"
        forward: "http://127.0.0.1:8081"
        project_key: "A FOURTH PROJECT KEY"
        preserve_host: true # Preserve host header, defaults to false
        trusted_proxies:
            - "127.0.0.1"
            - "192.168.10.0/28"
    -
        listen: "127.0.0.1:8081"
        forward: "http://127.0.0.1:8090"
        project_key: "A FIFTH PROJECT KEY"
        preserve_host: true # Preserve host header, defaults to false
        override_matching_request_scheme: https
        override_matching_request_host: "example.com"
    -
	    # proxify the incoming requests to the host resolved at "some-other-domain.tld", with this domain as the Host of the request
	    # use for this proxy configuration the same project key as the first proxy defined above
        listen: "127.0.0.1:8083"
        forward: "http://some-other-domain.tld:80"
        project_key: "A PROJECT KEY"
        preserve_host: false

# Define the metrics monitoring endpoint configuration. This enables a http endpoint which serves some metrics about the agent
metrics:
    # Where the metrics server should listen, example values: 127.0.0.1:8080, 0.0.0.0:8080, /run/redirectionio-metrics.sock
    listen: 127.0.0.1:31401
    # Metrics format, can either be "json" or "prometheus", defaults to "json"
    format: "prometheus"

# Sets log configuration. Each section is optional and independent of one another.
log:
    file:
        level: debug
        path: /proc/self/fd/2

    stderr:
        level: info

    syslog:
        # pushes to the local syslog
        network: "" # empty string
        address: "" # empty string
        level: info
        tag: redirectionio-agent

Agent command line options

The redirection.io agent supports several command line options:

$ redirectionio-agent --help

Usage of redirectionio-agent:
  --config-file string
        Use this option to explicitly pass a configuration file
  --debug
        Enable debug mode
  --test
        Test config and exit, return non zero exit code if there is an error in the config
  --version
        Show version and exit

Start-defined configuration file

By default, the agent will start using a /etc/redirectionio/agent.yml or $HOME/.redirectionio/agent.yml configuration file. If required, you can also pass the --config-file option to force a given configuration file:

/usr/bin/redirectionio-agent --config-file /path/to/some/configuration/directory/some-config-file.yml

Debug level

The --debug option gives more details on how the agent behaves. You may use it to troubleshoot your redirection.io agent install:

/usr/bin/redirectionio-agent --debug

Configuration test

The --test option allows to test the configuration file and get insights on mis-configurations. This command returns a non-zero code when the agent is not properly configured, so you may use it in provisioning scripts to only restart/reload the agent when the configuration is properly defined.

/usr/bin/redirectionio-agent --test

time="2021-05-03T12:42:42+00:00" level=info msg="redirectionio-agent (version: 2.2.0, build date: 2021-05-03T12:42:00Z)" instance_name=example-com

This option is available since the release 2.2.0.

Get the current agent version

The --version option displays the release name of the installed redirection.io agent:

/usr/bin/redirectionio-agent --version

2.3.0

This option is available since the release 2.2.0.

Using the agent behind a HTTPS proxy

In some enterprise configurations, it can be complicated to get a direct connection between the agent and the redirection.io APIs. Some hosting layouts require that connections to external services go through a company proxy.

The redirection.io agent is able to use a https proxy to communicate with the redirection.io APIs. This can be done by using a HTTPS_PROXY environment variable:

HTTPS_PROXY=http://192.168.0.254:8443 /usr/bin/redirectionio-agent

This option is available since the release 2.2.4.

Tweaking the redirectionio-agent service file

If you installed the agent using our official APT packages or our RPM official packages, you may need to override the systemd service file provided with the package. This can be done in an elegant way using systemctl:

$ sudo systemctl edit redirectionio-agent.service

In the editor, paste the following:

[Service]
Environment="https_proxy=http://192.168.0.254:8443"

Then, restart the agent, it should be using the configured proxy:

$ sudo systemctl restart redirectionio-agent.service

In order to remove this service override, delete the override file:

$ sudo rm -Rf /etc/systemd/system/redirectionio-agent.service.d
$ sudo systemctl daemon-reload
$ sudo systemctl restart redirectionio-agent.service

If you need to include several environment variables, you can use the following override (where /etc/environment is a file containing the environment variables):

[Service]
EnvironmentFile=/etc/environment