Command line options

Envoy is driven both by a configuration file as well as a set of command line options. The following are the command line options that Envoy supports.

-c <path string>, --config-path <path string>

(optional) The path to the JSON/YAML/proto3 configuration file. If this flag is missing, --config-yaml is required. This will be parsed as a bootstrap configuration file. Valid extensions are .json, .yaml, .pb and .pb_text, which indicate JSON, YAML, binary proto3 and text proto3 formats respectively.

--config-yaml <yaml string>

(optional) The YAML string for a bootstrap configuration. If --config-path is also set, the values in this YAML string will override and merge with the bootstrap loaded from --config-path. Because YAML is a superset of JSON, a JSON string may also be passed to --config-yaml.

Example overriding the node id on the command line:

./envoy -c bootstrap.yaml --config-yaml "node: {id: 'node1'}"

--mode <string>

(optional) One of the operating modes for Envoy:

• serve: (default) Validate the JSON configuration and then serve traffic normally.

• validate: Validate the JSON configuration and then exit, printing either an “OK” message (in which case the exit code is 0) or any errors generated by the configuration file (exit code 1). No network traffic is generated, and the hot restart process is not performed, so no other Envoy process on the machine will be disturbed.

--admin-address-path <path string>

(optional) The output file path where the admin address and port will be written.

--local-address-ip-version <string>

(optional) The IP address version that is used to populate the server local IP address. This parameter affects various headers including what is appended to the X-Forwarded-For (XFF) header. The options are v4 or v6. The default is v4.

--base-id <integer>

(optional) The base ID to use when allocating shared memory regions. Envoy uses shared memory regions during hot restart. Most users will never have to set this option. However, if Envoy needs to be run multiple times on the same machine, each running Envoy will need a unique base ID so that the shared memory regions do not conflict.

--use-dynamic-base-id

(optional) Selects an unused base ID to use when allocating shared memory regions. Using preselected values with --base-id is preferred, however. If this option is enabled, it supersedes the --base-id value. This flag may not be used when the value of --restart-epoch is non-zero. Instead, for subsequent hot restarts, set --base-id option with the selected base ID. See --base-id-path.

--skip-hot-restart-on-no-parent

(optional) In conjunction with --restart-epoch, this flag allows for a failing hot restart to fall back to normal startup behavior. When this flag is false, if the parent instance was terminated, the child instance will also terminate during startup.

This only impacts if the parent instance was terminated before the new instance is initialized - an unexpected parent termination after interprocess communication is established will still cause the child instance to terminate due to failing communication.

--skip-hot-restart-parent-stats

(optional) In conjunction with --restart-epoch, this flag allows for hot restart to proceed without duplicating stats from the parent instance. Transferring stats can be an expensive operation; skipping it can prevent overloading the main thread with this work, or potentially dramatically increased memory load.

Has no effect if hot restarting is not in use.

--base-id-path <path_string>

(optional) Writes the base ID to the given path. While this option is compatible with --base-id, its intended use is to provide access to the dynamic base ID selected by --use-dynamic-base-id.

--concurrency <integer>

(optional) The number of worker threads to run. If not specified defaults to the number of hardware threads on the machine. If set to zero, Envoy will still run one worker thread.

-l <string>, --log-level <string>

(optional) The logging level. Non developers should generally never set this option. See the help text for the available log levels and the default.

--component-log-level <string>

(optional) The comma separated list of logging level per component. Non developers should generally never set this option. For example, if you want upstream component to run at debug level and connection component to run at trace level, you should pass upstream:debug,connection:trace to this flag. See ALL_LOGGER_IDS in /source/common/common/logger.h for a list of components. This option is incompatible with --enable-fine-grain-logging.

--cpuset-threads

(optional) This flag is used to control the number of worker threads if --concurrency is not set. If enabled, the assigned cpuset size is used to determine the number of worker threads on Linux-based systems. Otherwise the number of worker threads is set to the number of hardware threads on the machine. You can read more about cpusets in the kernel documentation.

--log-path <path string>

(optional) The output file path where logs should be written. This file will be re-opened when SIGUSR1 is handled. If this is not set, log to stderr.

--log-format <format string>

(optional) The format string to use for laying out the log message metadata. If this is not set, a default format string "[%Y-%m-%d %T.%e][%t][%l][%n] [%g:%#] %v" is used.

When used in conjunction with --log-format-escaped, the logger can be configured to log in a format that is parsable by log viewers. Known integrations are documented in the application logging configuration section.

The supported format flags are (with example output):

%v:

The actual message to log (“some user text”)

%_:

The actual message to log, but with escaped newlines (from (if using %v) “some user textnbelow”, to “some user text\nbelow”)

%j:

The actual message to log as JSON escaped string (https://tools.ietf.org/html/rfc7159#page-8).

%t:

Thread id (“1232”)

%P:

Process id (“3456”)

%n:

Logger’s name (“filter”)

%l:

The log level of the message (“debug”, “info”, etc.)

%L:

Short log level of the message (“D”, “I”, etc.)

%a:

Abbreviated weekday name (“Tue”)

%A:

Full weekday name (“Tuesday”)

%b:

Abbreviated month name (“Mar”)

%B:

Full month name (“March”)

%c:

Date and time representation (“Tue Mar 27 15:25:06 2018”)

%C:

Year in 2 digits (“18”)

%Y:

Year in 4 digits (“2018”)

%D, %x:

Short MM/DD/YY date (“03/27/18”)

%m:

Month 01-12 (“03”)

%d:

Day of month 01-31 (“27”)

%H:

Hours in 24 format 00-23 (“15”)

%I:

Hours in 12 format 01-12 (“03”)

%M:

Minutes 00-59 (“25”)

%S:

Seconds 00-59 (“06”)

%e:

Millisecond part of the current second 000-999 (“008”)

%f:

Microsecond part of the current second 000000-999999 (“008789”)

%F:

Nanosecond part of the current second 000000000-999999999 (“008789123”)

%p:

AM/PM (“AM”)

%r:

12-hour clock (“03:25:06 PM”)

%R:

24-hour HH:MM time, equivalent to %H:%M (“15:25”)

%T, %X:

ISO 8601 time format (HH:MM:SS), equivalent to %H:%M:%S (“13:25:06”)

%z:

ISO 8601 offset from UTC in timezone ([+/-]HH:MM) (“-07:00”)

%%:

The % sign (“%”)

%@:

Source file and line (“my_file.cc:123”)

%s:

Basename of the source file (“my_file.cc”)

%g:

Full relative path of the source file (“/some/dir/my_file.cc”)

%#:

Source line (“123”)

%!:

Source function (“myFunc”)

--log-format-escaped

(optional) This flag enables application log sanitization to escape C-style escape sequences. This can be used to prevent a single log line from spanning multiple lines in the underlying log. This sanitizes all escape sequences in this list. Note that each line’s final EOL character will not be escaped to preserve line format.

--restart-epoch <integer>

(optional) The hot restart epoch. (The number of times Envoy has been hot restarted instead of a fresh start). Defaults to 0 for the first start. This option tells Envoy whether to attempt to create the shared memory region needed for hot restart, or whether to open an existing one. It should be incremented every time a hot restart takes place. The hot restart wrapper sets the RESTART_EPOCH environment variable which should be passed to this option in most cases.

--enable-fine-grain-logging

(optional) Enables fine-grain logger with file level log control and runtime update at administration interface. If enabled, main log macros including ENVOY_LOG, ENVOY_CONN_LOG, ENVOY_STREAM_LOG and ENVOY_FLUSH_LOG will use a per-file logger, and the usage doesn’t need Envoy::Logger::Loggable any more. The administration interface usage is similar. Please see Administration interface for more detail. This option is incompatible with --component-log-level.

--socket-path <path string>

(optional) The output file path to the socket address for hot restart. Default to “@envoy_domain_socket” which will be created in the abstract namespace. Suffix _{role}_{id} is appended to provide name. All envoy processes wanting to participate in hot-restart together must use the same value for this option.

NOTE: The path started with “@” will be created in the abstract namespace.

--socket-mode <string>

(optional) The socket file permission for hot restart. This must be a valid octal file permission, such as 644. The default value is 600. This flag may not be used when --socket-path is start with “@” or not set.

--hot-restart-version

(optional) Outputs an opaque hot restart compatibility version for the binary. This can be matched against the output of the GET /hot_restart_version admin endpoint to determine whether the new binary and the running binary are hot restart compatible.

--service-cluster <string>

(optional) Defines the local service cluster name where Envoy is running. The local service cluster name is first sourced from the Bootstrap node message’s cluster field. This CLI option provides an alternative method for specifying this value and will override any value set in bootstrap configuration. It should be set if any of the following features are used: statsd, health check cluster verification, runtime override directory, user agent addition, HTTP global rate limiting, CDS, and HTTP tracing, either via this CLI option or in the bootstrap configuration.

--service-node <string>

(optional) Defines the local service node name where Envoy is running. The local service node name is first sourced from the Bootstrap node message’s id field. This CLI option provides an alternative method for specifying this value and will override any value set in bootstrap configuration. It should be set if any of the following features are used: statsd, CDS, and HTTP tracing, either via this CLI option or in the bootstrap configuration.

--service-zone <string>

(optional) Defines the local service zone where Envoy is running. The local service zone is first sourced from the Bootstrap node message’s locality.zone field. This CLI option provides an alternative method for specifying this value and will override any value set in bootstrap configuration. It should be set if discovery service routing is used and the discovery service exposes zone data, either via this CLI option or in the bootstrap configuration. The meaning of zone is context dependent, e.g. Availability Zone (AZ) on AWS, Zone on GCP, etc.

--file-flush-interval-msec <integer>

(optional) The file flushing interval in milliseconds. Defaults to 10 seconds. This setting is used during file creation to determine the duration between flushes of buffers to files. The buffer will flush every time it gets full, or every time the interval has elapsed, whichever comes first. Adjusting this setting is useful when tailing access logs in order to get more (or less) immediate flushing.

--drain-time-s <integer>

(optional) The time in seconds that Envoy will drain connections during a hot restart or when individual listeners are being modified or removed via LDS. Defaults to 600 seconds (10 minutes). Generally the drain time should be less than the parent shutdown time set via the --parent-shutdown-time-s option. How the two settings are configured depends on the specific deployment. In edge scenarios, it might be desirable to have a very long drain time. In service to service scenarios, it might be possible to make the drain and shutdown time much shorter (e.g., 60s/90s).

--drain-strategy <string>

(optional) Determine behaviour of Envoy during the hot restart drain sequence. During the drain sequence, the drain manager encourages draining through terminating connections on request completion, sending “Connection: CLOSE” on HTTP1, and sending GOAWAY on HTTP2.

• gradual: (default) The percentage of requests encouraged to drain increases to 100% as the drain time elapses.

• immediate: All requests are encouraged to drain as soon as the drain sequence begins.

--parent-shutdown-time-s <integer>

(optional) The time in seconds that Envoy will wait before shutting down the parent process during a hot restart. See the hot restart overview for more information. Defaults to 900 seconds (15 minutes).

--disable-hot-restart

(optional) This flag disables Envoy hot restart for builds that have it enabled. By default, hot restart is enabled.

--enable-mutex-tracing

(optional) This flag enables the collection of mutex contention statistics (MutexStats) as well as a contention endpoint (GET /contention). Mutex tracing is not enabled by default, since it incurs a slight performance penalty for those Envoys which already experience mutex contention.

--allow-unknown-fields

(optional) Deprecated alias for --allow-unknown-static-fields.

--allow-unknown-static-fields

(optional) This flag disables validation of protobuf configurations for unknown fields. By default, the validation is enabled. For most deployments, the default should be used which ensures configuration errors are caught upfront and Envoy is configured as intended. Warnings are logged for the first use of any unknown field and these occurrences are counted in the server.static_unknown_fields statistic.

--reject-unknown-dynamic-fields

(optional) This flag disables validation of protobuf configuration for unknown fields in dynamic configuration. By default, this flag is set false, disabling validation for fields beyond bootstrap. This allows newer xDS configurations to be delivered to older Envoys. This can be set true for strict dynamic checking when this behavior is not wanted but the default should be desirable for most Envoy deployments. Warnings are logged for the first use of any unknown field and these occurrences are counted in the server.dynamic_unknown_fields statistic.

--ignore-unknown-dynamic-fields

(optional) This flag disables validation of protobuf configuration for unknown fields in dynamic configuration. Unlike setting --reject-unknown-dynamic-fields to false, it does not log warnings or count occurrences of unknown fields, in the interest of configuration processing speed. If --reject-unknown-dynamic-fields is set to true, this flag has no effect.

Attention

In addition to not logging warnings or counting occurrences of unknown fields, setting this option also disables counting and warnings of deprecated fields as well as work-in-progress message and fields. It is strongly recommended that this option is not set on at least a small portion of the fleet (staging, canary, etc.) in order to monitor for unknown, deprecated, or work-in-progress usage.

--disable-extensions <extension list>

(optional) This flag disabled the provided list of comma-separated extension names. Disabled extensions cannot be used by static or dynamic configuration, though they are still linked into Envoy and may run start-up code or have other runtime effects. Extension names are created by joining the extension category and name with a forward slash, e.g. envoy.grpc_credentials/envoy.grpc_credentials.file_based_metadata.

--version

(optional) This flag is used to display Envoy version and build information, e.g. c93f9f6c1e5adddd10a3e3646c7e049c649ae177/1.9.0-dev/Clean/RELEASE/BoringSSL-FIPS.

It consists of five slash-separated fields:

• source revision - git commit from which Envoy was built,

• release number - either release (e.g. 1.9.0) or a development build (e.g. 1.9.0-dev),

• status of the source tree at the build time - either Clean or Modified,

• build mode - either RELEASE or DEBUG,

• TLS library - either BoringSSL or BoringSSL-FIPS.

--enable-core-dump

(optional) This flag is intended for Linux-based systems and it’s a no-op for all other platforms. It enables core dumps by invoking prctl using the PR_SET_DUMPABLE option. This is useful for container environments when using capabilities, given that when Envoy has more capabilities than its base environment core dumping will be disabled by the kernel.

--stats-tag

(optional) This flag provides a universal tag for all stats generated by Envoy. The format is tag:value. Only alphanumeric values are allowed for tag names. For tag values all characters are permitted except for ‘.’ (dot). This flag can be repeated multiple times to set multiple universal tags. Multiple values for the same tag name are not allowed.