1. remco

remco is a lightweight configuration management tool that renders templates from backend data and reloads services when values change.

It watches backends like etcd, consul, vault, redis, zookeeper, NATS KV, or environment variables, pushes changes through template rendering, and optionally execs or signals a child process.

1.1 Sections

1.1.1 Runtime & Operations

• Template resource — how a template resource is structured
• Exec mode — running a child process per resource
• Commands — check and reload commands
• Process lifecycle — signal handling
• CLI reference — flags, exit codes, and version info
• Zombie reaping — automatic reaping when running as PID 1
• Telemetry — metrics sinks

1.1.2 Configuration & Backends

• Environment variables — variable substitution in config
• Configuration options — full config reference
• Sample config — complete TOML example
• Sample resource — resource block example
• Backends — supported key-value stores
• Plugins — JSON-RPC backend plugins

1.1.3 Template Authoring

• Template engine — pongo2 syntax and whitespace handling
• Template functions — available template functions
• Template filters — built-in and custom filters

1.1.4 Extensions & Examples

• Env plugin example — environment backend as a plugin
• Consul plugin example — consul service endpoint plugin
• Dynamic haproxy configuration — full tutorial with Docker

2. Template resource

A template resource in remco consists of the following parts:

• one optional exec command.
• one or many templates.
• one or many backends.

3. Exec mode

Remco can run one arbitrary child process per template resource. When any of the provided templates change and the check command (if any) succeeds, remco will notify or restart the child process.

3.1 How it works

If a reload_signal is configured, remco sends that signal to the child when templates change. If no reload_signal is set, remco kills the child process (with the configured kill_signal) and restarts it.

The child process must remain in the foreground. If it forks into the background, remco will be unable to track it and will restart it endlessly.

3.2 Child process failure and restart

If the child process dies, the template resource is marked as failed. Remco automatically restarts it after a random delay of 0 to 30 seconds.

This jitter helps prevent thundering-herd problems in large clusters where many instances might restart simultaneously.

3.3 Signal forwarding

Every signal that remco receives and does not handle itself (SIGINT, SIGTERM, SIGHUP, SIGCHLD) is forwarded to the child process. This means sending SIGUSR2 (or any custom signal) to the remco process will relay it to the child.

See process lifecycle for the full signal handling table.

3.4 Configuration

The exec configuration parameters can be found here: exec configuration.

4. Commands

Each template can have two optional commands:

4.1 Check command (check_cmd)

Executed before the rendered template is written to the destination path. The check command runs in a shell (/bin/sh -c).

• The rendered template is written to a temporary staging file first. You can reference this staging file with {{ .src }} in the check command.
• If the check command returns a non-zero exit code, the destination file is not overwritten. The old configuration is left in place.
• If no check_cmd is configured, the template is always written.

Example:

check_cmd = "nginx -t -c {{ .src }}"

4.2 Reload command (reload_cmd)

Executed after the destination file has been updated. The reload command also runs in a shell.

• You can reference the destination path with {{ .dst }}.
• If the reload command returns a non-zero exit code, remco logs an error but the destination file remains updated.

Example:

reload_cmd = "systemctl reload nginx"

4.3 Resource-level commands

A template resource also supports two higher-level commands:

• start_cmd — runs once when all templates in the resource have been processed successfully for the first time.
• reload_cmd (resource-level) — runs after any template in the resource is updated. This is distinct from the template-level reload_cmd.

4.4 Interaction with watch mode

When a backend is in watch mode and a change is detected:

1. The template is re-rendered.
2. If configured, check_cmd runs against the staging file.
3. If the check passes, the staging file replaces the destination.
4. If configured, the template-level reload_cmd runs.
5. If configured, the resource-level reload_cmd runs.

The template configuration parameters can be found here: template configuration.

5. Process lifecycle

Remco's lifecycle can be controlled with signals.

5.1 Supported signals

5.2 Graceful shutdown

On SIGINT or SIGTERM, remco:

1. Stops all backend watchers/pollers.
2. Sends the configured kill_signal (default: SIGTERM) to the child process if running in exec mode.
3. Waits up to kill_timeout seconds for the child to exit.
4. If the child has not exited, kills it (equivalent to SIGKILL).
5. Exits with a code equal to the number of resources that had errors (capped at 125).

5.3 Configuration reload (SIGHUP)

When remco receives SIGHUP, it re-reads the configuration file from disk (including environment variable expansion). The new configuration replaces the old one — resources are stopped and restarted to match the new config.

5.4 Exec-mode signal forwarding

When running in exec mode, any signal not explicitly handled by remco is forwarded to the child process. This includes SIGUSR2 and any custom signals.

6. Command-line reference

6.1 Flags

6.2 Exit codes

When remco exits after finishing its work, the exit code reflects the number of resources that encountered errors. This applies to -onetime runs and any other run where all resources complete on their own. The exit code is capped at 125 — if more than 125 resources fail, remco exits with 125.

If remco receives SIGINT or SIGTERM, it performs a graceful shutdown and exits with code 0.

6.3 Version output

remco -version prints:

remco Version: <version>UTC Build Time: <timestamp>Git Commit Hash: <hash>Go Version: <go version>Go OS/Arch: <os>/<arch>

6.4 Configuration reload

-onetime is not the only way to control remco's lifecycle. See process lifecycle for signal handling.

7. Zombie reaping

See: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/

If Remco detects that it runs as pid 1 (for example in a Docker container) it will automatically reap zombie processes. No additional init system is needed.

8. Telemetry

Remco can expose different metrics about its state using go-metrics. You can configure any type of sink supported by go-metrics through the configuration file. All the configured sinks will be aggregated using FanoutSink.

Currently supported sinks are:

• inmem
• prometheus
• statsd
• statsite

The different configuration parameters can be found here: telemetry configuration.

Exposed metrics:

• files.template_execution_duration — Duration of template execution
• files.check_command_duration — Duration of check_command execution
• files.reload_command_duration — Duration of reload_command execution
• files.stage_errors_total — Total number of errors in file staging action
• files.staged_total — Total number of successfully staged files
• files.sync_errors_total — Total number of errors in file syncing action
• files.synced_total — Total number of successfully synced files
• backends.sync_errors_total — Total errors in backend sync action
• backends.synced_total — Total number of successfully synced backends

9. Environment variables

Environment variable substitution is applied to the entire configuration file before TOML parsing. You can use $VARIABLE_NAME or ${VARIABLE_NAME} and the text will be replaced with the value of the environment variable.

[resource]  [resource.backend.etcd]    nodes = ["${ETCD_HOST}:2379"]    username = "$ETCD_USER"    password = "$ETCD_PASS"

9.1 How it works

Substitution is performed by Go's os.ExpandEnv, which means:

• Only simple $VAR and ${VAR} forms are supported.
• Bash-style defaults like ${VAR:-default} are not supported. Use a template function like getv with a default value inside templates instead.
• Undefined variables expand to an empty string.

9.2 Quoting

Because substitution happens before TOML parsing, values that may be empty or contain special characters should be quoted:

password = "${MY_PASSWORD}"

Without quotes, an empty expansion could produce invalid TOML.

10. Configuration options

10.1 Global configuration options

• log_level(string): Valid levels are panic, fatal, error, warn, info and debug. Default is info.
• log_format(string): The format of the log messages. Valid formats are text and json.
• include_dir(string): Specify an entire directory of resource configuration files to include. Data from files will be imported directly into resource array.
• filter_dir(string): A folder with custom JavaScript template filters.
• pid_file(string): A filename to write the process-id to.

10.2 Resource configuration options

• name(string, optional): You can give the resource a name which is added to the logs as field resource. Default is the name of the resource file.
• start_cmd(string, optional) An optional command which is executed once all templates have been processed successfully.
• reload_cmd(string, optional) An optional command which is executed as soon as a template belonging to the resource has been successfully recreated.

10.3 Exec configuration options

• command(string): This is the command to exec as a child process. Note that the child process must remain in the foreground.
• kill_signal(string): This defines the signal sent to the child process when remco is gracefully shutting down. The application needs to exit before the kill_timeout, it will be terminated otherwise (like kill -9). The default value is "SIGTERM".
• kill_timeout(int): The maximum amount of time (seconds) to wait for the child process to gracefully terminate. Default is 10.
• reload_signal(string): This defines the signal sent to the child process when some configuration data is changed. If no signal is specified the child process will be killed (gracefully) and started again.
• splay(int): A random splay to wait before killing the command. May be useful in large clusters to prevent all child processes to reload at the same time when configuration changes occur. Default is 0.

10.4 Template configuration options

• src(string): The path of the template that will be used to render the application's configuration file.
• dst(string): The location to place the rendered configuration file.
• make_directories(bool, optional): Make parent directories for the dst path as needed. Default is false.
• check_cmd(string, optional): An optional command to check the rendered source template before writing it to the destination. If this command returns non-zero, the destination will not be overwritten by the rendered source template. We can use {{.src}} here to reference the rendered source template.
• reload_cmd(string, optional): An optional command to run after the destination is updated. We can use {{.dst}} here to reference the destination.
• mode(string, optional): The permission mode of the file (e.g. "0644"). If empty and the destination file already exists, the existing file's mode is preserved. If the file does not exist, the default is "0644".
• UID(int, optional): The UID that should own the file. Defaults to the effective uid.
• GID(int, optional): The GID that should own the file. Defaults to the effective gid.

10.5 Backend configuration options

The default_backends section lets you define backend values that apply to every resource. When remco loads a resource, it first deep-copies the default_backends into that resource, then overlays the resource's own [backend] settings on top. This means resource-level values override defaults, and any field left empty in the resource inherits the default.

See the example configuration to see how global default values can be set for individual backends.

10.5.1 valid in every backend

• keys([]string): The backend keys that the template requires to be rendered correctly. The child keys are also loaded.
• watch(bool, optional): Enable watch support. Default is false.
• prefix(string, optional): Key path prefix. Default is "".
• watchKeys([]string, optional): Keys list to watch. Default is same as keys.
• interval(int, optional): The backend polling interval in seconds. Can be used as a reconciliation loop for watch or standalone. If interval is 0 or unset, and neither watch nor onetime is true, the interval defaults to 60.
• onetime(bool, optional): Render the config file and quit. Default is false.

10.5.2 etcd

• nodes([]string): List of backend nodes.
• srv_record(string, optional): A DNS server record to discover the etcd nodes.
• scheme(string, optional): The backend URI scheme (http or https). This is only used when the nodes are discovered via DNS srv records and the api level is 2. Default is http.
• client_cert(string, optional): The client cert file.
• client_key(string, optional): The client key file.
• client_ca_keys(string, optional): The client CA key file.
• username(string, optional): The username for the basic_auth authentication.
• password(string, optional): The password for the basic_auth authentication.
• version(uint, optional): The etcd api-level to use (2 or 3). Default is 2.

10.5.3 nats

• nodes([]string, optional): List of backend nodes. If none is provided the default URL nats://localhost:4222 is used.
• bucket(string): The nats kv bucket where your config keys are stored
• username(string, optional): The username for the basic_auth authentication.
• password(string, optional): The password for the basic_auth authentication.
• token(string, optional): The authentication token for the nats server
• creds(string, optional): The path to an NATS 2.0 and NATS NGS compatible user credentials file

10.5.4 consul

• nodes([]string): List of backend nodes.
• srv_record(string, optional): A DNS server record to discover the consul nodes.
• scheme(string, optional): The backend URI scheme (http or https).
• client_cert(string, optional): The client cert file.
• client_key(string, optional): The client key file.
• client_ca_keys(string, optional): The client CA key file.

10.5.5 file

• filepath(string): The filepath to a yaml or json file containing the key-value pairs. This can be a local file or a remote http/https location.
• httpheaders(map[string]string): Optional HTTP-headers to append to the request if the file path is a remote http/https location.

10.5.6 redis

• nodes([]string): List of backend nodes.
• srv_record(string, optional): A DNS server record to discover the redis nodes.
• password(string, optional): The redis password.
• database(int, optional): The redis database.

10.5.7 vault

• node(string): The backend node.
• auth_type(string): The vault authentication type. (token, approle, app-id, userpass, github, cert, kubernetes)
• auth_token(string): The vault authentication token. Only used with auth_type=token or github.
• role_id(string): The vault app role. Only used with auth_type=approle and kubernetes.
• secret_id(string): The vault secret id. Only used with auth_type=approle.
• app_id(string): The vault app ID. Only used with auth_type=app-id.
• user_id(string): The vault user ID. Only used with auth_type=app-id.
• username(string): The username for the userpass authentication.
• password(string): The password for the userpass authentication.
• client_cert(string, optional): The client cert file.
• client_key(string, optional): The client key file.
• client_ca_keys(string, optional): The client CA key file.

10.5.8 env

The environment backend has no configuration fields beyond the common backend options. It reads values directly from environment variables.

10.5.9 zookeeper

• nodes([]string): List of backend nodes.
• srv_record(string, optional): A DNS server record to discover the zookeeper nodes.

10.5.10 plugin

• path(string): The path to the plugin binary or script.
• config(map[string]interface{}): Arbitrary key-value configuration passed to the plugin. Values can be strings, numbers, booleans, or nested maps.

See the env plugin example and consul plugin example for full working plugins.

10.6 Telemetry configuration options

• enabled(bool): Flag to enable telemetry.
• service_name(string): Service name to add to every metric name. "remco" by default
• hostname(string): Hostname to use. If not provided and enable_hostname, it will be os.Hostname
• enable_hostname(bool): Enable prefixing gauge values with hostname. true by default
• enable_hostname_label(bool): Put hostname into label instead of metric name. false by default
• enable_runtime_metrics(bool): Enables profiling of runtime metrics (GC, Goroutines, Memory). true by default

10.7 Sink configuration options

10.7.1 inmem

• interval(int): How long is each aggregation interval (seconds).
• retain(int): Retain controls how many metrics interval we keep.

Sending SIGUSR1 to remco while an inmem sink is active will dump the current metrics to stderr.

10.7.2 prometheus

• addr(string): Address to expose metrics on. Prometheus stats will be available at /metrics endpoint.
• expiration(int): Expiration is the duration a metric is valid for, after which it will be untracked. If the value is zero, a metric is never expired.

10.7.3 statsd

• addr(string): Statsd/Statsite server address

10.7.4 statsite

• addr(string): Statsd/Statsite server address

11. Sample config file

#remco.toml################################################################# Global configuration################################################################log_level   = "debug"log_format  = "json"include_dir = "/etc/remco/resource.d/"pid_file    = "/var/run/remco/remco.pid"# default backend configurations.# these settings can be overwritten in the individual resource backend settings.[default_backends][default_backends.file]    onetime  = true    prefix   = "/bla"################################################################# Resource configuration################################################################[[resource]]  name = "haproxy"  start_cmd   = "echo 1"  reload_cmd  = "echo 1"  [[resource.template]]    src         = "/etc/remco/templates/haproxy.cfg"    dst         = "/etc/haproxy/haproxy.cfg"    check_cmd   = "somecommand"    reload_cmd  = "somecommand"    mode        = "0644"  [resource.backend]    # you can use as many backends as you like    # in this example vault and file    [resource.backend.vault]      node           = "http://127.0.0.1:8200"      ## Token based auth backend      auth_type      = "token"      auth_token     = "vault_token"      ## AppID based auth backend      # auth_type    = "app-id"      # app_id       = "vault_app_id"      # user_id      = "vault_user_id"      ## userpass based auth backend      # auth_type    = "userpass"      # username     = "username"      # password     = "password"      client_cert    = "/path/to/client_cert"      client_key     = "/path/to/client_key"      client_ca_keys = "/path/to/client_ca_keys"      # These values are valid in every backend      watch    = true      prefix   = "/"      onetime  = true      interval = 1      keys     = ["/"]      watchKeys = ["/haproxy/reload"]    [resource.backend.file]      httpheaders = { X-Test-Token = "XXX", X-Test-Token2 = "YYY" }      filepath = "/etc/remco/test.yml"      watch    = true      keys     = ["/prefix"]################################################################# Telemetry configuration################################################################[telemetry]  enabled = true  [telemetry.sinks.prometheus]    addr = ":2112"    expiration = 600

12. Sample resource

[exec]  command       = "/path/to/program"  kill_signal   = "SIGTERM"  reload_signal = "SIGHUP"  kill_timeout  = 10  splay         = 10[[template]]  src           = "/etc/remco/templates/haproxy.cfg"  dst           = "/etc/haproxy/haproxy.cfg"  reload_cmd    = "haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D -sf `cat /var/run/haproxy.pid`"  mode          = "0644"[backend]  [backend.etcd]    nodes    = ["http://localhost:2379"]    keys     = ["/service-registry"]    watchKeys = ["/haproxy/reload"]    watch    = true    interval = 60    version  = 3

13. Backends

Remco fetches configuration data from key-value stores via backends. Each backend can operate in two modes:

• Watch mode — the backend watches for changes in real time and triggers template re-rendering immediately.
• Interval mode — the backend polls at a fixed interval (in seconds). This is a reconciliation loop.

These modes are not mutually exclusive. You can enable both watch and interval simultaneously, so that watch provides low-latency updates and interval provides a safety net.

If neither watch nor onetime is set and interval is 0 or unset, the interval defaults to 60 seconds.

Every backend implements the easykv interface.

13.1 Supported backends

13.2 Backend configuration

The different configuration parameters can be found here: backend configuration.

13.3 Default backends

You can define shared backend defaults in a [default_backends] section. These values are deep-copied into every resource as a starting point. Resource-level backend settings then override the defaults.

[default_backends.etcd]  nodes = ["http://etcd1:2379"][[resource]]  [resource.backend.etcd]    nodes = ["http://etcd2:2379"]   # overrides the default    keys = ["/myapp"]

13.4 Plugin backends

Remco also supports backends as plugins via JSON-RPC. See plugins for details.

14. Plugins

Remco supports backends as plugins. There is no requirement that plugins be written in Go. Every language that can provide a JSON-RPC API is ok.

Example: env plugin.

15. Template engine

Remco uses pongo2, a Django-syntax template engine for Go. This is different from confd's Go text/template syntax. If you are migrating from confd, templates must be rewritten.

15.1 Syntax overview

Pongo2 uses {% %} for tags and {{ }} for variable output:

{% for key in gets("/config/*") %}{{ key }} = {{ getv(key) }}{% endfor %}

Auto-escaping is disabled, so HTML entities are not inserted.

15.2 Whitespace handling

Remco enables pongo2's TrimBlocks and LStripBlocks options:

• TrimBlocks — the first newline after a block tag ({% if %}, {% for %}, {% endfor %}, etc.) is stripped.
• LStripBlocks — leading whitespace (including tabs) before a block tag on the same line is stripped.

These match the behavior of Jinja2's similarly-named options. Without them, templates that mix tags and literal content often produce unwanted blank lines.

Example with both options enabled:

{% if true %}hello{% endif %}

Produces \nhello\n (not \n\nhello\n\n).

15.3 Available functions and filters

• Template functions — getv, getvs, ls, fileExists, etc.
• Template filters — parseInt, toYAML, base64, etc.

15.4 memkv store functions

The functions exists, get, gets, getv, getvs, ls, and lsdir come from the memkv library, which remco uses as an in-memory cache of the backend key-value data. They are available in every template without any additional configuration.

16. Template functions

16.1 exists

Checks if the key exists. Returns false if the key is not found.

{% if exists("/key") %}    value: {{ getv("/key") }}{% endif %}

16.2 get

Returns the KVPair where key matches its argument.

{% with get("/key") as dat %}    key: {{dat.Key}}    value: {{dat.Value}}{% endwith %}

16.3 gets

Returns all KVPair, []KVPair, where key matches its argument.

{% for i in gets("/*") %}    key: {{i.Key}}    value: {{i.Value}}{% endfor %}

16.4 getv

Returns the value as a string where key matches its argument, or an optional default value.

value: {{ getv("/key") }}

With a default value:

value: {{ getv("/key", "default_value") }}

16.5 getvs

Returns all values, []string, where key matches its argument.

{% for value in getvs("/*") %}    value: {{value}}{% endfor %}

16.6 getenv

Retrieves the value of the environment variable named by the key. It returns the value, which will be empty if the variable is not present. Optionally, you can give a default value that will be returned if the key is not present.

export HOSTNAME=`hostname`

hostname: {{getenv("HOSTNAME")}}

With a default value:

ipaddr: {{ getenv("HOST_IP", "127.0.0.1") }}

16.7 ls

Returns all subkeys, []string, where path matches its argument. Returns an empty list if path is not found.

{% for i in ls("/deis/services") %}   value: {{i}}{% endfor %}

16.8 lsdir

Returns all subkeys, []string, where path matches its argument. It only returns subkeys that also have subkeys. Returns an empty list if path is not found.

{% for dir in lsdir("/deis/services") %}   value: {{dir}}{% endfor %}

16.9 replace

Alias for the strings.Replace function.

backend = {{ replace(getv("/services/backend/nginx"), "-", "_", -1) }}

16.10 contains

Alias for the strings.Contains function.

{% if contains(getv("/services/backend/nginx"), "something") %}something{% endif %}

16.11 printf

Alias for the fmt.Sprintf function.

{{ getv (printf ("/config/%s/host_port", dir)) }}

16.12 unixTS

Wrapper for time.Now().Unix().

{{ unixTS }}

16.13 dateRFC3339

Wrapper for time.Now().Format(time.RFC3339).

{{ dateRFC3339 }}

16.14 fileExists

Checks whether a file exists at the given path. Returns true if the file exists, false otherwise.

{% if fileExists("/etc/myapp/config.yaml") %}key: {{ getv("/myapp/key") }}{% else %}key: default_value{% endif %}

16.15 lookupIP

Wrapper for the net.LookupIP function. The wrapper returns the IP addresses in alphabetical order.

{% for ip in lookupIP("kube-master") %} {{ ip }}{% endfor %}

16.16 lookupSRV

Wrapper for the net.LookupSRV function. The wrapper returns the SRV records in alphabetical order.

{% for srv in lookupSRV("xmpp-server", "tcp", "google.com") %}  target: {{ srv.Target }}  port: {{ srv.Port }}  priority: {{ srv.Priority }}  weight: {{ srv.Weight }}{% endfor %}

16.17 createMap

Creates a hashMap to store values at runtime. This can be useful if you want to generate json/yaml files.

{% set map = createMap() %}{{ map.Set("Moin", "Hallo2") }}{{ map.Set("Test", 105) }}{{ map | toYAML }}{% set map2 = createMap() %}{{ map2.Set("Moin", "Hallo") }}{{ map2.Set("Test", 300) }}{{ map2.Set("anotherMap", map) }}{{ map2 | toYAML }}

The hashmap supports the following methods:

• m.Set("key", value) adds a new value of arbitrary type referenced by "key" to the map
• m.Get("key") get the value for the given "key"
• m.Remove("key") removes the key and value from the map

16.18 createSet

Creates a set to store values at runtime. This can be useful if you want to generate json/yaml files.

{% set s = createSet() %}{{ s.Append("Moin") }}{{ s.Append("Moin") }}{{ s.Append("Hallo") }}{{ s.Append(1) }}{{ s.Remove("Hallo") }}{{ s | toYAML }}

The set supports the following methods:

• s.Append("string") adds a new string to the set. Attention - the set is not sorted or the order of appended elements guaranteed.
• s.Remove("string") removes the given element from the set.
• s.Contains("string") check if the given string is part of the set, returns true or false otherwise
• s.SortedSet() returns a new list where all elements are sorted in increasing order. This method should be used inside the template with a for-in loop to generate a stable output file not changing order of elements on every run.

{% set s = createSet() %}{% s.Append("Moin") %}{% s.Append("Hi") %}{% s.Append("Hallo") %}{% for greeting in s %}{{ greeting }}{% endfor %}{% for greeting in s.SortedSet() %}{{ greeting }}{% endfor %}

The output of the first loop is not defined, it can be in every order (like Moin Hallo Hi or Hi Hallo Moin and so on). The second loop returns every time Hallo Hi Moin (items sorted as string in increasing order).

17. Template filters

17.1 Builtin filters

17.1.1 parseInt

Takes the given string and parses it as a base-10 integer (64bit).

{{ "12000" | parseInt }}

17.1.2 parseFloat

Takes the given string and parses it as a float64.

{{ "12000.45" | parseFloat }}

17.1.3 base64

Encodes a string as base64.

{{ "somestring" | base64 }}

17.1.4 base64decode

Decodes a base64-encoded string.

{{ "c29tZXN0cmluZw==" | base64decode }}

17.1.5 base

Alias for the path.Base function.

{{ "/home/user/test" | base }}

17.1.6 dir

Alias for the path.Dir function.

{{ "/home/user/test" | dir }}

17.1.7 split

Alias for the strings.Split function.

{% for i in ("/home/user/test" | split:"/") %}{{i}}{% endfor %}

17.1.8 mapValue

Returns a map element by key.

{{ getv("/some_yaml_config") | parseYAML | mapValue:"key" }}

17.1.9 index

Returns an array element by index.

{{ "/home/user/test" | split:"/" | index:"1" }}

17.1.10 parseYAML

Returns an interface{} of the yaml value.

{% for value in getvs("/cache1/domains/*") %}{% set data = value | parseYAML %}{{ data.type }} {{ data.name }} {{ data.addr }}{% endfor %}

17.1.11 parseJSON

Returns an interface{} of the json value. (parseYAMLArray is a deprecated alias.)

{% for value in getvs("/cache1/domains/*") %}{% set data = value | parseJSON %}{{ data.type }} {{ data.name }} {{ data.addr }}{% endfor %}

17.1.12 toJSON

Converts data, for example the result of gets or lsdir, into a JSON object.

{{ gets("/myapp/database/*") | toJson}}

17.1.13 toPrettyJSON

Converts data, for example the result of gets or lsdir, into a pretty-printed JSON object, indented by four spaces.

{{ gets("/myapp/database/*") | toPrettyJson}}

17.1.14 toYAML

Converts data, for example the result of gets or lsdir, into a YAML string. Accepts an optional parameter to control indentation (e.g. indent=4).

{{ gets("/myapp/database/*") | toYAML }}

With custom indentation:

{{ gets("/myapp/database/*") | toYAML:"indent=4" }}

17.1.15 sortByLength

Returns the sorted array. Works with []string and []KVPair.

{% for dir in lsdir("/config") | sortByLength %}{{dir}}{% endfor %}

17.2 Custom filters

It is possible to create custom filters in JavaScript. If you want to create a toEnv filter, which transforms file system paths to environment variables, you must create the file toEnv.js in the configurable filter directory.

The filter code could look like:

In.split("/").join("_").substr(1).toUpperCase();

There are two predefined variables:

• In: the filter input (string)
• Param: the optional filter parameter (string)

As the parameter one string is possible only, the parameter string is added with a double-colon to the filter name ("yadda" | filter:"paramstr"). When the filter function needs multiple parameter all of them must be put into this one string and parsed inside the filter to extract all parameter from this string (example "replace" filter below).

Remark:

• console object for logging does not exist, therefore no output (for debugging and similar) possible.
• variable declaration must use var as other keywords like const or let are not defined
• the main script must not use return keyword, last output is the filter result.

17.2.1 Examples

reverse filter

Put file reverse.js into the configured "filter_dir" with following content:

function reverse(s) {     var o = "";     for (var i = s.length - 1; i >= 0; i--)        o += s[i];     return o;}reverse(In);

Call this filter inside your template (e.g. my-reverse-template.tmpl) with:

{% set myString = "hip-hip-hooray" %}myString is {{ myString }}reversed myString is {{ myString | reverse }}

Output is:

myString is hip-hip-hoorayreversed myString is yarooh-pih-pih

replace filter

Put file replace.js into the configured "filter_dir" with following content:

function replace(str, p) {    var params = [' ','_'];  // default: replace all spaces with underscore    if (p) {        params = p.split(',');  // split all params given at comma    }  // if third param is a "g" like "global" change search string to regexp    if (params.length > 2 && params[2] == 'g') {        params[0] = new RegExp(params[0], params[2]);    }    // javascript string.replace replaces first occurence only if search param is a string    // need regexp object to replace all occurences    return str.replace(params[0], params[1]);}replace(In, Param)

Use this inside the template as:

{% set myString = "hip-hip-hooray" %}myString is {{ myString }}replace with default params (spaces): {{ myString | replace }}only replace first "-" with underscore is {{ myString | replace:"-,_" }}replace all "-" with underscore is {{ myString | replace:"-,_,g" }}

Output is:

myString is hip-hip-hoorayreplace with default params (spaces): hip-hip-hoorayonly replace first "-" with underscore is hip_hip-hoorayreplace all "-" with underscore is hip_hip_hooray

18. Env plugin example

This is the env backend as a plugin. If you want to try it yourself, then just compile it and move the executable to /etc/remco/plugins.

package mainimport (	"context"	"log"	"net/rpc/jsonrpc"	easykv "github.com/HeavyHorst/easykv"	"github.com/HeavyHorst/easykv/env"	"github.com/HeavyHorst/remco/pkg/backends/plugin"	"github.com/natefinch/pie")func main() {	p := pie.NewProvider()	if err := p.RegisterName("Plugin", &EnvRPCServer{}); err != nil {		log.Fatalf("failed to register Plugin: %s", err)	}	p.ServeCodec(jsonrpc.NewServerCodec)}type EnvRPCServer struct {	// This is the real implementation	Impl easykv.ReadWatcher}func (e *EnvRPCServer) Init(args map[string]interface{}, resp *bool) error {	// use the data in args to create the ReadWatcher	// env var doesn't need any data	var err error	e.Impl, err = env.New()	return err}func (e *EnvRPCServer) GetValues(args []string, resp *map[string]string) error {	erg, err := e.Impl.GetValues(args)	if err != nil {		return err	}	*resp = erg	return nil}func (e *EnvRPCServer) Close(args interface{}, resp *interface{}) error {	e.Impl.Close()	return nil}func (e EnvRPCServer) WatchPrefix(args plugin.WatchConfig, resp *uint64) error {	var err error	*resp, err = e.Impl.WatchPrefix(context.Background(), args.Prefix, easykv.WithKeys(args.Opts.Keys), easykv.WithWaitIndex(args.Opts.WaitIndex))	return err}

Then create a config file with this backend section.

[backend]  [[backend.plugin]]    path = "/etc/remco/plugins/env"    keys = ["/"]    interval = 60    watch = false    [backend.plugin.config]     # these parameters are not used in the env backend plugin     # but other plugins may need some data (password, prefix ...)     a = "hallo"     b = "moin"

19. Consul plugin example

Here is another simple example plugin that speaks to the consul service endpoint instead of the consul kv-store like the built in consul backend.

package mainimport (	"encoding/json"	"fmt"	"log"	"net/rpc/jsonrpc"	"path"	"strconv"	easykv "github.com/HeavyHorst/easykv"	"github.com/HeavyHorst/remco/pkg/backends/plugin"	consul "github.com/hashicorp/consul/api"	"github.com/natefinch/pie")func NewConsulClient(addr string) (*consul.Client, error) {	config := consul.DefaultConfig()	config.Address = addr	c, err := consul.NewClient(config)	if err != nil {		return nil, err	}	return c, nil}type ConsulRPCServer struct {	client *consul.Client}func main() {	p := pie.NewProvider()	if err := p.RegisterName("Plugin", &ConsulRPCServer{}); err != nil {		log.Fatalf("failed to register Plugin: %s", err)	}	p.ServeCodec(jsonrpc.NewServerCodec)}func (c *ConsulRPCServer) Init(args map[string]string, resp *bool) error {	var err error	if addr, ok := args["addr"]; ok {		c.client, err = NewConsulClient(addr)		if err != nil {			return err		}		*resp = true		return nil	}	return fmt.Errorf("I need an Address !")}func (c *ConsulRPCServer) GetValues(args []string, resp *map[string]string) error {	r := make(map[string]string)	passingOnly := true	for _, v := range args {		addrs, _, err := c.client.Health().Service(v, "", passingOnly, nil)		if len(addrs) == 0 && err == nil {			log.Printf("service ( %s ) was not found", v)		}		if err != nil {			return err		}		for idx, addr := range addrs {			key := path.Join("/", "_consul", "service", addr.Service.Service, strconv.Itoa(idx))			service_json, _ := json.Marshal(addr)			r[key] = string(service_json)		}	}	*resp = r	return nil}func (c *ConsulRPCServer) Close(args interface{}, resp *interface{}) error {	// consul client doesn't need to be closed	return nil}func (c *ConsulRPCServer) WatchPrefix(args plugin.WatchConfig, resp *uint64) error {	return easykv.ErrWatchNotSupported}

The config backend section could look like this:

[backend]  [[backend.plugin]]    path = "/etc/remco/plugins/consul-service"    keys = ["consul"]    interval = 60    onetime = false    [backend.plugin.config]     addr = "localhost:8500"

20. Dynamic haproxy configuration with Docker, registrator and etcd

20.1 The haproxy template

We expect registrator to write the service data in this format to etcd:

/services/<service-name>/<service-id> = <ip>:<port>

The scheme (tcp, http) and the host_port of the service is configurable over the following keys:

/config/<service-name>/scheme/config/<service-name>/host_port

We create the template for the haproxy configuration file first. Create the file haproxy.tmpl and add the following config blocks:

Some default configuration parameters:

global    daemon    maxconn 2048defaults    timeout connect 5000ms    timeout client 500000ms    timeout server 500000ms    log globalfrontend name_resolver_http    bind *:80    mode http

This block creates the http acl's. We iterate over all directories under /config (the services) and create a url_beg and a hdr_beg(host) acl if the service has a scheme configured. Note that we sort the services by length and iterate in reversed order (longest services first). That way we can have services with the same prefix, for example redis_test, and redis.

{% for dir in lsdir("/config") | sortByLength reversed %}  {% if exists(printf("/config/%s/scheme", dir)) %}    {% if getv(printf("/config/%s/scheme", dir)) == "http" %}      {% if ls(printf("/services/%s", dir)) %}        acl is_{{ dir }} url_beg /{{ dir }}        acl is_{{ dir }} hdr_beg(host) {{ dir }}        use_backend {{ dir }}_servers if is_{{ dir }}      {% endif %}    {% endif %}  {% endif %}{% endfor %}

If we had one service named redis we would get:

acl is_redis url_beg /redis acl is_redis hdr_beg(host) redisuse_backend redis_servers if is_redis

Optional template block to expose a service on a host port: We iterate over all services under /config, test if the scheme and host_port is configured and create the host port configuration.

{% for dir in lsdir("/config") %}  {% if exists (printf ("/config/%s/scheme", dir )) %}    {% if exists (printf("/config/%s/host_port", dir )) %}      {% if ls(printf("/services/%s", dir)) %}                frontend {{ dir }}_port                mode {{ getv (printf("/config/%s/scheme", dir)) }}                bind *:{{ getv (printf ("/config/%s/host_port", dir)) }}                default_backend {{ dir }}_servers      {% endif %}    {% endif %}  {% endif %}{% endfor %}

If we had one service named redis with scheme=tcp and host_port=6379 we would get:

frontend redis_portmode tcpbind *:6379 default_backend redis_servers

The last block creates the haproxy backends. We iterate over all services and, if a scheme is set, create the backend {service_name}_servers.

{% for dir in lsdir("/services") %}  {% if exists(printf("/config/%s/scheme", dir)) %}backend {{ dir }}_servers        mode {{ getv (printf ("/config/%s/scheme", dir)) }}        {% for i in gets (printf("/services/%s/*", dir)) %}            server server_{{ dir }}_{{ base (i.Key) }} {{ i.Value }}        {% endfor %}    {% endif %}{% endfor %}

If we had one service named redis with scheme=tcp we could get for example:

backend redis_servers     mode tcp    server server_redis_1 192.168.0.10:32012    server server_redis_2 192.168.0.10:35013

20.2 The remco configuration file

We also need to create the remco configuration file. Create a file named config and insert the following toml configuration.

################################################################# Global configuration################################################################log_level = "debug"log_format = "text"[[resource]]name = "haproxy"[[resource.template]]  src = "/etc/remco/templates/haproxy.tmpl"  dst = "/etc/haproxy/haproxy.cfg"  reload_cmd 	  = "haproxy -f {{.dst}} -p /var/run/haproxy.pid -D -sf `cat /var/run/haproxy.pid`"  [resource.backend]    [resource.backend.etcd]      nodes = ["${ETCD_NODE}"]      keys = ["/services", "/config"]      watchKeys = ["/haproxy/reload"]      watch = true      interval = 60

20.3 The Dockerfile

FROM alpine:3.4ENV REMCO_VER 0.8.0RUN apk --update add --no-cache haproxy bash ca-certificatesRUN wget https://github.com/HeavyHorst/remco/releases/download/v${REMCO_VER}/remco_${REMCO_VER}_linux_amd64.zip && \    unzip remco_${REMCO_VER}_linux_amd64.zip && rm remco_${REMCO_VER}_linux_amd64.zip && \    mv remco_linux /bin/remcoCOPY config /etc/remco/configCOPY haproxy.tmpl /etc/remco/templates/haproxy.tmplENTRYPOINT ["remco"]

20.4 Build and run the container

You should have three files at this point:

.├── config├── Dockerfile└── haproxy.tmpl

20.4.1 Build the docker container

sudo docker build -t remcohaproxy .

20.4.2 Optionally test the container

20.4.2.1 Put some data into etcd

etcdctl set /services/exampleService/1 someip:portetcdctl set /config/exampleService/scheme httpetcdctl set /config/exampleService/host_port 1234

In this example we connect to a local etcd cluster.

sudo docker run --rm -ti --net=host -e ETCD_NODE=http://localhost:2379 remcohaproxy

You should see something like this:

[Dec 16 18:26:20]  INFO remco[1]: Target config out of sync config=/etc/haproxy/haproxy.cfg resource=haproxy source=resource.go:66[Dec 16 18:26:20] DEBUG remco[1]: Overwriting target config config=/etc/haproxy/haproxy.cfg resource=haproxy source=resource.go:66[Dec 16 18:26:20] DEBUG remco[1]: Running haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D -sf `cat /var/run/haproxy.pid` resource=haproxy source=resource.go:66[Dec 16 18:26:20] DEBUG remco[1]: "" resource=haproxy source=resource.go:66[Dec 16 18:26:20]  INFO remco[1]: Target config has been updated config=/etc/haproxy/haproxy.cfg resource=hapco source=resource.go:66[Dec 16 18:26:20] DEBUG remco[1]: [Reaped child process 60] source=main.go:87[Dec 16 18:26:24] DEBUG remco[1]: Retrieving keys backend=etcd key_prefix= resource=haproxy source=resource.go:66[Dec 16 18:26:24] DEBUG remco[1]: Compiling source template resource=haproxy source=resource.go:66 template=/etc/remco/templates/haproxy.tmpl[Dec 16 18:26:24] DEBUG remco[1]: Comparing staged and dest config files dest=/etc/haproxy/haproxy.cfg resource=haproxy source=resource.go:66 staged=.haproxy.cfg389124299[Dec 16 18:26:24] DEBUG remco[1]: Target config in sync config=/etc/haproxy/haproxy.cfg resource=haproxy source=resource.go:66

20.4.3 Run registrator

sudo docker run -d \    --name=registrator \    --net=host \    --volume=/var/run/docker.sock:/tmp/docker.sock \    gliderlabs/registrator:latest \      etcd://localhost:2379/services

Now every container gets automatically registered under /services. You can then configure the scheme and optionally the host_port of each service that you want to expose.