Install with Ansible¶

Use the shipped Ansible playbooks when ClickHouse runs on more than one Linux host. The standard topology is one Click-Dog systemd sidecar per ClickHouse node, reading that node's local span log and exporting to a shared OTLP receiver.

The playbook manages Click-Dog itself. It does not edit ClickHouse server configuration or create the ClickHouse monitoring user; complete those prerequisites through the configuration management you already use for ClickHouse.

Prerequisites¶

On the Ansible control node:

Ansible with SSH access to every target
a checkout containing deploy/ansible/
privilege escalation (become) on the managed hosts

On every ClickHouse host:

system.opentelemetry_span_log enabled and filling
system.query_log available
a click_dog_monitor user with SELECT on those two tables
network access from the host to the OTLP/gRPC receiver

The monitoring user should be created idempotently through your secured ClickHouse administration path. Hash the same plaintext password you store in Ansible Vault, then substitute that 64-character SHA-256 value for <hash>:

CREATE USER IF NOT EXISTS click_dog_monitor IDENTIFIED WITH sha256_hash BY '<hash>';
GRANT SELECT ON system.opentelemetry_span_log TO click_dog_monitor;
GRANT SELECT ON system.query_log TO click_dog_monitor;

Click-Dog additionally enforces readonly=2 on every connection. If the span table does not exist, follow ClickHouse's span-log instructions through the same fleet configuration system before deploying Click-Dog.

1. Create the inventory¶

Copy the example and list every ClickHouse node:

cp deploy/ansible/inventory.ini.example deploy/ansible/inventory.ini
vi deploy/ansible/inventory.ini

The important shape is:

[clickhouse]
ch-node-01.example.com
ch-node-02.example.com

[clickhouse:vars]
clickhouse_username=click_dog_monitor
otel_collector_address=otel-collector.monitoring.internal:4317

Do not commit the ClickHouse password to the inventory.

2. Store the password¶

The playbook requires clickhouse_password. Prefer an Ansible Vault file:

ansible-vault create deploy/ansible/secrets.yaml

Give it this content:

clickhouse_password: REPLACE_WITH_PASSWORD

The playbook writes the value to /etc/click-dog/.secret with mode 0600 and sets clickhouse.password_file in the generated config. Password-bearing tasks use no_log.

3. Deploy¶

Run the main playbook:

ansible-playbook \
  -i deploy/ansible/inventory.ini \
  deploy/ansible/playbook.yaml \
  --extra-vars @deploy/ansible/secrets.yaml \
  --ask-vault-pass

The playbook creates the runtime user, downloads and verifies the release, writes the config and secret, validates the config, installs the systemd unit, starts the service, and checks /healthz on every host. When it creates the runtime user, it records that numeric identity—and a group created alongside it—in the root-only installer state used by click-dog deploy uninstall.

Bound SSH concurrency with Ansible's -f flag:

ansible-playbook \
  -i deploy/ansible/inventory.ini \
  deploy/ansible/playbook.yaml \
  --extra-vars @deploy/ansible/secrets.yaml \
  --ask-vault-pass \
  -f 10

-f 10 limits hosts in flight; it does not create completed rollout waves. To finish one batch before starting the next, add serial: 10 at play level in playbook.yaml.

Variables¶

Variable	Purpose
`clickhouse_password`	Required monitoring-user password; pass through Vault or another protected variable source
`clickhouse_username`	ClickHouse username; default `click_dog_monitor`
`clickhouse_host`	Per-node ClickHouse address; default `localhost`
`clickhouse_port`	Native-protocol port; default `9000`
`otel_collector_address`	Shared OTLP/gRPC receiver; default `localhost:4317`
`keeper_hosts`	Optional Keeper hosts for HA coordination
`click_dog_version`	Release tag or `latest`
`click_dog_local_binary`	Pre-verified binary copied from the control node
`click_dog_download_url`	Mirrored archive URL; requires `click_dog_download_checksum`
`click_dog_download_checksum`	Trusted SHA-256 for a mirrored archive
`click_dog_dangerously_ignore_cosign`	Explicitly allow checksum-only official-release verification

Inventory variables or a protected variable file can override the defaults.

Canary rollout¶

Put the first rollout hosts in a clickhouse_canary group, then run the shipped canary playbook:

[clickhouse_canary]
ch-node-01.example.com

ansible-playbook \
  -i deploy/ansible/inventory.ini \
  deploy/ansible/playbook-canary.yaml \
  --extra-vars @deploy/ansible/secrets.yaml \
  --ask-vault-pass \
  -f 10

It deploys canaries first, requires the service and /readyz to become healthy, waits five minutes by default, then proceeds to the remaining fleet. Override the soak with canary_soak_minutes.

Verify the fleet¶

The playbook checks each host during deployment. For a later status pass:

ansible clickhouse \
  -i deploy/ansible/inventory.ini \
  --become \
  -m command \
  -a "/usr/local/bin/click-dog deploy status"

Follow recent service logs across the fleet with your normal log aggregation, or inspect a host directly:

ssh ch-node-01.example.com journalctl -u click-dog -n 50 --no-pager

Update and roll back¶

Resolve one version and pass it to every host:

LATEST_URL="$(curl -fsSL -o /dev/null -w '%{url_effective}' \
  https://github.com/coltconsulting/click-dog/releases/latest)" || exit 1
VERSION="${LATEST_URL##*/}"

ansible-playbook \
  -i deploy/ansible/inventory.ini \
  deploy/ansible/playbook.yaml \
  --extra-vars @deploy/ansible/secrets.yaml \
  --extra-vars "click_dog_version=${VERSION}" \
  --ask-vault-pass \
  -f 10

The playbook copies the outgoing binary to click-dog.prev before replacing it. Roll back the most recent update with:

ansible-playbook \
  -i deploy/ansible/inventory.ini \
  deploy/ansible/playbook.yaml \
  --extra-vars @deploy/ansible/secrets.yaml \
  --ask-vault-pass \
  --tags rollback

Only one previous binary is retained. See Updating for the shared version and configuration contract.

Air-gapped and mirrored releases¶

For an air-gapped fleet, verify and extract the binary on a connected host, then set click_dog_local_binary to its path on the control node:

ansible-playbook \
  -i deploy/ansible/inventory.ini \
  deploy/ansible/playbook.yaml \
  --extra-vars @deploy/ansible/secrets.yaml \
  --extra-vars "click_dog_local_binary=/secure/staging/click-dog" \
  --ask-vault-pass

This bypasses automatic download because the supplied binary is already your verified input. For an internal mirror, set both click_dog_download_url and a trusted click_dog_download_checksum; the playbook refuses a custom URL without its SHA-256. See Verify releases.

Remove the fleet deployment¶

Run the installed uninstaller on every managed host:

ansible clickhouse \
  -i deploy/ansible/inventory.ini \
  --become \
  -m command \
  -a "/usr/local/bin/click-dog deploy uninstall --yes"

This permanently removes each standard systemd installation, including its configuration and credentials. An OS user or group is removed only when the installer or Ansible playbook's root-owned metadata proves it created that exact numeric identity; pre-provisioned accounts are preserved. Repeating this command after a successful removal is a successful no-op. If the OS refuses to remove a proven runtime identity, the command exits nonzero after removing the managed Click-Dog files and reports the account or group retained for manual cleanup. Resolve that identity before removing the hosts from inventory; a repeated --yes confirms the managed installation is absent but does not retry account deletion after its ownership metadata has been removed.