Public beta

Turn ClickHouse telemetry into query insight.

Click-Dog turns ClickHouse spans and query logs into traces, query families, and dashboards in your observability backend.

Get started View on GitHub

trace · slow-query-monitor live · clickhouse

SELECT events JOIN users GROUP BY day 1,284 ms

query · root1284ms

read events487ms

merge parts312ms

hash join users436ms

aggregate day231ms

sort + limit140ms

Exports to

DatadogOTLP/gRPC

SplunkHEC

OpenTelemetry backendsHoneycomb · Grafana Tempo · Jaeger · Elastic · Any OTLP

Features

Understand your ClickHouse queries.

Follow slow queries, compare performance, and control what reaches your observability backend.

Native slow-query traces

Export ClickHouse's native trace topology, enriched with query-log context so slow traces carry the database behavior that caused them.

Normalized query families

Group thousands of statements into stable shapes that make p95, volume, and trend changes easier to reason about.

Baselines and regressions

Capture an explicit known-good window, then detect conservative latency regressions and failure spikes without silently moving the baseline.

Finding policy and notifications

Gate automation with severity-based exit codes and send privacy-reviewed new or critical findings to webhooks and Datadog Events.

Query-text privacy

Choose raw, redacted, normalized-only, or no query text at the final export boundary, with fail-closed normalized-only behavior.

Datadog dashboards

Create dashboards for application query analysis, exported user activity, and Click-Dog health.

Architecture

One bounded pipeline, from source to sink.

Click-Dog runs beside the database, outside the request path. It reads telemetry over a read-only connection, filters and batches at the source, then sends data to one or more backends.

Source

ClickHouse

system.opentelemetry_span_log

read-only · no DDL

Query log

system.query_log · backfill

Click-Dog

Collector

poll → filter → dedupe → batch

SQL filterLRU dedupebatchcircuit breakerbackoffleader election

Sinks · fan-out

Datadog

OTLP/gRPC

Splunk

HEC

Honeycomb

OTLP/gRPC

Any OTLP

Tempo · Jaeger · more

Query analysis

Inspect a query and its related spans.

Explore bounded operational activity across users, databases, tables, and operations. It is not an audit log.

Group and enrich queries

Roll normalized_query_hash values into stable shapes and add bounded query-log context.

Compare with known good

Capture a fixed baseline before a change, then compare a later non-overlapping window for latency regressions and failure spikes.

Gate and notify

Choose a warning or critical failure threshold and explicitly deliver eligible findings to webhooks or Datadog Events.

Drill into the evidence

Keep the bounded local report, then follow an exact hash through query and native-trace evidence.

query families · p95 last 24h 12 shapes

Query familyCallsp95Trend

SELECT events JOIN users …48.2k1.28s

SELECT count() FROM orders …211k340ms

INSERT INTO metrics …1.9M22ms

SELECT * FROM sessions WHERE …17.4k512ms

SELECT uniq(uid) FROM events …9.1k2.04s

Safety

Run Click-Dog safely in production.

Click-Dog limits its source access and stays outside the request path.

Source and export protection

Read-only ClickHouse sessionsAll runtime connections enforce readonly=2.

No schema changesNo DDL requirement and no write grants to your database.

Bounded pollingSQL-level filters, lookback windows, and max spans per cycle cap load.

Failure backpressureCircuit breaker and adaptive backoff reduce pressure during incidents.

Export healthTrack each sink independently with export metrics. Batching controls export pacing.

Rollout control

Validate offlineCheck configuration before connecting to source or sink.

Test exporter routingclick-dog test export asks every configured sink to accept a local span.

Prove native tracingclick-dog test tracing verifies ClickHouse's sampled parent-child topology end to end.

Dry-run real readsExercise the fetch and filter path while discarding exports.

Probe readinessWatch /healthz, /readyz, /status, and metrics before and after enabling scheduled mode.

Use secure transportTLS and mTLS are available for collector and source links.

Deployment

Deploy with the tools you already use.

Binary or container

Run the static Linux binary directly or generate a Docker Compose stack.

systemd

Installer-generated service, config validation, and local health checks.

Kubernetes

Generated manifests for a centralized cluster reader with readiness probes.

Ansible

Fleet rollout, canary playbook, bounded concurrency, and repeatable config.

Configuration

Start with the essentials.

Use a practical starter, see the bare minimum, or explore production controls.

click-dog.yaml

Starter config

Adds a password and normalized query text. Set CLICKHOUSE_PASSWORD before running.

clickhouse:
  host: localhost
  password: ${CLICKHOUSE_PASSWORD}

exporters:
  otel:
    - collector_address: localhost:4317

filters:
  query_text_mode: normalized_only

monitor:
  min_trace_duration_ms: 1000

Minimal config

For a local ClickHouse with no password and an OTLP collector. Uses the default raw query text policy.

clickhouse:
  host: localhost

exporters:
  otel:
    - collector_address: localhost:4317

monitor:
  min_trace_duration_ms: 1000

Advanced config

Production controls for TLS, filtering, resilience, and monitoring. Replace the example hosts and certificate paths.

clickhouse:
  host: ch.example.com
  port: 9000
  database: system
  username: default
  password: ${CLICKHOUSE_PASSWORD}
  secure: true
  ca_cert: /etc/click-dog/clickhouse-ca.pem
  max_open_conns: 2
  max_idle_conns: 1
  query_timeout_s: 30

exporters:
  otel:
    - collector_address: otel.example.com:4317
      service_name: click-dog-monitor
      secure: true
      ca_cert: /etc/click-dog/otel-ca.pem

monitor:
  enabled: true
  min_trace_duration_ms: 1000
  check_interval_s: 30
  max_spans_per_cycle: 1000
  dedup_cache_size: 10000
  circuit_breaker:
    enabled: true
    failure_threshold: 3
    success_threshold: 1
    reset_timeout_s: 60
  backoff:
    enabled: true
    max_interval_s: 300
    backoff_factor: 2.0

filters:
  query_text_mode: normalized_only
  blacklist_queries:
    - "^SYSTEM"
    - "^INSERT INTO.*\\.inner\\."
  blacklist_operations:
    - MergeTreeSource
    - MergeTreeMarksLoader
    - MergeTreeIndex
    - MergeTreeSequentialSource
    - VFSWrite
    - WriteBufferFromS3
    - ConcurrentJoin
    - QueryPipelineEx

metrics:
  enabled: true
  listen_address: ":9090"
  otlp:
    enabled: true

health:
  enabled: true
  listen_address: ":8686"

log_level: info

Validate and run → Production profile YAML → Full configuration reference →

Click-Dog production support

Take Click-Dog into production with its maintainer.

Work directly with Colt Consulting, Click-Dog's creator and maintainer, on topology, configuration, privacy, integrations, and go-live checks.

View production support Get started