ClickHouse on GKE Autopilot — Lab Guide

Overview

Estimated time: 30–60 minutes

ClickHouse is an open-source (Apache-2.0) column-oriented OLAP database for real-time analytics. This module deploys it as a single-node StatefulSet on GKE Autopilot — it is the mandatory event store for the Plausible Analytics module (Plausible_GKE): Plausible's PostgreSQL holds only accounts and configuration, while every analytics event lives in this ClickHouse instance.

This lab takes you through deploying the module, connecting and querying it with clickhouse-client and curl using the Secret Manager password, the day-2 operations that matter for a stateful database (logs, restart, persistence verification, version pinning), and teardown. For the complete list of provisioned services and every configuration input (organised by group), see the Configuration Guide — this lab deliberately does not duplicate that detail so it stays accurate over time.

Objectives

By the end of this lab you will be able to:

Deploy the module from the RAD platform and locate the resources it provisions.
Retrieve the auto-generated ClickHouse password from Secret Manager and query the server over HTTP (curl) and with the native clickhouse-client.
Perform day-2 operations — read logs, restart the pod, prove PVC persistence, and understand the version pin.
Tear the deployment down cleanly.

Prerequisites

Services_GCP (provides the VPC, GKE Autopilot cluster, Artifact Registry, and shared service accounts this module depends on). You do not need to deploy this yourself first — the platform automatically detects whether it already exists in the target project and provisions it before this module if not (see Task 1).
A Google Cloud project with billing enabled.
gcloud CLI and kubectl installed; gcloud auth login and gcloud auth application-default login completed.
Project Owner (or equivalent) IAM on the project.
RAD platform access with permission to deploy modules into the project.

Set these shell variables once; every task below reuses them:

export PROJECT="<your-gcp-project-id>"
export REGION="us-central1"           # the region you deploy into

Task 1 — Deploy the module [Automated]

Click Deploy in the RAD platform top navigation, open ClickHouse (GKE) from the Platform Modules list to start configuration, set project_id, and review the inputs. The defaults are production-sensible: a StatefulSet with a 30 GiB PVC at /var/lib/clickhouse (stateful_pvc_enabled = true by default — no need to set it), a pinned 24.12-alpine image ("latest" is rejected at plan time), a bootstrapped plausible_events_db database and plausible user, and single-node mode (max_instance_count = 1, enforced at plan time). On projects capped on external global static IP quota, when the only consumer is in-cluster (Plausible in the same cluster), set service_type = "ClusterIP", reserve_static_ip = false, and enable_custom_domain = false so the database consumes no global static external IP. Review the estimated cost (if credits are enabled) and click Deploy, which opens the deployment status page with real-time logs.
The platform mirrors the clickhouse/clickhouse-server image into Artifact Registry, generates the ClickHouse user password into Secret Manager, and rolls out the StatefulSet. There is no database-init job — the image bootstraps the database and user itself on the first start of the empty data dir. On a fresh Autopilot cluster allow up to ~10 minutes before the pod is Ready (node provisioning + PVC attach + image pull; the startup probe is sized for exactly this).

Connect to the cluster and discover the resources with name-agnostic filters:

gcloud container clusters list --project "$PROJECT"
CLUSTER=$(gcloud container clusters list --project="$PROJECT" --format="value(name)" --limit=1)
gcloud container clusters get-credentials "$CLUSTER" --region="$REGION" --project="$PROJECT"

kubectl get statefulset,pvc,svc -A | grep clickhouse
NS=$(kubectl get ns -o name | grep clickhouse | head -1 | cut -d/ -f2)
echo "Cluster: $CLUSTER   Namespace: $NS"
kubectl get all,pvc -n "$NS"

Confirm the StatefulSet shows 1/1, the PVC is Bound, and the LoadBalancer Service has an external IP on port 8123.

Task 2 — Connect & query [Manual]

Retrieve the connection pieces — the LoadBalancer IP and the auto-generated password from Secret Manager:

CH_IP=$(kubectl get svc -n "$NS" \
  -o jsonpath='{.items[?(@.spec.type=="LoadBalancer")].status.loadBalancer.ingress[0].ip}')

gcloud secrets list --project "$PROJECT" --filter="name~clickhouse-password"
CH_SECRET=$(gcloud secrets list --project "$PROJECT" \
  --filter="name~clickhouse-password" --format="value(name)" --limit=1)
CH_PASS=$(gcloud secrets versions access latest --secret="$CH_SECRET" --project "$PROJECT")
echo "Endpoint: http://$CH_IP:8123"

Verify liveness and authentication over HTTP:
```
curl -s "http://$CH_IP:8123/ping"            # expect: Ok.
curl -s "http://$CH_IP:8123/ping" | wc -c    # expect: non-zero (a 200 with an empty body is a failure)
echo "SELECT version()" | curl -s "http://$CH_IP:8123/" --user "plausible:$CH_PASS" --data-binary @-
echo "SHOW DATABASES" | curl -s "http://$CH_IP:8123/" --user "plausible:$CH_PASS" --data-binary @-
```
Check the response body, not just the status code — /ping must return the literal Ok., and the wc -c count must be non-zero (a 200 with content-length: 0 is a real failure mode). SHOW DATABASES must list plausible_events_db — the database the image bootstrapped on first start. A wrong password returns an authentication error (code 516): the endpoint is never open, because the module always generates and injects CLICKHOUSE_PASSWORD.

Query with the native client (bundled in the image — no local install needed):

POD=$(kubectl get pods -n "$NS" -o name | grep clickhouse | head -1 | cut -d/ -f2)
kubectl exec -n "$NS" -it "$POD" -- \
  clickhouse-client --user plausible --password "$CH_PASS" \
  --query "SELECT currentUser(), version()"

Create a table and insert a row — this doubles as the persistence marker for Task 3 (the bootstrapped user can manage databases and tables because the module sets CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1):

cat <<'SQL' | curl -s "http://$CH_IP:8123/" --user "plausible:$CH_PASS" --data-binary @-
CREATE TABLE IF NOT EXISTS plausible_events_db.lab_check
  (id UInt32, note String, ts DateTime DEFAULT now())
  ENGINE = MergeTree ORDER BY id
SQL

echo "INSERT INTO plausible_events_db.lab_check (id, note) VALUES (1, 'survives-restart')" \
  | curl -s "http://$CH_IP:8123/" --user "plausible:$CH_PASS" --data-binary @-

echo "SELECT * FROM plausible_events_db.lab_check" \
  | curl -s "http://$CH_IP:8123/" --user "plausible:$CH_PASS" --data-binary @-

Task 3 — Operate & keep it running (Day-2) [Manual]

Logs — the ClickHouse server log flows to stdout and Cloud Logging:

kubectl logs -n "$NS" "$POD" --tail=50
gcloud logging read 'resource.type="k8s_container" AND resource.labels.namespace_name="'"$NS"'"' \
  --project "$PROJECT" --limit 20

Restart & PVC persistence check — delete the pod; the StatefulSet recreates it against the same PVC, so the row from Task 2 must survive:
```
kubectl delete pod -n "$NS" "$POD"
kubectl get pods -n "$NS" -w        # wait for 1/1 Running (Ctrl-C to stop)

echo "SELECT * FROM plausible_events_db.lab_check" \
  | curl -s "http://$CH_IP:8123/" --user "plausible:$CH_PASS" --data-binary @-
# expect: 1  survives-restart  <timestamp>
```
This is the property the whole module exists for: analytics events survive restarts, updates, and node evictions. Losing the PVC — not the pod — is what loses data. Also check disk headroom while you are here (background merges temporarily need extra space):
```
kubectl exec -n "$NS" "$POD" -- df -h /var/lib/clickhouse
```
Version pin — check what is actually running versus what was configured:
```
echo "SELECT version()" | curl -s "http://$CH_IP:8123/" --user "plausible:$CH_PASS" --data-binary @-
kubectl get statefulset -n "$NS" -o jsonpath='{.items[0].spec.template.spec.containers[0].image}'; echo
```
The image tag matches application_version, which defaults to the Plausible-pinned known-good tag 24.12-alpine — and "latest" is rejected at plan time, because Plausible CE version-pins ClickHouse and untested versions have broken it upstream (plausible/analytics#3855). To change versions, set an explicit tag in the RAD platform and click Update — and treat it as a change to validate against your Plausible release, not a routine bump. Do not kubectl edit the image; the module owns the workload spec and would revert it on the next apply.
Hand-off to Plausible — the deployment outputs are exactly what Plausible_GKE consumes: clickhouse_internal_endpoint (preferred, same cluster) or clickhouse_endpoint as clickhouse_url, and clickhouse_password_secret_id as clickhouse_password_secret. Deploy ClickHouse first, Plausible second — Plausible's migrations create the events schema in plausible_events_db.

Optional cleanup of the lab table:

echo "DROP TABLE plausible_events_db.lab_check" \
  | curl -s "http://$CH_IP:8123/" --user "plausible:$CH_PASS" --data-binary @-

Task 4 — Tear down [Automated]

On the Deployments page, open the deployment and click the Trash icon (Delete). Delete runs terraform destroy and is irreversible (the deployment record is retained for history). If a deployment is stuck and the RAD platform can no longer manage it (for example after manual changes that conflict with the Terraform state), use Purge instead — it removes the deployment from RAD's records without destroying the cloud resources. Teardown removes everything the module created — the StatefulSet, namespace, the PVC and all event data, the Secret Manager password secret, and the mirrored Artifact Registry images. If a Plausible deployment is consuming this instance, destroy or repoint Plausible first — its event history lives here. Resources owned by Services_GCP (the VPC, GKE cluster, registry) are managed separately and are not removed here.

Summary

Task	Type	Outcome
1 — Deploy	Automated	Single-node ClickHouse StatefulSet with a 30 GiB PVC, LoadBalancer on 8123, and a Secret Manager-managed password
2 — Connect & query	Manual	Ping, authenticate, and query via `curl` and `clickhouse-client` using the Secret Manager password; create a marker row
3 — Operate	Manual	Read logs, restart the pod, prove the marker row survives (PVC persistence), verify the 24.12-alpine version pin
4 — Tear down	Automated	Delete (Trash) removes all module resources including the PVC and its data

Need RAD to do something it does not do yet? Request it on the roadmap, or vote on what is already there.

By Dr Shiyghan Emmanuel Navti · Google Cloud certified: ACE · PCA · PCD · PDE · PSE · CDL

Overview​

Objectives​

Prerequisites​

Task 1 — Deploy the module [Automated]​

Task 2 — Connect & query [Manual]​

Task 3 — Operate & keep it running (Day-2) [Manual]​

Task 4 — Tear down [Automated]​

Summary​