Paperless-ngx on GKE Autopilot — Lab Guide

📖 Configuration Guide

Overview

Estimated time: 45–90 minutes

Paperless-ngx is an open-source document management system that transforms scanned documents into a searchable digital archive using OCR (Tesseract), machine-learning classification, and full-text search. This lab takes you through the full operational lifecycle of the Paperless-ngx on GKE Autopilot module on Google Cloud: deploy it, access and verify it, run it day-to-day, observe it, diagnose common problems, and tear it down.

The lab focuses on operating the GKE module and the Google Cloud platform, not on Paperless-ngx product features. For the complete list of provisioned services and every configuration input (organised by group), see the Configuration Guide — this lab deliberately does not duplicate that detail so it stays accurate over time.

Objectives

By the end of this lab you will be able to:

• Deploy the module from the RAD platform and locate the resources it provisions.
• Connect to the GKE cluster and access the running workload.
• Perform day-2 operations — inspect, scale, update, and manage secrets and storage.
• Observe the workload with Cloud Logging and Cloud Monitoring.
• Diagnose and resolve the most common deployment and runtime issues.
• Tear the deployment down cleanly.

Prerequisites

• Services_GCP deployed in the target project (provides the VPC, GKE Autopilot cluster, Cloud SQL, Redis/NFS, Artifact Registry, and shared service accounts this module depends on).
• A Google Cloud project with billing enabled.
• gcloud CLI and kubectl installed; gcloud auth login and gcloud auth application-default login completed.
• Project Owner (or equivalent) IAM on the project.
• RAD platform access with permission to deploy modules into the project.

Set these shell variables once; every task below reuses them:

export PROJECT="<your-gcp-project-id>"
export REGION="us-central1"           # the region you deploy into

---

Task 1 — Deploy the module [Automated]

1. Click Deploy in the RAD platform top navigation, open Paperless-ngx (GKE) from the Platform Modules list to start configuration, set project_id, and review the inputs. Configure only what you need — the Configuration Guide documents every input by group, with defaults. Review the estimated cost (if credits are enabled) and click Deploy, which opens the deployment status page with real-time logs.

2. The platform deploys the workload into the GKE Autopilot cluster, provisions a Cloud SQL (PostgreSQL) database with its Secret Manager secrets, a GCS Fuse media bucket, Redis connectivity, and builds the container image. Paperless-ngx runs Django database migrations automatically on first pod startup — there is no separate init job. First deploys take roughly 20–35 minutes (Cloud SQL creation dominates).

3. Connect to the cluster and discover the namespace with name-agnostic filters:

   CLUSTER=$(gcloud container clusters list --project="$PROJECT" --format="value(name)" --limit=1)
   gcloud container clusters get-credentials "$CLUSTER" --region="$REGION" --project="$PROJECT"
   
   NS=$(kubectl get ns -o name | grep paperless | head -1 | cut -d/ -f2)
   echo "Cluster: $CLUSTER   Namespace: $NS"
   kubectl get all -n "$NS"

---

Task 2 — Access & verify [Manual]

1. Confirm the workload is running and find its external address:

   kubectl get pods,svc -n "$NS"
   EXTERNAL_IP=$(kubectl get svc -n "$NS" \
     -o jsonpath='{.items[?(@.spec.type=="LoadBalancer")].status.loadBalancer.ingress[0].ip}')
   echo "External IP: $EXTERNAL_IP"
   curl -s -o /dev/null -w "%{http_code}\n" "http://${EXTERNAL_IP}"   # expect 200

2. Retrieve the admin password from Secret Manager and sign in at http://${EXTERNAL_IP}:

   ADMIN_SECRET=$(gcloud secrets list --project="$PROJECT" \
     --filter="name~paperless-admin" --format="value(name)" --limit=1)
   gcloud secrets versions access latest --secret="$ADMIN_SECRET" --project="$PROJECT"

   Sign in with the username set by admin_user (default: admin) and the password retrieved above. Paperless-ngx's own documentation covers its product features.

---

Task 3 — Operate & keep it running (Day-2) [Manual]

1. Inspect the workload — deployment, pods, and (if enabled) the horizontal autoscaler and persistent volumes:

   kubectl get deploy,pods,hpa,pvc -n "$NS"
   kubectl describe deploy -n "$NS"

2. Scale by changing the min/max instance inputs and clicking Update on the deployment details page — the module owns the workload spec, so scaling is a configuration change, not a manual kubectl scale (a manual edit would be reverted on the next apply).

3. Update the application version by changing the version input via Update on the deployment details page; a new image builds and a rolling update replaces the pods.

4. Manage secrets, storage, and jobs:

   kubectl get secrets -n "$NS"
   gcloud secrets list --project="$PROJECT" --filter="name~paperless"
   kubectl get jobs -n "$NS"          # any scheduled maintenance jobs

5. Open a database session for inspection or maintenance:

   INSTANCE=$(gcloud sql instances list --project="$PROJECT" --format="value(name)" --limit=1)
   gcloud sql connect "$INSTANCE" --user=paperless --project="$PROJECT"

---

Task 4 — Observe: Logging & Monitoring [Manual]

1. Logs — from kubectl or the Logs Explorer:

   kubectl logs -n "$NS" deploy/"$(kubectl get deploy -n "$NS" -o jsonpath='{.items[0].metadata.name}')" --tail=50

   Logs Explorer filter: resource.type="k8s_container" AND resource.labels.namespace_name="<namespace>".

2. Monitoring — open the GKE / Kubernetes dashboards and review pod CPU and memory utilisation, restart counts, and request metrics. The module can provision an uptime check when enabled; review Monitoring → Uptime checks and Alerting → Policies.

---

Task 5 — Troubleshoot & debug [Manual]

Durable techniques for the failure modes you are most likely to hit. These are platform-level diagnostics and do not change with Paperless-ngx releases.

• Pod not Ready / CrashLoopBackOff: inspect events and logs. The health probe targets / (port 8000); a probe failure means gunicorn or the database migration did not complete within the startup allowance.

  kubectl describe pod -n "$NS" <pod>          # Events section shows scheduling/probe/mount errors
  kubectl logs -n "$NS" <pod> --previous       # logs from the crashed container

• Database connection errors: confirm the Cloud SQL instance is RUNNABLE, the DB password secret materialised into the namespace, and the pod startup logs show migrations completing (Paperless-ngx migrates automatically on first boot).
• Celery / Redis connection errors: confirm the Redis host is reachable from the pod. If redis_host is blank, the NFS server IP is used — verify NFS is deployed and the pod can reach that IP.

  kubectl logs -n "$NS" <pod> | grep -i "celery\|redis\|broker"

• Pending pod / no external IP: check kubectl describe pod events for resource or quota issues, and confirm the LoadBalancer Service has an assigned IP.
• Image pull errors: confirm the image exists in Artifact Registry and the node service account can pull it.

See the Configuration Guide's Configuration Pitfalls section for setting-specific gotchas.

---

Task 6 — Tear down [Automated]

On the Deployments page, open the deployment and click the Trash icon (Delete). Delete runs terraform destroy and is irreversible (the deployment record is retained for history). If a deployment is stuck and the RAD platform can no longer manage it (for example after manual changes that conflict with the Terraform state), use Purge instead — it removes the deployment from RAD's records without destroying the cloud resources (it makes RAD forget the project). This removes everything the module created — the Kubernetes workload and namespace, Cloud SQL database, Secret Manager secrets, GCS buckets (including the media bucket), and Artifact Registry images. Resources owned by Services_GCP (the VPC, GKE cluster, shared Cloud SQL, Redis/NFS, registry) are managed separately and are not removed here.

---

Summary

Task	Type	Outcome
1 — Deploy	Automated	Module deploys the GKE workload, Cloud SQL, GCS media bucket, and secrets; migrations run on first pod start
2 — Access & verify	Manual	Connect to the cluster; health check passes; admin credential retrieved; sign in confirmed
3 — Operate	Manual	Inspect workload, scale, update version, manage secrets/storage, DB access
4 — Observe	Manual	Query Cloud Logging; review Cloud Monitoring metrics and uptime check
5 — Troubleshoot	Manual	Diagnose pod, database, Celery/Redis, scheduling, and image-pull issues
6 — Tear down	Automated	Delete (Trash) removes all module resources