ACE Certification Preparation Guide: Section 3 — Ensuring the successful operation of a cloud solution (~30% of the exam)

📚 Official exam guide: Associate Cloud Engineer certification — always confirm section weightings against the current Google Cloud exam guide.

This guide covers exam Section 3 — day-2 operations — using the RAD platform foundation modules. App_CloudRun and App_GKE carry most of the load (revisions, traffic, CI/CD, backups, alerts); Services_GCP supplies the infrastructure alerts and audit logging. Deploy the Serverless application profile (plus the Kubernetes application profile for the kubectl labs and the Operations & security add-ons profile for 3.4) from the Lab Map.

---

3.1 Managing compute resources

⏱ ~90 min · 💰 low; Cloud Deploy stages each run their own service — destroy after the lab · ⚙️ Requires: Serverless application profile; enable_cicd_trigger + a GitHub repo for the CI/CD lab

Why the exam cares — Operating compute means deploying new versions safely (canary/blue-green via traffic splitting), scaling manually and automatically, and working a Kubernetes cluster from the CLI (kubectl get/describe/logs/scale). Expect questions on shifting Cloud Run traffic between revisions and on diagnosing pods. The current guide also lists GKE inventory, node pools, horizontal and vertical Pod autoscaling, Autopilot Pod resource requests, GKE access to Artifact Registry, Cloud Run autoscaling, traffic splitting across Cloud Run, Cloud Run functions and GKE, attaching GPUs/TPUs, deploying an agent to Agent Runtime on Gemini Enterprise Agent Platform (formerly Vertex AI Agent Engine), managing notebooks in Gemini Enterprise Agent Platform Workbench (formerly Vertex AI Workbench) and BigQuery, and developer environments such as Cloud Workstations.

How RAD implements it —

Revisions and traffic (Cloud Run): every portal update creates a new revision; max_revisions_to_retain (default 7) prunes older non-serving revisions. traffic_split (default [] = 100% to latest) takes a list of { type, revision, percent, tag } entries that must sum to exactly 100 (validated at plan time), e.g. 90% TRAFFIC_TARGET_ALLOCATION_TYPE_LATEST / 10% to a named revision — a declarative canary.

CI/CD: enable_cicd_trigger (default false) with github_repository_url and github_token creates a Cloud Build trigger (cicd_trigger_config.branch_pattern default "^main$") that builds with Kaniko, pushes to Artifact Registry, and either updates the service directly or — when enable_cloud_deploy = true — creates a Cloud Deploy release through cloud_deploy_stages (default dev → staging → prod, with require_approval = true on prod). Note that enable_cloud_deploy = true without enable_cicd_trigger = true is rejected at plan time — Cloud Deploy releases only come from the CI/CD pipeline.

Kubernetes operations (App_GKE): the HPA spans min_instance_count (default 1) to max_instance_count (default 3); enable_pod_disruption_budget (default true) creates a PDB with pdb_min_available (default "1", skipped when max_instance_count = 1); enable_resource_quota (default false) caps the namespace at quota_cpu_requests/quota_cpu_limits (default "4"), quota_memory_requests (default "4Gi") / quota_memory_limits (default "8Gi") — memory values must carry a binary suffix (Gi/Mi), enforced by a plan-time validation, because Kubernetes treats a bare "4" as 4 bytes and would block all scheduling. cron_jobs deploys Kubernetes CronJobs. enable_vertical_pod_autoscaling (default false) adds a VPA alongside the HPA, and container_resources is rendered into the Pod requests that Autopilot schedules and bills on.

Artifact Registry access: images live in the project's Artifact Registry (public images are mirrored there by enable_image_mirroring, default true), and the GKE node service account gke-sa-{prefix} holds roles/artifactregistry.reader, which is all a cluster needs to pull from a same-project repository.

Compute Engine operations: the Services_GCP NFS VM runs in a MIG with auto-healing health checks and a daily snapshot schedule with 7-day retention — a live example of snapshot-based VM protection.

Try it

1. Deploy a visible change (e.g. set an env var in environment_variables), then split traffic in the portal: traffic_split = [{ type = "TRAFFIC_TARGET_ALLOCATION_TYPE_LATEST", percent = 90 }, { type = "TRAFFIC_TARGET_ALLOCATION_TYPE_REVISION", revision = "<service>-0000X", percent = 10, tag = "previous" }]. Verify, then practice the imperative equivalent the exam tests:

   gcloud run revisions list --service=<service-name> --region=us-central1
   gcloud run services update-traffic <service-name> --region=us-central1 \
     --to-revisions=<service>-0000X=10,LATEST=90
   gcloud run services describe <service-name> --region=us-central1 --format="yaml(status.traffic)"

2. On GKE, exercise the core kubectl loop:

   gcloud container clusters get-credentials gke-cluster-1 --region=us-central1
   kubectl get pods -n <namespace> -o wide
   kubectl describe pod <pod-name> -n <namespace>
   kubectl logs <pod-name> -n <namespace> --tail=50
   kubectl scale deployment <name> -n <namespace> --replicas=3   # HPA will reconcile this
   kubectl get pdb,resourcequota -n <namespace>

3. With CI/CD enabled, push a commit to main and watch: gcloud builds list --limit=3, then Cloud Deploy > Delivery pipelines to promote dev → staging and approve prod (gcloud deploy rollouts approve ... is the CLI form).
4. You know it worked when status.traffic shows your 90/10 split and the tagged revision serves on its own tag URL.

Check yourself

Q1: Five minutes after a release, error rates spike on the new Cloud Run revision. Fastest rollback?

A: gcloud run services update-traffic <service> --to-revisions=<previous-revision>=100. Revisions are immutable, so the previous one is still deployed and warm — shifting traffic is instant and requires no rebuild or redeploy. This is exactly what traffic_split declares in Terraform form.

Q2: You run kubectl scale deployment app --replicas=10 but pods drop back to 3. Why?

A: A HorizontalPodAutoscaler owns the replica count (the module creates one spanning min_instance_count–max_instance_count, default max 3). The HPA reconciles manual scaling back into its bounds; to scale higher you raise max_instance_count (or edit/remove the HPA), not the Deployment.

Q3: During cluster maintenance, why does at least one pod of the app always stay up?

A: enable_pod_disruption_budget (default true) creates a PDB with minAvailable: 1, so voluntary disruptions (node drains, upgrades) cannot evict the last available pod. Note it does not protect against involuntary failures like node crashes.

Beyond the modules — VM SSH workflows are not part of the app modules: practice gcloud compute ssh <vm> --tunnel-through-iap (the module's fw-allow-iap-ssh rule for 35.235.240.0/20 already permits this to the NFS VM), gcloud compute instances list --filter="status=RUNNING", on-demand snapshots (gcloud compute disks snapshot), creating images from disks, and MIG rolling updates (gcloud compute instance-groups managed rolling-action start-update). Also study GKE Standard node-pool add/resize/autoscale/delete commands (gcloud container node-pools create|update --enable-autoscaling|delete), which Autopilot hides. And, none of which the modules touch: traffic splitting on GKE (weighted backendRefs in a Gateway API HTTPRoute) and on Cloud Run functions (gcloud run services update-traffic, since they run on Cloud Run); attaching GPUs (gcloud compute instances create --accelerator=type=nvidia-l4,count=1, GKE GPU node pools or Autopilot nodeSelector, Cloud Run GPU services) and TPUs (TPU VMs, GKE TPU node pools); deploying an agent to Agent Runtime from the ADK or the Agent Platform SDK; creating and stopping Workbench instances and running notebooks in BigQuery Studio; and Cloud Workstations configurations and workstations as managed, IAM-controlled developer environments.

⚠️ Exam trap — gcloud run deploy always sends 100% of traffic to the new revision unless the service was previously set to manual traffic control (--no-traffic / explicit splits). If a question says "deploy without serving traffic", the answer involves --no-traffic and tags, mirroring the traffic_split tag field here.

---

3.2 Managing storage and data solutions

⏱ ~60 min · 💰 negligible (backup bucket storage) · ⚙️ Requires: Serverless application profile with a database (database_type ≠ NONE)

Why the exam cares — Day-2 data work: running and restoring backups, understanding PITR vs snapshot restore, lifecycle-managing objects, and connecting to databases to run queries. Scenario questions usually test whether you know which recovery mechanism fits an RPO, and how to keep storage costs down automatically. The current guide also covers estimating storage costs, reviewing Dataflow and BigQuery job status, backing up Firestore, Spanner, AlloyDB and Bigtable as well as Cloud SQL, managing the database fleet from Database Center, and configuring customer-managed encryption keys (CMEK).

How RAD implements it — App_CloudRun/App_GKE provision an automated logical-backup pipeline: a Cloud Scheduler job triggers a Cloud Run job that runs the backup export on backup_schedule (default "0 2 * * *" UTC), dumping the application database to a dedicated GCS backup bucket whose lifecycle rule deletes objects older than backup_retention_days (default 7). Restores are first-class: enable_backup_import (default false) with backup_source (gcs or gdrive), backup_file, and backup_format runs a one-time import job. enable_custom_sql_scripts (default false) executes .sql files from custom_sql_scripts_bucket/custom_sql_scripts_path in lexicographic order (a non-empty path is enforced at plan time), optionally as the root DB user.

Independently of these logical dumps, the Cloud SQL instance itself keeps 7 automated daily backups (04:00 UTC) with PITR enabled and 7-day transaction-log retention. Bucket hygiene is demonstrated by the platform's object-storage layer: per-bucket versioning_enabled, lifecycle_rules (age, newer-version count, storage-class transitions), soft-delete policy, and public_access_prevention (default enforced).

CMEK: Services_GCP's enable_cmek (default false) creates a Cloud KMS key ring and keys (rotated every cmek_key_rotation_period, default "7776000s" = 90 days) and grants each product's Google-managed service agent roles/cloudkms.cryptoKeyEncrypterDecrypter so Cloud SQL, AlloyDB, Artifact Registry and Cloud Storage can encrypt with your key. Set it at the first deploy only: a resource's key is fixed at creation, so turning it on later replaces the Cloud SQL instance (and its data) and the Artifact Registry repository.

Try it

1. Trigger a backup right now instead of waiting for the schedule:

   gcloud scheduler jobs list --location=us-central1
   gcloud scheduler jobs run <backup-job-name> --location=us-central1
   gcloud run jobs executions list --region=us-central1 --limit=3
   gcloud storage ls -l gs://<backup-bucket>/

2. Inspect the managed-backup side: gcloud sql backups list --instance=<instance-name> and gcloud sql instances describe <instance-name> --format="yaml(settings.backupConfiguration)" — note transactionLogRetentionDays: 7.
3. Add a lifecycle transition to a bucket in the portal (lifecycle_rules with an age-based SetStorageClass to NEARLINE), redeploy, and verify: gcloud storage buckets describe gs://<bucket> --format="yaml(lifecycle_config)".
4. Run a custom SQL script: upload 001_create_table.sql to a bucket, set enable_custom_sql_scripts = true with the bucket/path, redeploy, and read the job logs with gcloud run jobs executions describe <execution> --region=us-central1.
5. You know it worked when a fresh timestamped dump appears in the backup bucket and gcloud sql backups list shows the 7 retained automatic backups.

Check yourself

Q1: An engineer dropped a table at 14:32. The last nightly dump is from 02:00. What's the lowest-data-loss recovery, and why is it available here?

A: Point-in-time recovery — restore (clone) the Cloud SQL instance to 14:31. PITR is enabled on the instance with 7-day transaction log retention, so any second in that window is recoverable; the 02:00 GCS dump would lose 12.5 hours of writes. The exam expects you to know PITR creates a new instance rather than rewinding the existing one.

Q2: How do you keep backup-bucket costs flat without any manual cleanup?

A: An object lifecycle rule that deletes objects older than N days — exactly what backup_retention_days configures on the backup bucket. Lifecycle management is evaluated daily by GCS itself; no jobs or cron needed on your side.

Beyond the modules — gcloud sql connect is worth practicing but won't work against these instances directly because they have no public IP — connect via the Cloud SQL Auth Proxy (cloud-sql-proxy <connection-name>) or Cloud SQL Studio in the console. Also study: on-demand backups (gcloud sql backups create --instance=...), cross-product backup surfaces (Firestore export/import to GCS, GKE Backup — note Services_GCP has enable_gke_backup, default false, schedule 0 3 * * *, 30-day retention), BigQuery job history (bq ls -j), and the Pricing Calculator for storage cost estimation. Also not implemented: Firestore scheduled backups and PITR, Spanner and Bigtable backups, on-demand AlloyDB backups and restores (the module's AlloyDB cluster, when enable_alloydb = true, already has an automated weekly backup policy retaining 7 backups — inspect it with gcloud alloydb backups list --region=us-central1, then practice gcloud alloydb backups create), Dataflow job status (gcloud dataflow jobs list / describe), running queries against BigQuery (bq query), Firestore, Spanner (gcloud spanner databases execute-sql) and Bigtable (cbt read), and Database Center — the fleet-wide console view of Cloud SQL, AlloyDB, Spanner, Bigtable, Firestore and Memorystore health, inventory and recommendations.

⚠️ Exam trap — Object versioning and lifecycle deletion interact: deleting a versioned object creates a noncurrent version that still bills until a num_newer_versions/age rule purges it. "We enabled versioning and storage costs doubled" is a classic scenario.

---

3.3 Managing networking resources

⏱ ~40 min · 💰 a reserved-but-unattached static IP bills hourly — release after the lab · ⚙️ Requires: Kubernetes application profile (for reserve_static_ip) or any Cloud Armor deployment

Why the exam cares — Operations on live networks: reserving static internal/external IPs, adding subnets or expanding ranges as workloads grow, and keeping firewall rules current. The exam favors gcloud compute addresses and subnet-expansion commands, and the current guide adds custom static routes, operating Cloud DNS and Cloud NAT, and managing both VPC firewall rules and Cloud NGFW policies.

How RAD implements it — Two operational patterns are live:

• Static IPs: App_GKE's reserve_static_ip (default true) reserves a global static external IP for the Gateway/load balancer (static_ip_name optional override); App_CloudRun likewise creates a global static IP when its load balancer is enabled, and the Services_GCP NFS VM holds a reserved internal address so the share's IP survives instance replacement.
• Subnets per region: adding a region to availability_regions in Services_GCP creates a new subnet, router, and NAT gateway in that region on the next apply without touching existing subnets.
• Cloud NAT: each region gets a Cloud Router and NAT gateway ({network}-nat-gw-{region}) — inspect it with gcloud compute routers nats describe <nat-name> --router=<router-name> --region=us-central1.
• VPC firewall rules: the module owns its tag- and range-based rules (see Section 2.3); list and inspect them with gcloud compute firewall-rules list/describe rather than editing them in place.

Try it

1. List reserved addresses and identify which are global vs regional, internal vs external:

   gcloud compute addresses list
   gcloud compute addresses describe <address-name> --global

2. In the portal, set reserve_static_ip = false on App_GKE and redeploy; observe the Gateway now uses an ephemeral IP that can change on recreation — then set it back to true (production hygiene).
3. Practice the manual exam commands in your project:

   gcloud compute addresses create lab-ip --region=us-central1
   gcloud compute addresses delete lab-ip --region=us-central1 --quiet

4. You know it worked when gcloud compute addresses list shows the module's global address with status IN_USE (attached to a forwarding rule).

Check yourself

Q1: After a maintenance redeploy, customers report DNS no longer resolves to the application. The Gateway IP changed. What was misconfigured?

A: The load balancer was using an ephemeral IP (reserve_static_ip = false). Ephemeral external IPs can change whenever the fronting resource is recreated; production endpoints referenced by DNS must use a reserved static address — exactly why the module defaults to true.

Q2: A global external Application Load Balancer needs an IP. Regional or global reservation?

A: Global (gcloud compute addresses create NAME --global). Global LBs use a single anycast IP; regional addresses attach to regional resources (VMs, regional LB forwarding rules). Picking the wrong scope is a common wrong-answer option.

Beyond the modules — Not implemented: custom static routes, VPC peering management, Cloud DNS record operations, and subnet IP-range expansion (the module creates subnets but you should practice growing one): gcloud compute networks subnets expand-ip-range <subnet> --region=us-central1 --prefix-length=23 (expansion only — ranges can never shrink). Also review VPC network > Routes to understand system-generated routes vs custom routes with next hops (gcloud compute routes create lab-route --network=<vpc> --destination-range=192.168.100.0/24 --next-hop-gateway=default-internet-gateway), practice Cloud DNS record changes (gcloud dns record-sets create in a scratch zone), and manage a Cloud NGFW network firewall policy's rules and associations (gcloud compute network-firewall-policies rules update).

⚠️ Exam trap — A reserved external static IP that is not attached to anything still incurs charges; releasing unused addresses is a standard cost-cleanup answer (and an Active Assist recommendation).

---

3.4 Monitoring and logging

⏱ ~75 min · 💰 audit logging increases Cloud Logging volume/cost · ⚙️ Requires: support_users set; configure_email_notification = true + notification_alert_emails on Services_GCP; enable_audit_logging = true for the audit lab

Why the exam cares — You must read and filter logs in Logs Explorer, create alert policies on metrics, understand notification channels, know which audit logs exist by default (Admin Activity: always on; Data Access: opt-in), and diagnose workloads from their telemetry. The current guide goes further: custom metrics, VPC Flow Logs and firewall logs, exporting logs, log buckets, Log Analytics and the Log Router, diagnostic tools (Cloud Trace, Cloud Profiler, Query Insights, index advisor), the Personalized Service Health dashboard, the Ops Agent, Google Cloud Managed Service for Prometheus, Gemini Cloud Assist for Cloud Monitoring, Active Assist, and Cloud Hub.

How RAD implements it —

Metrics and alerts: setting support_users creates email notification channels plus built-in alert policies — for Cloud Run, CPU utilization > 90% and memory utilization > 90% (P99 over 60s windows, via the platform's monitoring layer); alert_policies (default []) adds custom threshold policies on any metric type, auto-filtered to your service. Services_GCP adds infrastructure alerts when configure_email_notification = true with notification_alert_emails: Cloud SQL CPU/memory/disk against alert_cpu_threshold/alert_memory_threshold/alert_disk_threshold (all default 80), and NFS VM CPU/memory/instance-down policies — the NFS memory alert reads the Ops Agent metric agent.googleapis.com/memory/percent_used. A Cloud Monitoring dashboard is created per application.

Probes: startup_probe_config and health_check_config define HTTP/TCP startup and liveness probes on both platforms — Kubernetes-style health checking you can see in the revision/pod spec.

Agents and Prometheus: the NFS VM's startup script installs the Ops Agent, which is what feeds its agent.googleapis.com/memory/percent_used alert. GKE clusters enable Google Cloud Managed Service for Prometheus (managed_prometheus { enabled = true }).

Diagnostics: enable_query_insights (Services_GCP, default false) turns on Cloud SQL Query Insights for the PostgreSQL and MySQL primaries.

Logging: GKE clusters ship SYSTEM_COMPONENTS and WORKLOADS logs. enable_audit_logging (default false) turns on allServices ADMIN_READ/DATA_READ/DATA_WRITE Data Access audit logs plus explicit Secret Manager and KMS configs.

Uptime checks: uptime_check_config (default { enabled = false, path = "/" } — you must set enabled = true; check_interval default "60s", timeout default "10s") creates a <service>-uptime-check — an HTTP GET probe from multiple global regions — plus a <service>-uptime-check-alert policy on monitoring.googleapis.com/uptime_check/check_passed that notifies the support_users channels (via the platform's monitoring layer). The check is only created when the endpoint is publicly reachable (e.g. a custom domain, the nip.io LB host, or the run.app URL with ingress_settings = "all"); internal-only deployments get none. The uptime_check_names output returns the created check's name.

Try it

1. List what monitoring the modules created:

   gcloud beta monitoring channels list --format="table(displayName, labels.email_address)"
   gcloud alpha monitoring policies list --format="table(displayName, enabled)"

   (Console: Monitoring > Alerting and Monitoring > Dashboards. If your service is publicly reachable, also open Monitoring > Uptime checks and find <service>-uptime-check probing from multiple regions.)
2. Read your application's logs with exam-style filters:

   gcloud logging read 'resource.type="cloud_run_revision" AND severity>=ERROR' --limit=10
   gcloud logging read 'resource.type="k8s_container" AND resource.labels.namespace_name="<namespace>"' --limit=10

3. Enable enable_audit_logging = true, redeploy, perform an action (read a secret value in the console), then find it:

   gcloud logging read 'logName:"cloudaudit.googleapis.com%2Fdata_access" AND protoPayload.serviceName="secretmanager.googleapis.com"' --limit=5

4. Break a probe on purpose: set health_check_config.path to /broken, redeploy, and watch the revision fail to become ready (Cloud Run) or the pod restart-loop (kubectl describe pod shows failing liveness probes). Revert.
5. You know it worked when the policies list shows the CPU/memory alerts and step 3 returns a Data Access entry naming your principal.

Check yourself

Q1: Security asks "who read the database password last Tuesday?" — can you answer with default settings?

A: No. Secret reads are Data Access (DATA_READ) audit events, which are disabled by default; only Admin Activity (e.g. changing IAM, creating secrets) is always on. With enable_audit_logging = true the module enables Data Access logs for Secret Manager (and all services), making the question answerable from Logs Explorer.

Q2: An alert policy exists and its condition fires, but nobody is emailed. First thing to check?

A: Notification channels — a policy with no (or unverified) channels evaluates conditions but notifies no one. In RAD terms: support_users/notification_alert_emails must be non-empty, which is what creates and attaches the email channels.

Q3: A GKE pod is in CrashLoopBackOff. Give the two-command diagnosis sequence.

A: kubectl describe pod <pod> -n <ns> (events: image pull errors, OOMKilled, failing probes) then kubectl logs <pod> -n <ns> --previous (output of the crashed container, not the current restart). The --previous flag is the detail the exam likes.

Beyond the modules — Not implemented: log sinks/Log Router exports (to BigQuery, Cloud Storage, or Pub/Sub for an on-premises or third-party system), custom log buckets and Log Analytics, custom metrics (log-based metrics, or application metrics written through the Monitoring API/OpenTelemetry), VPC Flow Logs and firewall rules logging, Cloud Trace/Profiler instrumentation, and index advisor. Practice: create a log-based counter metric from a Logs Explorer query, create a sink with gcloud logging sinks create, upgrade a log bucket to Log Analytics and query it with SQL, enable flow logs on a subnet (gcloud compute networks subnets update <subnet> --region=us-central1 --enable-flow-logs) and logging on a firewall rule (--enable-logging). Know the _Required bucket (Admin Activity and other required logs, 400-day retention, cannot be changed) vs _Default (30-day retention by default). Also browse, against your own deployment: the Personalized Service Health dashboard (Google incidents affecting your projects), Gemini Cloud Assist in Cloud Monitoring (ask it about a metric or alert), Active Assist recommendations (idle resources, rightsizing, over-granted IAM), and Cloud Hub (one view of active events and application health).

⚠️ Exam trap — Admin Activity audit logs are free and always on; Data Access audit logs are opt-in, billable, and high-volume (BigQuery is the one service with Data Access enabled by default). Mixing these up is the most common Section 3.4 error.