Skip to main content

AWS Elastic Kubernetes Service on GKE Fleet β€” Lab Guide

πŸ“– Configuration Guide

This lab guide walks you through deploying an AWS Elastic Kubernetes Service (EKS) cluster and registering it as a GKE Attached Cluster in Google Cloud Fleet using the EKS_GKE module. You will then explore unified multi-cloud operations: accessing the EKS cluster via Google Cloud's Connect Gateway, centralised logging and monitoring through Google Cloud Observability, and fleet-wide access control β€” all from a single Google Cloud control plane.


Table of Contents​

  1. Overview
  2. Architecture
  3. Prerequisites
  4. Lab Setup
  5. Exercise 1 β€” Verify the Fleet Membership
  6. Exercise 2 β€” Access via Connect Gateway
  7. Exercise 3 β€” Deploy a Sample Workload
  8. Exercise 4 β€” Centralised Logging with Cloud Logging
  9. Exercise 5 β€” Managed Prometheus and Cloud Monitoring
  10. Exercise 6 β€” Fleet Access Control
  11. Exercise 7 β€” OIDC Federation Deep Dive
  12. Exercise 8 β€” Network Topology and Private Subnets
  13. Cleanup
  14. Reference

1. Overview​

What Is GKE Fleet?​

Google Cloud Fleet (formerly Anthos) provides a unified control plane for Kubernetes clusters across clouds and on-premises environments. By registering an AWS EKS cluster as a GKE Attached Cluster, you gain:

CapabilityWhat It Enables
Connect Gatewaykubectl access to EKS clusters via Google Cloud IAM β€” no VPN or bastion required
Cloud LoggingUnified Kubernetes system and workload logs from EKS in Cloud Logging
Managed PrometheusEKS cluster metrics collected and queryable in Cloud Monitoring
Fleet IAMSingle IAM model for access control across all fleet clusters
Multi-cloud visibilitySingle pane of glass for cluster health, nodes, and workloads

Why AWS + GCP?​

AWS and GCP represent the most common multi-cloud combination. Organisations choose this pattern for several reasons:

  • Risk mitigation: no single-cloud dependency for critical workloads
  • Regulatory requirements: some sectors mandate multi-cloud resilience
  • Incremental migration: move workloads to GCP gradually while keeping existing EKS investments
  • GCP service access: use Cloud AI/ML, BigQuery, or Spanner from AWS-hosted services

2. Architecture​

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ AWS (us-west-2) β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ VPC (10.0.0.0/16) β”‚ β”‚
β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚
β”‚ β”‚ β”‚ Public Subnets β”‚ β”‚ Private Subnets β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ (3 AZs) β”‚ β”‚ (3 AZs) β€” optional β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ β”‚ NAT Gateway β”‚ β”‚ β”‚ β”‚ EKS Node Group β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ (2–5 nodes) β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚
β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚
β”‚ β”‚ β”‚ EKS Cluster β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ β€’ Kubernetes 1.34 β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ β€’ OIDC issuer enabled β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ β€’ IAM roles for service accounts β”‚ β”‚ β”‚
β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ OIDC Federation + GKE Connect Agent (outbound HTTPS)
β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Google Cloud (us-central1) β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ GKE Fleet Hub β”‚ β”‚
β”‚ β”‚ β€’ Fleet membership: eks-cluster-<id> β”‚ β”‚
β”‚ β”‚ β€’ Platform version: 1.34.0-gke.1 β”‚ β”‚
β”‚ β”‚ β€’ Logging: SYSTEM + WORKLOADS β”‚ β”‚
β”‚ β”‚ β€’ Managed Prometheus enabled β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ Cloud Logging β”‚ β”‚Cloud Monitoringβ”‚ β”‚ Connect Gateway API β”‚ β”‚
β”‚ β”‚ (EKS logs) β”‚ β”‚(EKS metrics) β”‚ β”‚ (kubectl access) β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Module variable wiring:

EKS_GKE
aws_access_key / aws_secret_key β†’ AWS API authentication
aws_region = "us-west-2" β†’ EKS cluster region
k8s_version = "1.34" β†’ EKS Kubernetes version
min_size = 2, desired_size = 2,
max_size = 5 β†’ Node group auto-scaling bounds
trusted_users = ["user@example.com"] β†’ cluster-admin via Connect Gateway

Network Topology​

The module creates a complete AWS VPC with both public and private subnets across three availability zones. A NAT Gateway in the public subnet enables internet egress for nodes in private subnets. The GKE Connect Agent communicates outbound over HTTPS β€” no inbound AWS Security Group rules are required.


3. Prerequisites​

Required Tools​

ToolMinimum VersionInstall
gcloud CLI480.0.0Install guide
kubectl1.29+gcloud components install kubectl
gke-gcloud-auth-pluginAnygcloud components install gke-gcloud-auth-plugin
aws CLI2.xAWS CLI install
curl / jqAnySystem package manager

AWS Requirements​

You need an AWS IAM user or IAM role with permissions to create:

  • VPC, subnets, internet gateway, NAT gateway, route tables
  • EKS cluster and managed node groups
  • IAM roles and policies

Collect these two values before deploying:

  • Access Key ID (aws_access_key)
  • Secret Access Key (aws_secret_key)

GCP Permissions​

roles/container.admin
roles/gkehub.admin
roles/iam.serviceAccountAdmin
roles/logging.admin
roles/monitoring.admin

Environment Variables​

export PROJECT_ID="your-gcp-project-id"
export GCP_REGION="us-central1"
export AWS_REGION="us-west-2"
export CLUSTER_NAME="eks-cluster" # adjust if cluster_name_prefix was changed

gcloud config set project "${PROJECT_ID}"

4. Lab Setup​

4.1 Deploy via RAD UI​

Deploy the EKS_GKE module via the RAD UI. In the variable form, set the following key variables:

VariableValueNotes
project_idyour-gcp-project-idRequired
gcp_locationus-central1GCP region for Fleet membership
aws_regionus-west-2AWS region for EKS cluster
aws_access_key<your-access-key-id>AWS IAM credentials
aws_secret_key<your-secret-access-key>AWS IAM credentials
k8s_version1.34Kubernetes version
platform_version1.34.0-gke.1GKE Connect Agent version
min_size2Node group minimum nodes
desired_size2Node group desired nodes
max_size5Node group maximum nodes
trusted_users["your-email@example.com"]Users granted cluster-admin

Click Deploy and wait for provisioning to complete (approximately 20–30 minutes).

What this provisions: An AWS VPC with public and private subnets across 3 AZs, NAT Gateway, EKS cluster with OIDC issuer, managed node group (2 nodes, t3/m5 class), IAM roles for the cluster and nodes, GKE Attached Cluster registration in Fleet Hub with OIDC trust, Cloud Logging for system and workload logs, and Managed Prometheus for metrics collection.

4.2 Configure AWS CLI (Optional)​

aws configure set aws_access_key_id "${AWS_ACCESS_KEY_ID}"
aws configure set aws_secret_access_key "${AWS_SECRET_ACCESS_KEY}"
aws configure set default.region "${AWS_REGION}"

# Verify EKS cluster was created
aws eks list-clusters --region "${AWS_REGION}"

Exercise 1 β€” Verify the Fleet Membership​

Objective​

Confirm that the EKS cluster is correctly registered in Google Cloud Fleet and all managed components are healthy.

Step 1.1 β€” List Fleet Memberships​

gcloud:

gcloud container fleet memberships list --project="${PROJECT_ID}"

Expected output:

NAME                         EXTERNAL_ID                            LOCATION
eks-cluster-<id> xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx global

REST API:

curl -s \
"https://gkehub.googleapis.com/v1/projects/${PROJECT_ID}/locations/global/memberships" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
| jq '.resources[] | {name, state: .state.code}'

Step 1.2 β€” Inspect Membership Details​

gcloud:

gcloud container fleet memberships describe "${CLUSTER_NAME}" \
--project="${PROJECT_ID}"

Look for:

  • state.code: READY β€” membership is active
  • endpoint.kubernetesMetadata.kubernetesApiServerVersion β€” Kubernetes version
  • authority.issuer β€” OIDC issuer URL from EKS

REST API:

curl -s \
"https://gkehub.googleapis.com/v1/projects/${PROJECT_ID}/locations/global/memberships/${CLUSTER_NAME}" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
| jq '{
name,
state: .state.code,
k8sVersion: .endpoint.kubernetesMetadata.kubernetesApiServerVersion,
oidcIssuer: .authority.issuer
}'

Step 1.3 β€” View in Google Cloud Console​

echo "https://console.cloud.google.com/kubernetes/list/overview?project=${PROJECT_ID}"

The EKS cluster appears in the Kubernetes Engine cluster list with an AWS icon.

Step 1.4 β€” Check EKS Cluster in AWS Console​

aws eks describe-cluster \
--name "${CLUSTER_NAME}" \
--region "${AWS_REGION}" \
--query 'cluster.{name:name,status:status,version:version,endpoint:endpoint}' \
--output table

Exercise 2 β€” Access via Connect Gateway​

Objective​

Use Google Cloud's Connect Gateway to access the EKS cluster with kubectl using your Google Cloud IAM identity β€” without needing AWS credentials or direct network access.

Step 2.1 β€” Configure kubectl via Connect Gateway​

gcloud container fleet memberships get-credentials "${CLUSTER_NAME}" \
--project="${PROJECT_ID}"

# Verify the context was added
kubectl config get-contexts
kubectl config current-context

Step 2.2 β€” Verify Cluster Connectivity​

kubectl cluster-info

# Expected:
# Kubernetes control plane is running at https://connectgateway.googleapis.com/...

kubectl get nodes -o wide

Expected node output:

NAME                                           STATUS   ROLES    AGE   VERSION
ip-10-0-x-xxx.us-west-2.compute.internal Ready <none> 5m v1.34.x
ip-10-0-x-xxx.us-west-2.compute.internal Ready <none> 5m v1.34.x

Step 2.3 β€” Inspect Cluster Namespaces​

kubectl get namespaces

# Standard EKS namespaces:
# default
# kube-system
# kube-public
# kube-node-lease
# gke-connect ← GKE Connect Agent

Step 2.4 β€” Verify Admin Access​

kubectl auth can-i list pods --all-namespaces
# Expected: yes

kubectl auth can-i create clusterrolebindings
# Expected: yes

Step 2.5 β€” Inspect the GKE Connect Agent​

kubectl get pods -n gke-connect -o wide

kubectl describe pod -n gke-connect -l app=gke-connect-agent
# Note: image tag corresponds to platform_version (e.g. 1.34.0-gke.1)

Exercise 3 β€” Deploy a Sample Workload​

Objective​

Deploy a sample application to the EKS cluster via Connect Gateway and verify it appears in Cloud Logging and Cloud Monitoring.

Step 3.1 β€” Create a Namespace​

kubectl create namespace sample-workload

Step 3.2 β€” Deploy nginx​

# nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: sample-workload
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:stable
ports:
- containerPort: 80
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 200m
memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: sample-workload
annotations:
# AWS NLB annotation (for AWS load balancer)
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
selector:
app: nginx
ports:
- port: 80
targetPort: 80
type: LoadBalancer
kubectl apply -f nginx-deployment.yaml

kubectl get pods -n sample-workload -w

Step 3.3 β€” Get the Service External Endpoint​

# AWS NLB hostname (not IP) β€” may take 3-5 minutes to provision
kubectl get service nginx -n sample-workload -w

NGINX_HOST=$(kubectl get service nginx -n sample-workload \
-o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "Nginx endpoint: http://${NGINX_HOST}"

# Test the endpoint
curl -s "http://${NGINX_HOST}" | grep "<title>"

Step 3.4 β€” Verify Pod Distribution​

# Verify pods are spread across nodes/AZs
kubectl get pods -n sample-workload -o wide

# Check resource consumption
kubectl top pods -n sample-workload
kubectl top nodes

Step 3.5 β€” Generate Traffic for Logs​

for i in $(seq 1 100); do
curl -s -o /dev/null "http://${NGINX_HOST}" || true
sleep 0.3
done

Exercise 4 β€” Centralised Logging with Cloud Logging​

Objective​

Explore Kubernetes system and workload logs from the EKS cluster collected automatically by Cloud Logging via the GKE Connect Agent.

Step 4.1 β€” View Logs Explorer​

echo "https://console.cloud.google.com/logs/query?project=${PROJECT_ID}"

Step 4.2 β€” Query System Component Logs​

gcloud:

gcloud logging read \
"resource.type=k8s_cluster \
AND resource.labels.cluster_name=${CLUSTER_NAME}" \
--project="${PROJECT_ID}" \
--limit=20 \
--format=json \
| jq '.[] | {timestamp, message: .textPayload}'

REST API:

curl -s -X POST \
"https://logging.googleapis.com/v2/entries:list" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
-d "{
\"resourceNames\": [\"projects/${PROJECT_ID}\"],
\"filter\": \"resource.type=k8s_cluster resource.labels.cluster_name=${CLUSTER_NAME}\",
\"pageSize\": 10
}" | jq '.entries[] | {timestamp, severity}'

Step 4.3 β€” Query Workload Logs (nginx)​

gcloud:

gcloud logging read \
"resource.type=k8s_container \
AND resource.labels.namespace_name=sample-workload \
AND resource.labels.container_name=nginx" \
--project="${PROJECT_ID}" \
--limit=20 \
--format=json \
| jq '.[] | {timestamp, httpRequest}'

REST API:

curl -s -X POST \
"https://logging.googleapis.com/v2/entries:list" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
-d "{
\"resourceNames\": [\"projects/${PROJECT_ID}\"],
\"filter\": \"resource.type=k8s_container resource.labels.cluster_name=${CLUSTER_NAME} resource.labels.namespace_name=sample-workload\",
\"pageSize\": 10
}" | jq '.entries[].jsonPayload'

Step 4.4 β€” kube-system Logs​

gcloud logging read \
"resource.type=k8s_container \
AND resource.labels.cluster_name=${CLUSTER_NAME} \
AND resource.labels.namespace_name=kube-system" \
--project="${PROJECT_ID}" \
--limit=10 \
--format=json \
| jq '.[] | {timestamp, container: .resource.labels.container_name, message: .textPayload}'

Exercise 5 β€” Managed Prometheus and Cloud Monitoring​

Objective​

Explore Kubernetes metrics from the EKS cluster collected by Managed Prometheus and visualised in Cloud Monitoring.

Step 5.1 β€” Open the Kubernetes Engine Dashboard​

echo "https://console.cloud.google.com/monitoring/dashboards?project=${PROJECT_ID}"

Navigate to Dashboards β†’ Kubernetes Engine β†’ select the EKS cluster.

Step 5.2 β€” Node Resource Usage​

kubectl top nodes

# Expected:
# NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
# ip-10-0-x-xxx.us-west-2.compute.internal 120m 6% 512Mi 13%

Step 5.3 β€” Query Metrics via Cloud Monitoring API​

gcloud (CPU allocatable utilisation per cluster):

gcloud monitoring metrics list \
--filter="metric.type:kubernetes.io/node" \
--project="${PROJECT_ID}" \
| head -20

REST API (MQL query):

curl -s -X POST \
"https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/timeSeries:query" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
-d "{
\"query\": \"fetch k8s_node::kubernetes.io/node/cpu/allocatable_utilization | filter resource.cluster_name = '${CLUSTER_NAME}' | within 1h | group_by [resource.node_name], mean(val())\"
}" | jq '.timeSeriesData[].labelValues'

Step 5.4 β€” Pod Memory Utilisation​

REST API (MQL):

curl -s -X POST \
"https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/timeSeries:query" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
-d "{
\"query\": \"fetch k8s_container::kubernetes.io/container/memory/used_bytes | filter resource.cluster_name = '${CLUSTER_NAME}' AND resource.namespace_name = 'sample-workload' | within 30m | group_by [resource.pod_name], mean(val())\"
}" | jq '.timeSeriesData[].labelValues'

Step 5.5 β€” Create an Alert Policy​

gcloud:

gcloud alpha monitoring policies create \
--display-name="EKS High Memory" \
--notification-channels="" \
--condition-filter="metric.type=\"kubernetes.io/node/memory/allocatable_utilization\" resource.type=\"k8s_node\" resource.label.\"cluster_name\"=\"${CLUSTER_NAME}\"" \
--condition-threshold-value=0.85 \
--condition-threshold-duration=300s \
--condition-threshold-comparison=COMPARISON_GT \
--project="${PROJECT_ID}"

Exercise 6 β€” Fleet Access Control​

Objective​

Grant a colleague access to the EKS cluster using the two-layer authorisation model: Google Cloud IAM for Connect Gateway traversal and Kubernetes RBAC for API-level access.

Background: Authorisation Layers​

Developer
β”‚
β–Ό Layer 1: Google Cloud IAM
roles/gkehub.gatewayReader or gatewayEditor
(controls who can send requests through Connect Gateway)
β”‚
β–Ό Layer 2: Kubernetes RBAC
ClusterRoleBinding mapping Google identity β†’ Kubernetes ClusterRole
(controls what Kubernetes actions are allowed)
β”‚
β–Ό
EKS API Server

Step 6.1 β€” View Existing RBAC Bindings​

kubectl get clusterrolebindings \
| grep -v "^system:"

kubectl get rolebindings --all-namespaces \
| grep -v "^kube-system"

Step 6.2 β€” Grant Read-Only Access​

# Step 1: IAM permission for Connect Gateway traversal
gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
--member="user:colleague@example.com" \
--role="roles/gkehub.gatewayReader"

# Step 2: Kubernetes RBAC β€” view access to all namespaces
kubectl create clusterrolebinding colleague-view \
--clusterrole=view \
--user="colleague@example.com"

Step 6.3 β€” Grant Namespace-Scoped Edit Access​

# IAM permission (same as above β€” gateway access is project-level)
gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
--member="user:developer@example.com" \
--role="roles/gkehub.gatewayEditor"

# Namespace-scoped edit access
kubectl create rolebinding developer-edit \
--rolebinding=edit \
--user="developer@example.com" \
--namespace=sample-workload

REST API (IAM):

curl -s -X POST \
"https://cloudresourcemanager.googleapis.com/v1/projects/${PROJECT_ID}:setIamPolicy" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
-d '{
"policy": {
"bindings": [
{"role": "roles/gkehub.gatewayReader", "members": ["user:colleague@example.com"]},
{"role": "roles/gkehub.gatewayEditor", "members": ["user:developer@example.com"]}
]
}
}'

Step 6.4 β€” Audit Who Has Accessed the Cluster​

gcloud logging read \
"protoPayload.serviceName=connectgateway.googleapis.com \
AND protoPayload.request.cluster_name=${CLUSTER_NAME}" \
--project="${PROJECT_ID}" \
--limit=20 \
--format=json \
| jq '.[] | {
timestamp,
caller: .protoPayload.authenticationInfo.principalEmail,
method: .protoPayload.methodName,
status: .protoPayload.status.code
}'

Exercise 7 β€” OIDC Federation Deep Dive​

Objective​

Understand how OIDC federation works between AWS EKS and Google Cloud, enabling Google identities to authenticate to the EKS API server via Connect Gateway.

Step 7.1 β€” Inspect the OIDC Issuer​

# View the EKS OIDC issuer URL registered in Fleet
gcloud container fleet memberships describe "${CLUSTER_NAME}" \
--project="${PROJECT_ID}" \
--format="yaml(authority)"

Expected:

authority:
issuer: https://oidc.eks.us-west-2.amazonaws.com/id/<cluster-id>
workloadIdentityPool: <project-id>.hub.id.goog
identityProvider: https://gkehub.googleapis.com/projects/...

REST API:

curl -s \
"https://gkehub.googleapis.com/v1/projects/${PROJECT_ID}/locations/global/memberships/${CLUSTER_NAME}" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
| jq '.authority'

Step 7.2 β€” Verify OIDC Discovery Endpoint​

# EKS publishes an OIDC discovery document (public endpoint)
OIDC_ISSUER=$(gcloud container fleet memberships describe "${CLUSTER_NAME}" \
--project="${PROJECT_ID}" \
--format="value(authority.issuer)")

curl -s "${OIDC_ISSUER}/.well-known/openid-configuration" | jq '{issuer, jwks_uri}'

Step 7.3 β€” Direct Connect Gateway API Call​

# Get the Connect Gateway endpoint from kubeconfig
GATEWAY_URL=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
ACCESS_TOKEN=$(gcloud auth print-access-token)

# Direct REST call β€” list namespaces
curl -s "${GATEWAY_URL}/api/v1/namespaces" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
| jq '.items[].metadata.name'

# Direct REST call β€” list pods in sample-workload
curl -s "${GATEWAY_URL}/api/v1/namespaces/sample-workload/pods" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
| jq '.items[].metadata.name'

Step 7.4 β€” Understand the Token Exchange Flow​

When you run kubectl via Connect Gateway, the following happens:

  1. kubectl sends the request with your Google OAuth2 access token
  2. Connect Gateway validates the token against Google Cloud IAM
  3. Gateway exchanges the Google token for a short-lived Kubernetes token via the OIDC issuer
  4. The Kubernetes token is presented to the EKS API server
  5. EKS validates the token via its OIDC provider configuration
  6. Kubernetes RBAC evaluates the identity against ClusterRoleBindings

Exercise 8 β€” Network Topology and Private Subnets​

Objective​

Explore the AWS VPC networking created by the module and understand how the Connect Agent reaches Google Cloud without any inbound firewall rules.

Step 8.1 β€” Inspect VPC Subnets from kubectl​

# Node addresses reveal subnet assignment
kubectl get nodes -o json \
| jq '.items[] | {
name: .metadata.name,
internalIP: (.status.addresses[] | select(.type=="InternalIP") | .address),
externalIP: (.status.addresses[] | select(.type=="ExternalIP") | .address)
}'

Step 8.2 β€” Inspect Nodes via AWS CLI​

aws ec2 describe-instances \
--filters "Name=tag:kubernetes.io/cluster/${CLUSTER_NAME},Values=owned" \
--query 'Reservations[].Instances[] | [].{
ID: InstanceId,
State: State.Name,
Type: InstanceType,
PrivateIP: PrivateIpAddress,
SubnetId: SubnetId,
AZ: Placement.AvailabilityZone
}' \
--output table \
--region "${AWS_REGION}"

Step 8.3 β€” Verify Outbound Connectivity (NAT Gateway)​

The GKE Connect Agent requires outbound HTTPS access to gkehub.googleapis.com. With the NAT Gateway in place, nodes in private subnets can reach Google Cloud without public IPs:

# Verify Connect Agent is connected (Running = outbound connection maintained)
kubectl get pods -n gke-connect
kubectl describe pods -n gke-connect | grep -E "Status|Ready"

Step 8.4 β€” Review Security Groups​

aws ec2 describe-security-groups \
--filters "Name=tag:aws:eks:cluster-name,Values=${CLUSTER_NAME}" \
--query 'SecurityGroups[].{
GroupId: GroupId,
GroupName: GroupName,
InboundRules: IpPermissions | length(@),
OutboundRules: IpPermissionsEgress | length(@)
}' \
--output table \
--region "${AWS_REGION}"

Note: The Connect Agent only requires outbound port 443 egress. No inbound rules are needed for Connect Gateway access.


13. Cleanup​

Return to the RAD UI and click Undeploy on the EKS_GKE deployment. This removes:

  • The GKE Fleet membership
  • The AWS EKS cluster and node group
  • The AWS VPC, subnets, NAT Gateway, internet gateway, and all associated resources
  • IAM roles created for EKS

Manual Cleanup (if needed)​

gcloud β€” remove Fleet membership:

gcloud container fleet memberships delete "${CLUSTER_NAME}" \
--project="${PROJECT_ID}" \
--quiet

REST API β€” delete Fleet membership:

curl -s -X DELETE \
"https://gkehub.googleapis.com/v1/projects/${PROJECT_ID}/locations/global/memberships/${CLUSTER_NAME}" \
-H "Authorization: Bearer $(gcloud auth print-access-token)"

aws β€” delete EKS cluster:

# Delete node group first
aws eks delete-nodegroup \
--cluster-name "${CLUSTER_NAME}" \
--nodegroup-name "eks-node-group" \
--region "${AWS_REGION}"

# Wait for node group deletion
aws eks wait nodegroup-deleted \
--cluster-name "${CLUSTER_NAME}" \
--nodegroup-name "eks-node-group" \
--region "${AWS_REGION}"

# Then delete cluster
aws eks delete-cluster \
--name "${CLUSTER_NAME}" \
--region "${AWS_REGION}"

Clean up kubectl context:

kubectl config delete-context \
"connectgateway_${PROJECT_ID}_global_${CLUSTER_NAME}"

14. Reference​

Key Module Variables​

VariableTypeDefaultDescription
project_idstringβ€”GCP project ID (required)
gcp_locationstringus-central1GCP region for Fleet membership
aws_regionstringus-west-2AWS region for EKS cluster
cluster_name_prefixstringeks-clusterResource name prefix
k8s_versionstring1.34Kubernetes version for EKS
platform_versionstring1.34.0-gke.1GKE Connect Agent platform version
min_sizenumber2Node group minimum nodes
desired_sizenumber2Node group desired nodes
max_sizenumber5Node group maximum nodes
trusted_userslist(string)[]Google identities granted cluster-admin
aws_access_keystringβ€”AWS IAM Access Key ID (required)
aws_secret_keystringβ€”AWS IAM Secret Access Key (required)
vpc_cidrstring10.0.0.0/16VPC CIDR block
public_subnetsbooltrueCreate public subnets with IGW

IAM Roles Created by the Module​

AWS:

RolePurpose
eks-cluster-role-<id>EKS control plane (trust: eks.amazonaws.com)
eks-node-group-role-<id>EC2 worker nodes (trust: ec2.amazonaws.com)

GCP APIs Enabled:

APIPurpose
gkemulticloud.googleapis.comGKE Attached Clusters management
gkeconnect.googleapis.comConnect Agent
connectgateway.googleapis.comConnect Gateway kubectl proxy
anthos.googleapis.comAnthos/Fleet platform
logging.googleapis.comCloud Logging
monitoring.googleapis.comCloud Monitoring
gkehub.googleapis.comFleet Hub
opsconfigmonitoring.googleapis.comManaged Prometheus
kubernetesmetadata.googleapis.comKubernetes metadata

Useful Commands Reference​

# List fleet memberships
gcloud container fleet memberships list --project="${PROJECT_ID}"

# Configure kubectl via Connect Gateway
gcloud container fleet memberships get-credentials <cluster-name> --project="${PROJECT_ID}"

# Query EKS cluster info
aws eks describe-cluster --name <cluster-name> --region "${AWS_REGION}"

# List available platform versions
gcloud container attached get-server-config --location="${GCP_REGION}" --project="${PROJECT_ID}"

# Top nodes
kubectl top nodes

# Audit Connect Gateway access
gcloud logging read "protoPayload.serviceName=connectgateway.googleapis.com" --project="${PROJECT_ID}"

Further Reading​