Shell scripts that audit Kubernetes RBAC and workload security (PSA labels, pod security context, capabilities). Optional Terraform under terraform/ can create a small lab cluster and drop in test workloads.
Use for labs and learning only — not a replacement for cloud IAM reviews, admission controllers, or org policy.
Testing status: Validated end-to-end on local Kubernetes (Vanilla), AKS, GKE, EKS, and OpenShift CRC (scripts + Terraform lab fixtures where applicable). Not yet tested on managed OpenShift ARO or ROSA (optional Check 20 / cloud flags only).
OpenShift-related names:
| Acronym | Meaning |
|---|---|
| CRC | CodeReady Containers (also branded OpenShift Local) — single-node OpenShift for a laptop/desktop lab. |
| OKD | Origin Kubernetes Distribution — community/open-source OpenShift upstream; some CRC bundles can run OKD instead of Red Hat OpenShift. |
| ARO | Azure Red Hat OpenShift — managed OpenShift on Azure. |
| ROSA | Red Hat OpenShift Service on AWS — managed OpenShift on AWS. |
| You have… | Do this |
|---|---|
kubectl already works (kubectl get ns OK) |
Run the audits with Vanilla-* scripts (or cloud-named scripts if the cluster is on that cloud). |
| No cluster — want Azure / AWS / GCP lab | Deploy with Terraform, then run the audits. |
| Kind / minikube / home cluster, need test pods only | terraform/local-vanilla/ applies fixtures to your kubeconfig (no new cluster). |
| OpenShift CRC already running, need test fixtures | terraform/openshift-crc/ applies RBAC + capability fixtures (SCC grants included); then run OpenShift scripts. |
| Tool | When |
|---|---|
| bash, kubectl, jq | Almost every script |
| terraform | Only if you use terraform/* |
| aws | EKS kubeconfig + EKS Terraform |
| gcloud | GKE kubeconfig + GKE RBAC Check 2 |
| az | AKS kubeconfig + AKS RBAC Check 2 |
| oc | OpenShift scripts only |
Cluster access: Your kubeconfig must be able to list namespaces and read RBAC + pods. Scripts stop early if kubectl get ns fails.
OpenShift: use oc instead of kubectl; OpenShift-ContainerCapabilities.sh needs ./jq-linux-amd64 in the repo root. If CRC runs on Windows Hyper-V and you use Kali in WSL2, see OpenShift CRC from Kali WSL.
Each folder under terraform/ is independent. Always run Terraform from that folder, not the repo root.
- Log in to the cloud (see cloud login below).
- Copy settings:
cp terraform.tfvars.example terraform.tfvarsand edit (region, names,project_idon GKE). - Install infra:
terraform init→terraform plan→terraform apply(often 10–20+ minutes for a new cluster). - Connect kubectl: run the command from
terraform output(see after apply). - Run scripts from the repo root.
- Tear down when done:
terraform destroy(stops cloud charges).
terraform.tfvars is gitignored. By default, RBAC + capability test fixtures are deployed (including intentional “bad” examples like pods/exec and escape-chain pods).
terraform output run_rbac_auditonly prints a copy/paste recipe; it does not run commands for you.
| Cloud | Before terraform apply |
|---|---|
| Azure (AKS) | az login → az account show (switch with az account set --subscription <id> if needed). If creates fail with MFA / RequestDisallowedByAzure, use a service principal instead. |
| AWS (EKS) | aws configure or AWS_PROFILE=... |
| GCP (GKE) | gcloud auth login and gcloud auth application-default login; enable billing + Container API on your project |
Examples in this repo’s terraform.tfvars.example files often default to US regions. Pick a region close to you (latency, data residency, quota). Your physical location does not auto-configure anything — you set it in terraform.tfvars.
| Cloud | Terraform variable | What to set | UK examples (common lab choice) |
|---|---|---|---|
| Azure AKS | location |
Azure region code | uksouth (London), ukwest |
| AWS EKS | aws_region |
AWS region code | eu-west-2 (London), eu-west-1 (Ireland) |
| GCP GKE | zone (+ region) |
Zonal cluster: zone must belong to region |
region = "europe-west2", zone = "europe-west2-a" (London) |
Do not use display-only geography labels (e.g. Azure’s uk row from az account list-locations) as location — use deployable codes like uksouth.
Check the region supports Kubernetes before apply:
# AKS
az account list-locations -o table | grep -i uk
az aks get-versions --location uksouth -o table
# EKS (replace region)
aws ec2 describe-regions --region-names eu-west-2 --query 'Regions[0].RegionName' --output text
aws eks describe-cluster-versions --region eu-west-2 --query 'clusterVersions[0:5]' --output table 2>/dev/null || true
# GKE
gcloud compute regions list --filter="name:europe-west2"
gcloud container get-server-config --zone europe-west2-a --format='yaml(validMasterVersions)' 2>/dev/null | headAfter apply, kubeconfig and Check 2 env must match the same region/location you chose (e.g. aws eks update-kubeconfig --region eu-west-2, GKE_LOCATION = your cluster zone).
Repo default if you change nothing: AKS eastus, EKS us-east-1, GKE us-central1 / us-central1-a — fine for a lab, but not required.
AKS (terraform/aks/):
location = "uksouth" # UK South; or ukwest, eastus, westeurope, …
resource_group_name = "kube-rbac-polp-aks-test-rg"
cluster_name = "kube-rbac-polp-aks-test"
vm_size = "Standard_B2s"EKS (terraform/eks/):
aws_region = "eu-west-2" # UK (London); or eu-west-1, us-east-1, …
cluster_name = "kube-rbac-polp-eks-test"GKE (terraform/gke/) — project_id is required; keep region and zone aligned:
project_id = "my-gcp-project-id"
region = "europe-west2"
zone = "europe-west2-a" # UK-adjacent; or us-central1 + us-central1-a, …
cluster_name = "kube-rbac-polp-gke-test"More knobs (node size, K8s version, fixture flags) live in each stack’s variables.tf.
From the stack directory (e.g. terraform/aks/):
# 1) kubeconfig — use the output name for your stack:
eval "$(terraform output -raw configure_kubectl)" # AKS, EKS
# GKE uses:
# eval "$(terraform output -raw get_credentials)"
kubectl get ns
# 2) Optional env vars for cloud RBAC “Check 2” (see below)
# AKS:
export AKS_RESOURCE_GROUP="$(terraform output -raw resource_group_name)"
export AKS_CLUSTER_NAME="$(terraform output -raw cluster_name)"
# GKE:
# export GKE_PROJECT="$(terraform output -raw project_id)"
# export GKE_CLUSTER_NAME="$(terraform output -raw cluster_name)"
# export GKE_LOCATION="$(terraform output -raw zone)"
# EKS: Check 2 uses kubectl aws-auth + aws Access Entries (cluster/region from kubeconfig ARN,
# or export EKS_CLUSTER_NAME / AWS_REGION)
# 3) Audits (from repo root)
cd ../..
./AKS-rbac.sh --quiet
./AKS-ContainerCapabilities.sh --output textSwap AKS-* for EKS-* or GKE-* on other clouds.
Fixtures-only (existing cluster):
cd terraform/local-vanilla
terraform init && terraform apply
cd ../..
./Vanilla-RBAC.sh --quiet
./Vanilla-ContainerCapabilities.sh --output textOpenShift CRC fixtures (cluster already running; apply as kubeadmin — developer is not enough for SCC bindings). From Kali/WSL, API reachability must work first — see OpenShift CRC from Kali WSL.
# Password from Windows: crc console --credentials
oc login https://api.crc.testing:6443 -u kubeadmin -p '<password>' --insecure-skip-tls-verify
cd terraform/openshift-crc
cp terraform.tfvars.example terraform.tfvars # optional
terraform init && terraform apply
cd ../..
./OpenShift-RBAC.sh --quiet
./OpenShift-ContainerCapabilities.sh --only-user-ns --output text| Directory | Creates cluster? | Scripts to run after |
|---|---|---|
terraform/aks/ |
Yes (AKS) | AKS-rbac.sh, AKS-ContainerCapabilities.sh |
terraform/eks/ |
Yes (EKS + VPC) | EKS-rbac.sh, EKS-ContainerCapabilities.sh |
terraform/gke/ |
Yes (GKE) | GKE-rbac.sh, GKE-ContainerCapabilities.sh |
terraform/local-vanilla/ |
No | Vanilla-RBAC.sh, Vanilla-ContainerCapabilities.sh |
terraform/openshift-crc/ |
No (existing CRC) | OpenShift-RBAC.sh, OpenShift-ContainerCapabilities.sh |
Useful outputs (all clouds): configure_kubectl or get_credentials, capability_test_namespace, rbac_test_namespace, run_rbac_audit, run_container_capabilities_audit.
Run from the repository root. Confirm context first:
kubectl config current-context
kubectl get ns| Your cluster | RBAC | Capabilities / PSA |
|---|---|---|
| Generic / local / kind | Vanilla-RBAC.sh |
Vanilla-ContainerCapabilities.sh |
| Amazon EKS | EKS-rbac.sh |
EKS-ContainerCapabilities.sh |
| Google GKE | GKE-rbac.sh |
GKE-ContainerCapabilities.sh |
| Azure AKS | AKS-rbac.sh |
AKS-ContainerCapabilities.sh |
| Red Hat OpenShift | OpenShift-RBAC.sh (oc) |
OpenShift-ContainerCapabilities.sh |
RBAC scripts answer: who can do dangerous things? (cluster-admin, secrets, exec, wildcards, token minting, etc.). High-risk checks may show an Attack path: line under Checking:.
Capabilities scripts answer: how risky are pods on paper? (privileged, caps, host namespaces, PSA labels). They read declared spec, not live enforcement. Escape chains (e.g. caps + hostNetwork) print as separate Escape chain: lines — they do not change the main Status severity.
On EKS / GKE / AKS, known managed-plane pods (aws-node / kube-proxy, GKE netd, AKS CNI, etc.) still report CRITICAL/HIGH when the spec warrants it, but Status and escape-chain lines are tagged [known … platform component — expected on managed nodes] (annotate, do not hide). Other findings in system namespaces get a milder [platform namespace workload …] tag. Use --only-user-ns to focus on app/lab namespaces.
chmod +x *.sh # once
./Vanilla-RBAC.sh --help
./Vanilla-RBAC.sh --quiet # less noise; good first run
./Vanilla-RBAC.sh --list-checks
./Vanilla-ContainerCapabilities.sh --output text
./Vanilla-ContainerCapabilities.sh --only-user-ns --output jsonSame flags work on EKS-rbac.sh, GKE-rbac.sh, AKS-rbac.sh, OpenShift-RBAC.sh (--checks, --quiet, --list-checks, --help, …).
Cloud RBAC Check 2 (extra cloud IAM slice — optional but needs env on GKE/AKS):
| Cloud | Extra env before RBAC script |
|---|---|
| EKS | (usually none) — aws-auth via kubectl; Access Entries via aws (cluster/region from kubeconfig ARN or EKS_CLUSTER_NAME + AWS_REGION) |
| GKE | GKE_PROJECT (project IAM; cluster name/location not required for Check 2) |
| AKS | AKS_RESOURCE_GROUP, AKS_CLUSTER_NAME |
Without those exports, most RBAC checks still run; Check 2 is skipped or thin on GKE/AKS.
Manual kubeconfig (no Terraform):
aws eks update-kubeconfig --name <cluster> --region <aws_region> # EKS — region must match cluster
gcloud container clusters get-credentials <cluster> --location <zone-or-region> --project <proj> # GKE
az aks get-credentials --resource-group <rg> --name <cluster> # AKS| Script | Purpose |
|---|---|
check_subject_access.sh |
RBAC risk for one User / Group / ServiceAccount |
ControlPlane_WorkerNodes_fromAllPods.sh |
Probe node ports from pods; optional --check-imds-creds |
network_segreg_via_endpoints.sh |
Namespace → service endpoint reachability sweep |
See each script’s header for flags.
- Run
kubectl config current-context— is it the lab cluster, not an old kind/minikube context? - Re-fetch credentials after apply:
- AKS/EKS:
eval "$(terraform output -raw configure_kubectl)"from the correctterraform/<stack>/directory - GKE:
eval "$(terraform output -raw get_credentials)"
- AKS/EKS:
- Cloud login expired:
az login,aws sts get-caller-identity, orgcloud auth list. - EKS:
aws eks update-kubeconfig --name … --region …must use the sameaws_regionas Terraform. - AKS: wrong subscription →
az account set --subscription <id>then get-credentials again.
Symptom: az login as your user works, and reads often work (az group list, az account show), but terraform apply / resource create, update, or delete fails with something like:
RequestDisallowedByAzure: … must authenticate through MFA …
(or policy text pointing at aka.ms/MFAforAzure).
What’s going on
Microsoft is enforcing MFA for Azure resource management (ARM Phase 2) on user identities. Your CLI session can still get an ARM token that only proves password login (amr: ["pwd"] — no mfa). Reads may succeed; writes are denied.
On personal Microsoft accounts (@outlook.com / @live.com) without Entra ID P1/P2, you usually cannot use Conditional Access to force MFA at az login. Entra also may not re-prompt for MFA for a long window (“remember MFA” / session lifetime). Clearing ~/.azure and logging in again often still yields a password-only token. The CLI --claims-challenge path only helps if the error returns a challenge blob (many deny errors do not).
Workload identities are exempt: service principals and managed identities are not subject to this user-MFA rule. For a personal / free-tier lab, a service principal with Contributor on the subscription is the practical workaround.
Workaround: service principal for Terraform
- Portal → correct directory (the one that owns your subscription — check Directory (tenant) ID on the subscription Overview).
- Microsoft Entra ID → App registrations → New registration → create an app.
- Certificates & secrets → New client secret → copy the secret Value immediately (not the Secret ID). You cannot view the Value again later.
- Subscriptions → your sub → Access control (IAM) → Add role assignment → Contributor → assign to that app (search by name or Application ID).
- From the app Overview, note:
- Application (client) ID →
ARM_CLIENT_ID(a GUID) - Directory (tenant) ID →
ARM_TENANT_ID - Secret Value →
ARM_CLIENT_SECRET(not Secret ID, not Object ID)
- Application (client) ID →
- In the same shell you will run Terraform:
export ARM_CLIENT_ID="<Application (client) ID>"
export ARM_CLIENT_SECRET="<client secret Value>"
export ARM_TENANT_ID="<Directory (tenant) ID>"
export ARM_SUBSCRIPTION_ID="<subscription GUID>"
az login --service-principal \
-u "$ARM_CLIENT_ID" \
-p "$ARM_CLIENT_SECRET" \
--tenant "$ARM_TENANT_ID"
az account set --subscription "$ARM_SUBSCRIPTION_ID"
cd terraform/aks
terraform init
terraform applyThe AzureRM provider also reads ARM_* directly; keep them exported for the whole Terraform run — including terraform destroy (destroy is also a write and hits the same MFA gate).
Common mistakes
| You used… | Result |
|---|---|
Secret Value as -u / ARM_CLIENT_ID |
AADSTS700016 (app not found) |
| Secret ID as client ID or secret | Login fails or wrong credential |
| App Object ID as client ID | AADSTS700016 |
| App created in a different Entra directory than the subscription | AADSTS700016 for that tenant |
Optional “proper” user-login fix: Entra ID P1/P2 + Conditional Access requiring MFA for Windows Azure Service Management API / Azure Resource Manager, so az login itself gets an MFA-stamped token. Without P1/P2, that path is unavailable (“insufficient licensing”).
Cleanup after the lab: terraform destroy, then delete the client secret (or the app registration) and remove the Contributor assignment. Treat the secret like a powerful password.
Verify a user token has MFA (optional; needs az login as a user):
python3 - <<'PY'
import json, base64, subprocess
tok = json.loads(subprocess.check_output(
["az", "account", "get-access-token",
"--resource", "https://management.azure.com/", "-o", "json"]))["accessToken"]
pad = tok.split(".")[1] + "=" * (-len(tok.split(".")[1]) % 4)
claims = json.loads(base64.urlsafe_b64decode(pad))
print("amr:", claims.get("amr"), "mfa:", "mfa" in (claims.get("amr") or []))
PYIf mfa is False, user-based Terraform creates will keep failing under Phase 2 enforcement.
Symptom: plan runs many minutes with little output, or after Ctrl+C you see
Encountered an error whilst ensuring Resource Providers are registered / Microsoft.EventHub / context canceled.
Cause: Azure provider 4.x may try to register many resource providers in your subscription on first run. That is slow and can fail if you lack permission or cancel mid-way.
Fix (this repo): terraform/aks/main.tf sets resource_provider_registrations = "core" (not "all") so first plan is much faster.
Then:
cd terraform/aks
rm -f .terraform.tfstate.lock.info
terraform planIf registration still fails, disable auto-registration and register AKS providers yourself:
# in provider "azurerm" { … }
resource_provider_registrations = "none"az provider register --namespace Microsoft.ContainerService --wait
az provider register --namespace Microsoft.Compute --wait
az provider register --namespace Microsoft.Network --wait
terraform plan| Symptom | Things to try |
|---|---|
| Insufficient quota (vCPUs, AKS/EKS limits) | Cloud portal → Quotas; try another region; reduce node_count / smaller VM (Standard_B2s, t3.small). |
| SKU not available in region | Change vm_size / node_instance_type / machine_type; try paired region (e.g. ukwest if uksouth fails). |
| AKS version error | Leave kubernetes_version unset (null = latest in region) or pick from az aks get-versions --location <location>. |
| GKE “project not found” / API disabled | Set correct project_id; enable Kubernetes Engine API; billing enabled. |
| Wrong directory | Run Terraform only inside terraform/aks, terraform/eks, or terraform/gke — not repo root. |
Microsoft’s InstallAzureCLIDeb script does not recognize kali-rolling. Options:
- Recommended on Kali:
sudo apt update && sudo apt install -y azure-cli - Or:
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo DIST_CODE=bookworm bash SyntaxWarningspam frompython3-azext-devopsduring install is harmless;az versionshould still work.
Symptom: oc login https://api.crc.testing:6443 works from Windows PowerShell, but from Kali WSL you get:
error: dial tcp 127.0.0.1:6443: connect: connection refused
Why /etc/hosts alone is not enough
CRC adds api.crc.testing → 127.0.0.1 in the Windows hosts file, and the API listens only on Windows localhost (127.0.0.1:6443), not on 0.0.0.0 or the WSL virtual adapter.
In WSL2, 127.0.0.1 is the Linux VM itself — not Windows — so that mapping always fails from Kali.
Pointing Kali’s /etc/hosts at the Windows side of the WSL link (often 172.x.x.1 from grep nameserver /etc/resolv.conf) only fixes name resolution. The TCP connect still targets that 172.x address on Windows, where nothing is listening on port 6443 (CRC is bound to 127.0.0.1 only). You still get connection refused until something forwards 172.x:6443 → 127.0.0.1:6443.
Fix on Windows 10 (port proxy) — run in Admin PowerShell. Replace 172.17.112.1 with your current WSL gateway if different:
$gw = (Get-NetIPAddress -AddressFamily IPv4 |
Where-Object { $_.InterfaceAlias -eq 'vEthernet (WSL)' }).IPAddress
netsh interface portproxy add v4tov4 listenaddress=$gw listenport=6443 connectaddress=127.0.0.1 connectport=6443
netsh interface portproxy add v4tov4 listenaddress=$gw listenport=443 connectaddress=127.0.0.1 connectport=443
netsh interface portproxy add v4tov4 listenaddress=$gw listenport=80 connectaddress=127.0.0.1 connectport=80
New-NetFirewallRule -DisplayName 'CRC from WSL' -Direction Inbound -Protocol TCP -LocalPort 80,443,6443 -Action AllowThen on Kali, map CRC names to that same gateway IP (not 127.0.0.1):
GW=$(grep -m1 nameserver /etc/resolv.conf | awk '{print $2}')
# If a CRC line already exists, change its IP; otherwise append one:
sudo sed -i -E "s/^[0-9.]+[[:space:]]+(api\.crc\.testing)/${GW}\t\1/" /etc/hosts
# Or edit /etc/hosts so the CRC hostnames use $GWLogin:
oc login https://api.crc.testing:6443 -u developer -p developer --insecure-skip-tls-verify
# Admin (password from: crc console --credentials on Windows):
# oc login https://api.crc.testing:6443 -u kubeadmin -p '<password>' --insecure-skip-tls-verifyAfter reboot: the WSL gateway IP can change. Re-run the portproxy commands with the new $gw, and update Kali /etc/hosts to match.
Windows 11 alternative: WSL mirrored networking (networkingMode=mirrored in %UserProfile%\.wslconfig, then wsl --shutdown) shares localhost with Windows so CRC’s 127.0.0.1 mapping works as-is. That mode is not available on Windows 10 (e.g. build 19045); use the port-proxy path above.
Check 2 needs cloud env vars in the same shell as the script. Run terraform output from the stack directory (e.g. terraform/aks), not the repo root — otherwise the exports are empty and Check 2 prints “not run”.
# AKS (from terraform/aks after apply; az must still be logged in — user or service principal)
export AKS_RESOURCE_GROUP="$(terraform output -raw resource_group_name)"
export AKS_CLUSTER_NAME="$(terraform output -raw cluster_name)"
cd ../.. # repo root
./AKS-rbac.sh --checks=2
# GKE Check 2 only needs the project (from terraform/gke)
export GKE_PROJECT="$(terraform output -raw project_id)"
# Optional helpers for get-credentials / docs:
# export GKE_CLUSTER_NAME="$(terraform output -raw cluster_name)"
# export GKE_LOCATION="$(terraform output -raw zone)"If AKS Check 2 says it could not list role assignments: confirm az account show works and the identity has at least Reader on the subscription/cluster. This repo uses az role assignment list --scope … --include-inherited (Azure CLI 2.88+ rejects combining --scope with --all).
Also install gke-gcloud-auth-plugin (sudo apt install google-cloud-cli-gke-gcloud-auth-plugin when gcloud is from apt) so Terraform’s Kubernetes provider and kubectl can authenticate to GKE.
EKS Check 2: aws-auth needs only kubectl. Access Entries need the AWS CLI (same creds as the cluster); cluster/region are taken from the kubeconfig context ARN when possible, else:
export EKS_CLUSTER_NAME="$(terraform output -raw cluster_name)"
export AWS_REGION="$(terraform output -raw aws_region)"- Confirm fixtures were applied:
terraform applywithdeploy_capability_test_workloads = trueanddeploy_rbac_test_fixtures = true(defaults). - Check outputs:
terraform output capability_test_namespaceandrbac_test_namespace. - Wait for pods:
kubectl get pods -A | grep -E 'cap-test|rbac'.
That usually means only platform roles matched and were filtered out — not necessarily “all clear.” Re-run without --quiet once to see skipped system roles, or use --checks=N for one check.
Note the exact command, cloud + region from terraform.tfvars, and the last 20 lines of error. Use ./<Script>.sh --help and the script header for check IDs.
Not a full cloud IAM audit — only a small, documented slice:
- EKS:
aws-auth→system:masters, plus Access Entries withAmazonEKSClusterAdminPolicy/AmazonEKSAdminPolicy - GKE: project IAM (
Owner/Editor/container.clusterAdmin/container.adminon the GCP project; GKE has no cluster-resourceget-iam-policy) - AKS: Azure role assignments on the AKS cluster resource (including inherited subscription/RG roles via
--include-inherited)
Cloud RBAC scripts do not hide vendor break-glass paths. They tag them so triage can separate platform noise from your lab/app findings:
| Cloud | Tagged in output |
|---|---|
| AKS | ClusterRole/aks-service, Users aks-support / clusterAdmin / clusterUser; Check 2 Azure RBAC hits; cluster-admin |
| EKS | Check 2 aws-auth → system:masters and/or Access Entry AmazonEKSClusterAdminPolicy; Check 1 system:masters; cluster-admin (no AKS-style aks-service role) |
| GKE | Check 2 GCP project Owner / Editor / container.clusterAdmin / container.admin; Check 1 system:masters; cluster-admin |
Look for the suffix [known … platform break-glass] (or [known Kubernetes break-glass ClusterRole]). These remain listed on purpose.
Optional identity_baseline.conf hides noisy subject names (e.g. platform SAs) in many listings. Override with IDENTITY_BASELINE_FILE. Not applied to cluster-admin subject listings (by design).
Heuristic only: estimated effective caps from pod spec + conditions (hostPath, hostNetwork, …). Output: text Escape chain: lines, JSON escape_chains, CSV column escape_chains.
- RBAC ≠ complete Azure/AWS/GCP IAM.
- Capabilities ≠ admission controller or runtime truth.
- Terraform labs use small node pools and permissive test namespaces — destroy when finished.
- For every check ID and nuance:
./<script> --help,--list-checks, and the script header comments.