Container Security Best Practices: From Image to Runtime
Containers ship fast, but speed without security is recklessness. A single vulnerable base image can propagate across hundreds of pods in seconds. This guide covers the full container security lifecycle — from building hardened images to detecting anomalous behavior at runtime.
The Container Threat Model#
Before diving into tools, understand what you are defending against:
- Vulnerable dependencies — known CVEs in OS packages or application libraries.
- Image tampering — a compromised registry serves malicious layers.
- Container escape — an attacker breaks out of the container namespace into the host.
- Lateral movement — one compromised pod reaches other services via the network.
- Privilege escalation — a container running as root gains host-level access.
Build ──► Ship ──► Run
│ │ │
│ │ └─ Runtime detection (Falco)
│ └────────── Registry scanning, signing
└──────────────────── Image scanning, minimal base
Image Scanning#
Trivy#
Trivy is an open-source scanner from Aqua Security that checks container images, filesystems, and IaC configurations for vulnerabilities and misconfigurations.
# Scan a local image
trivy image myapp:latest
# Fail CI if critical or high CVEs are found
trivy image --severity CRITICAL,HIGH --exit-code 1 myapp:latest
# Scan a filesystem (useful in CI before building the image)
trivy fs --security-checks vuln,secret .
Integrate Trivy into your CI pipeline so that no image ships without a scan. Most teams gate on CRITICAL and HIGH severity, allowing MEDIUM and LOW to be tracked and remediated on a cadence.
Snyk Container#
Snyk provides commercial-grade scanning with a developer-friendly UI, fix recommendations, and base image upgrade suggestions.
# Authenticate
snyk auth
# Test an image
snyk container test myapp:latest
# Monitor continuously
snyk container monitor myapp:latest
Snyk excels at suggesting the smallest base image upgrade that eliminates the most CVEs — a feature that saves hours of manual research.
Scanning Strategy#
| Stage | Tool | Action |
|---|---|---|
| Local dev | Trivy CLI | Scan before push |
| CI pipeline | Trivy or Snyk | Gate on severity |
| Registry | Harbor / ECR scanning | Scan on push |
| Production | Continuous monitoring | Alert on new CVEs |
Minimal Base Images#
The fewer packages in your image, the smaller your attack surface.
Distroless Images#
Google's distroless images contain only your application and its runtime dependencies — no shell, no package manager, no coreutils.
# Build stage
FROM golang:1.22 AS builder
WORKDIR /app
COPY . .
RUN CGO_ENABLED=0 go build -o server .
# Runtime stage — distroless
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /app/server /server
ENTRYPOINT ["/server"]
Alpine-Based Images#
When you need a shell for debugging, Alpine Linux provides a minimal footprint (~5 MB) with a package manager.
FROM node:22-alpine
RUN apk add --no-cache tini
USER node
ENTRYPOINT ["tini", "--"]
CMD ["node", "server.js"]
Scratch Images#
For statically linked binaries (Go, Rust), scratch provides a zero-dependency base.
FROM scratch
COPY --from=builder /app/server /server
ENTRYPOINT ["/server"]
Non-Root Containers#
Running as root inside a container means a container escape grants root on the host. Always run as a non-root user.
# Create a non-root user
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
# Own application files
COPY --chown=appuser:appgroup . /app
# Switch to non-root
USER appuser
Verify at runtime:
# Should NOT print "root"
docker run --rm myapp:latest whoami
Seccomp and AppArmor#
Seccomp Profiles#
Seccomp (Secure Computing Mode) restricts which system calls a container can make. The default Docker seccomp profile blocks ~44 dangerous syscalls including mount, reboot, and kexec_load.
For tighter security, create a custom profile:
{
"defaultAction": "SCMP_ACT_ERRNO",
"architectures": ["SCMP_ARCH_X86_64"],
"syscalls": [
{
"names": ["read", "write", "open", "close", "stat", "fstat",
"mmap", "mprotect", "munmap", "brk", "exit_group"],
"action": "SCMP_ACT_ALLOW"
}
]
}
Apply it in Kubernetes:
securityContext:
seccompProfile:
type: Localhost
localhostProfile: profiles/custom-profile.json
AppArmor Profiles#
AppArmor provides mandatory access control on file paths, capabilities, and network access. Most Kubernetes nodes ship with a default AppArmor profile.
metadata:
annotations:
container.apparmor.security.beta.kubernetes.io/mycontainer: runtime/default
For custom profiles, load them on each node and reference by name.
Network Policies#
By default, every pod in a Kubernetes cluster can communicate with every other pod. Network policies restrict traffic to only what is necessary.
Default Deny#
Start by denying all ingress and egress, then explicitly allow what each service needs.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: production
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
Allow Specific Traffic#
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-frontend-to-api
namespace: production
spec:
podSelector:
matchLabels:
app: api
ingress:
- from:
- podSelector:
matchLabels:
app: frontend
ports:
- protocol: TCP
port: 8080
Runtime Security with Falco#
Falco is a CNCF runtime security tool that detects anomalous behavior inside containers using eBPF or kernel modules.
What Falco Detects#
- Shell spawned inside a container
- Unexpected network connections
- Sensitive file reads (
/etc/shadow,/etc/passwd) - Privilege escalation attempts
- Binary execution not part of the original image
Example Rules#
- rule: Terminal Shell in Container
desc: Detect a shell being spawned in a container
condition: >
spawned_process and container and
proc.name in (bash, sh, zsh, dash)
output: >
Shell spawned in container
(user=%user.name container=%container.name
shell=%proc.name parent=%proc.pname)
priority: WARNING
Deployment#
# Install via Helm
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm install falco falcosecurity/falco \
--namespace falco --create-namespace \
--set driver.kind=ebpf
Route Falco alerts to your SIEM or incident management tool (PagerDuty, Opsgenie) for immediate response.
Pod Security Standards#
Kubernetes Pod Security Standards (PSS) define three levels of restriction:
| Level | Description |
|---|---|
| Privileged | No restrictions — for system-level workloads |
| Baseline | Prevents known privilege escalations |
| Restricted | Hardened — requires non-root, read-only root filesystem, dropped capabilities |
Enforcing with Pod Security Admission#
apiVersion: v1
kind: Namespace
metadata:
name: production
labels:
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/warn: restricted
Restricted Pod Example#
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
seccompProfile:
type: RuntimeDefault
containers:
- name: app
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
Supply Chain Security#
Image Signing with Cosign#
Sign images at build time and verify before deployment.
# Generate a key pair
cosign generate-key-pair
# Sign an image
cosign sign --key cosign.key myregistry/myapp:latest
# Verify before deploy
cosign verify --key cosign.pub myregistry/myapp:latest
SBOM Generation#
A Software Bill of Materials (SBOM) catalogs every component in your image.
# Generate SBOM with Syft
syft myapp:latest -o spdx-json > sbom.json
# Attach SBOM to image with Cosign
cosign attach sbom --sbom sbom.json myregistry/myapp:latest
Admission Controllers#
Use admission controllers to enforce policies at deploy time:
- Kyverno or OPA Gatekeeper — reject pods that use unsigned images, run as root, or pull from untrusted registries.
# Kyverno policy: require image signatures
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: verify-image-signatures
spec:
rules:
- name: verify-cosign
match:
resources:
kinds: ["Pod"]
verifyImages:
- imageReferences: ["myregistry/*"]
attestors:
- entries:
- keys:
publicKeys: |-
-----BEGIN PUBLIC KEY-----
...
-----END PUBLIC KEY-----
Security Checklist#
- Scan every image in CI — gate on CRITICAL and HIGH CVEs.
- Use minimal base images (distroless, Alpine, or scratch).
- Never run containers as root.
- Apply seccomp and AppArmor profiles.
- Enforce default-deny network policies.
- Deploy Falco for runtime anomaly detection.
- Enforce Pod Security Standards at the namespace level.
- Sign images and verify signatures before deployment.
- Generate and attach SBOMs to every release.
- Rotate secrets and credentials — never bake them into images.
If this guide helped, explore the rest of the engineering blog at codelit.io/blog to keep leveling up.
Try these templates
Continue learning
Go deeper on infrastructure and delivery
As an Amazon Associate I earn from qualifying purchases. Codelit may receive a commission at no extra cost to you.
The DevOps Handbook
Gene Kim, Jez Humble · 2021
Case studies from Etsy, Amazon, and Target on rewiring deployment pipelines and org structure.
Site Reliability Engineering
Betsy Beyer, Niall Richard Murphy · 2016
Google's on-call postmortems, error-budget math, and staffing models across 550 pages of practice.
4.7 (1.2k)#1 in Network Disaster & Recovery AdministrationKubernetes: Up and Running
Brendan Burns, Joe Beda · 2022
Builds from a single Pod to multi-cluster deployments with YAML you type, not diagrams you skim.
4.6 (118)#10 in Web ServicesInvestments Unlimited
Helen Beal, Bill Bensing, et al. · 2022
A 160-page story of a bank automating audit evidence, showing what regulators actually ask for.
4.4 (163)Kindle edition