Kubernetes pod gets OOMKilled because JVM heap plus metaspace exceeds the container memory limit

2026-11-14 · 3 min read

Kubernetes pod gets OOMKilled because JVM heap plus metaspace exceeds the container memory limit

The pod restarts every few minutes with exit code 137, the CrashLoopBackOff counter climbs, and the JVM logs contain no OutOfMemoryError. Nobody changed the code. Someone set -Xmx to exactly the container limit, which leaves zero room for everything the JVM allocates outside the heap. That is what a kubernetes OOMKilled JVM heap memory limit failure looks like in production, and it costs you a restart loop plus paged on-call time until someone does the arithmetic.

The kubernetes OOMKilled JVM heap memory limit math

Exit code 137 is 128 + 9: the kernel sent SIGKILL to the container's main process. The cgroup memory limit applies to every byte that process and its children touch. The Java heap is one line item in that budget. Metaspace (class metadata and the classloader graph), the JIT code cache, thread stacks, GC bookkeeping, direct ByteBuffer allocations, and JNI buffers all live inside the same limit. [link needed]

A heap sized to 100% of the limit is therefore guaranteed to be killed, usually during startup or at the first large class-loading burst.

Since JDK 10, UseContainerSupport is on by default, so an unset max heap defaults to a fraction of the container limit rather than the node's RAM. An explicit -Xmx overrides that safety net. [link needed]

Measure before you change anything

1. Confirm the kill reason: kubectl describe pod should show Reason: OOMKilled on the last terminated state. A liveness-probe kill also reports 137, so check the reason, not just the code.

2. Pull a peak, not a snapshot: query container_memory_working_set_bytes for that container over at least 24 hours of real traffic and record the maximum.

3. Enable -XX:NativeMemoryTracking=summary in staging and run jcmd 1 VM.native_memory summary. The committed total is your real footprint.

4. Read the components you can't see in a heap dump: jcmd 1 VM.metaspace, the GC log's live-set numbers, and jcmd 1 Thread.print | grep -c '^"' for the thread count.

5. If the working set is much larger than NMT reports, suspect direct buffers or native libraries and profile allocations before touching heap flags.

Set the flags so the total fits

Budget every region explicitly, then verify the sum in staging before rollout:

Re-run the NMT summary. If the committed total sits with comfortable headroom under the limit, ship it. If it doesn't, the limit is wrong too — raise it rather than squeezing heap to a number the application can't run in.

When NOT to do this

Skip the tuning if the pod is being killed by a failing liveness probe rather than the OOM killer, if the working set never approaches the limit, or if this is a JVM-less sidecar in the same pod. Also skip it as a fix for a genuine leak: a larger -Xmx delays the restart, it doesn't remove it. If the growth is entirely off-heap in direct buffers, heap flags are the wrong lever.

Checklist