CPU limits, throttling and the JVM

Key points

• CPU requests and limits are enforced by different mechanisms. Requests become a scheduling weight; limits become a hard CFS quota - a ceiling per 100ms period.
• Hitting the quota does not slow the process smoothly. The kernel stops it until the next period, so latency arrives in 100ms steps. The container looks under its limit on average while being repeatedly frozen.
• This is why container_cpu_cfs_throttled_seconds_total matters more than CPU utilisation. A pod at 40% average CPU can be throttled constantly if its work is bursty.
• The JVM makes it worse. It sizes GC threads, the fork-join common pool and connection pools from availableProcessors(), so a container with a low CPU limit but many visible cores creates far more threads than it can ever run.
• Modern JVMs read the cgroup limit, but a fractional limit rounds up - a limit of 0.5 still reports one CPU, and 1.5 reports two. Setting -XX:ActiveProcessorCount explicitly removes the guesswork.
• Many teams set CPU requests and deliberately omit CPU limits for latency-sensitive services, letting them burst into idle capacity. Memory limits are different - memory is not compressible, so exceeding it means an OOM kill.

Example

resources:
  requests:
    cpu: "500m"        # scheduling weight, and the guaranteed share
    memory: "512Mi"
  limits:
    # cpu: "1000m"     # often deliberately omitted for latency-sensitive work
    memory: "1Gi"      # ALWAYS set: memory is not compressible

---
# QoS class is derived from these, and decides who is evicted first:
#
#   Guaranteed   requests == limits for every resource   evicted last
#   Burstable    requests < limits                        evicted next
#   BestEffort   nothing set                              evicted FIRST

---
# Throttling is invisible in CPU utilisation. Check it directly:
#
#   rate(container_cpu_cfs_throttled_seconds_total[5m])
#     > 0 sustained -> the pod is being frozen, whatever utilisation says

---
# The JVM inside. A fractional CPU limit rounds UP, so be explicit.
env:
  - name: JAVA_TOOL_OPTIONS
    value: >-
      -XX:MaxRAMPercentage=70
      -XX:ActiveProcessorCount=1
      -XX:+UseSerialGC

# Why: at limits.cpu = 500m the JVM still reports 1 processor, and at 1500m
# it reports 2. Without ActiveProcessorCount the JVM may start a parallel
# collector and a ForkJoinPool sized for hardware it cannot use, so it
# spends its quota context-switching.

# And MaxRAMPercentage rather than -Xmx: the percentage is read from the
# cgroup limit, so it follows the manifest instead of a stale hard-coded MB
# value that becomes wrong the moment someone edits limits.memory.

A CPU limit freezes the container every 100ms rather than slowing it - watch throttling, not utilisation, and pin ActiveProcessorCount.