A Kubernetes scheduler that forecasts memory: the same work in 16 minutes, from 37

The challenge

Kubernetes places Pods by their declared requests and limits, static numbers that describe a dynamic workload poorly. Set a cyclic application’s request at its peak and the reserved capacity sits idle for most of the cycle; set it lower and Pods risk OOM kills when applications peak together. Teams pick a conservative middle and pay for half-empty nodes, and tuning requests and limits by hand takes engineering time and goes stale as workloads change.

What was at risk:

• Paying for half-empty nodes, OOM kills when workloads peak together, and engineering time spent tuning requests and limits by hand

What we did

• Collected resource-utilization metrics with Prometheus and built analytical models that detect usage patterns and forecast them.
• Learned three kinds of workload: cyclic applications, non-cyclic workloads through time-series forecasting, and Jobs, whose Pods repeat the same resource signature across runs.
• Stored per-workload and aggregated per-node forecasts in InfluxDB, with a node evaluator scoring every node against them continuously.
• Wrote a custom Go plugin that hooks Kubernetes’ filtering and scoring through a REST API, inside the existing scheduling framework, with the scheduler itself unchanged.
• Measured it against the standard scheduler on four 16Gi worker nodes running cyclic applications and three batch Jobs of 50 Pod completions each.

How it works

Learning and scheduling run as separate parts that meet at Kubernetes’ own extension points, in five stages.

1. Observe Prometheus collects resource-utilization metrics from every workload in the cluster.
2. Learn Analytical models identify each workload’s pattern. Cyclic applications repeat, so they get an hour-by-hour forecast in place of a constant peak reservation. Non-cyclic workloads get time-series forecasting. Jobs and CronJobs run concurrent Pods with the same code, so they show a repeatable signature across runs. Workloads that peak at different times, such as one in the morning and one in the evening, can then share a node.
3. Forecast Forecasts are stored in InfluxDB per workload and aggregated per node, so the system knows how much memory each node will use in the coming period.
4. Score A node evaluator scores every node continuously against those forecasts.
5. Place When a Pod reaches the scheduler, the Go plugin joins the filtering and scoring stages through a REST API, and the standard scheduler binds the Pod to the node with forecast headroom. Static requests, limits, autoscaling and policy constraints all stay in force.

In the lab test the three Jobs had different memory shapes (flat, high-start and low-start, up to 4G per Pod). With requests sized for the peak, the standard scheduler ran one Job Pod per node at a time. The forecasting scheduler recognized both the application cycles and the Job signatures, put more Pods into the valleys and fewer at the peaks, stayed under capacity, and finished in 16 minutes, from 37.

The approach adds software components and analytics compute, and that cost grows with the number of deployments and patterns modelled, so it pays off on large clusters. A move to production goes through shadow scheduling and canary rollouts, with confidence-aware placement, drift detection and a conservative fallback.