Defeating the Linux OOM Killer for Good

There is nothing more terrifying to a systems administrator than a quiet server that suddenly stops responding. You log in, run dmesg, and find the chilling words: Out of memory: Kill process. The Linux Out-Of-Memory (OOM Killer) has struck again, terminating your database or critical web server just as traffic was spiking.

We decided to solve this once and for all in ODAC. The latest release introduces a fully autonomous host-swap manager written in Go. This service dynamically provisions, mounts, and reclaims virtual memory space as real-world load shifts. It keeps your self-hosted cloud running smoothly without requiring manual configuration.

The Silent Killer in Your Server Room

Managing physical memory on developer servers has historically been a manual and error-prone process. If an application suddenly spikes in memory usage, the kernel faces a binary choice: either crash the process or let the whole system freeze. Usually, it chooses the former, and your critical processes are summarily executed.

Traditional solutions tell you to create a static swap partition or allocate a fixed swapfile during initial server provisioning. However, this is suboptimal for modern, dense workloads. Static swap either wastes valuable disk space when idle or falls short when massive parallel builds or heavy database migrations execute.

Furthermore, configuring swap requires running low-level shell commands, calculating block sizes, and editing fstab files. For a zero-config platform like ODAC, this manual approach is completely unacceptable. Developers should focus on building products, not tuning virtual memory parameters.

Inside ODAC's Autonomous Swap Manager

The new host-swap manager (located under internal/system/swap) eliminates configuration entirely. Instead of making crude guesses based on idle memory statistics, ODAC collects Linux Pressure Stall Information (PSI) directly from /proc/pressure/memory.

PSI is the absolute gold standard for measuring resource starvation. It reports the exact percentage of time that processes are stalled waiting for memory. ODAC couples this with honest, non-reclaimable anonymous RAM calculations:

$\text{Memory Used} = \text{MemTotal} - \text{MemAvailable}$

By tracking MemAvailable instead of raw free memory, ODAC avoids counting safe, reclaimable page caches as active memory pressure. This ensures that the swap manager only reacts when your server is genuinely starving.

The system operates on a dedicated decision loop (defined in decide.go) that executes every 30 seconds. This check gate prevents the manager from over-reacting to short, transient memory spikes. To initiate a swap expansion, the system must observe sustained pressure across a validation streak. This deliberate pacing keeps your server extremely stable.

The Dynamic Lifecycle: Grow, Shrink, and Adopt

When memory pressure exceeds defined limits, ODAC's actuator (defined in actuator.go) swings into action. It executes a clean, low-level system pipeline to provision additional headroom:

1. Capacity Planning: The decision engine calculates the exact size of the new increment. The default step is half of your total physical RAM, capped between 1 GiB and 16 GiB. It also checks your disk budget, ensuring that ODAC swap files never consume more than 25% of your available disk space.
2. On-the-fly Allocation: ODAC creates a new swap file directly on host-backed storage under /swapfile.odac.N, where N is a sequential index.
3. Activation: The manager formats the file using mkswap and immediately mounts it into the active kernel pool via swapon.

This entire lifecycle is fully integrated. If you deploy an application with one click from the cloud dashboard at app.odac.run, or trigger it via the terminal with odac app create, you never have to worry about running out of memory. Behind the scenes, the swap manager will automatically scale your virtual memory to accommodate the build pipeline.

Once the spike subsides and memory pages are freed, the shrink cycle safely unmounts the swapfile via swapoff and unlinks it from the disk. This ensures your server reclaims its storage and remains lean.

Adopt-Under-Boot Resilience

A common failure mode of dynamic swap managers is rebooting. If a server restarts, any dynamically created swapfiles are lost to the kernel, leaving the host operating on physical RAM alone. This is a massive risk if a high-pressure spike occurs immediately after boot before the daemon can react.

ODAC solves this with adopt-under-boot resilience. During initialization, the swap manager scans the swap directory for any existing /swapfile.odac.N files left over from previous runs. It automatically reconciles, validates, and activates them before any other application starts.

This provides a robust defense-in-depth security model. The host is never left vulnerable, and your applications enjoy enterprise-grade uptime from the very first second the server boots.

Zero Config, Maximum Peace of Mind

Let's look at the difference this makes for your production environment:

• Before: A sudden, unexpected traffic spike exhausted your server's RAM. The Linux kernel's OOM killer instantly terminated your database, bringing down your entire application and requiring manual ssh intervention.
• After: The traffic spike arrives. ODAC detects the rising pressure stall signals, confirms the sustained load, and mounts a fresh 2 GiB swap increment in seconds. Your database continues serving requests, and once the traffic subsides, the swapfile is cleanly deleted.

This is the power of a modern, autonomous cloud platform. By offloading low-level systems engineering to ODAC's smart control plane, you get the absolute highest levels of reliability without any of the operational burden.