Resource limits and health checks

On a shared machine, one runaway process is all it takes. A queue worker stuck in a loop or a Node process leaking memory can drag every other project down with it. The Resources section of a project is where you draw the line.

Limits

Limits come in two groups. Application containers are the web server, PHP, Node.js, Python, Go, Ruby, Java, .NET and every worker; services are the database, Redis, Memcached, Mailpit, RabbitMQ, the search engines and object storage. For each group you set a CPU limit in cores (say 1.5) and a memory limit, in GiB unless you switch the unit next to the field to MiB.

One thing is easy to overlook: Docker limits containers one by one, so the numbers apply to each container in the group, not to the group as a whole. A project with PHP, Node.js and two workers may use four times the application limit in total.

There’s also a limit on processes per container, 4096 by default. It stops fork bombs and worker pools that keep spawning new children, and it applies to every container even if you set nothing else. Leave a field empty and there’s no limit - that’s how every project starts. OpenSearch needs at least 1.5 GiB, so don’t go below that there. For a Java project I’d give the application containers about 2 GiB: in development the build tool’s daemon, the application JVM and a test run each want their share, and a Gradle test run under 1 GiB can take the dev server down with it.

Changes reach running containers right away. The one exception is removing a CPU or memory limit altogether: Docker can’t lift a limit in place, so the affected containers get recreated. The card shows you each running container’s CPU and memory next to its limit. Java is the odd one out here: a JVM sizes its heap once when it starts, so after you lower a limit on a running project restart it, or the kernel kills the JVM instead of Java throwing an OutOfMemoryError.

When a container hits its memory limit, the kernel ends a process inside it. If that’s the main process, the container restarts; it can also be a child, like a php-fpm worker. Either way, Envoryx puts a warning on the project for a day (“php ran out of memory at 14:03 (limit 512 MiB)”) and sends you a notification. Usually that’s within seconds. Some Docker hosts don’t report such a kill as an event; there Envoryx finds it in the container’s cgroup v2 counter at its next reading, so the warning can take up to a minute.

Limits are part of envoryx.yml as well (limits:), see environment and envoryx.yml.

Resource history

Once a minute, Envoryx notes what every running project container uses: CPU, memory, network traffic and disk I/O. Once an hour it also measures how much disk space each project takes up - volumes, project folder and backups.

The Resources section draws all of that for anything from the last hour to the last year, with a table view for every chart if you prefer numbers. The dashboard lists every project with its average and peak CPU and memory, its disk space and a small CPU sparkline, busiest first. That’s the quickest way to find out which project is eating the server. How long the values are kept you set under Settings → Retention → Resource history: 7, 30 or 90 days (the default), or a year.

Health checks

A running container doesn’t mean a working application. So in a project’s Overview section you can set up a health check: a path like /health, or Laravel’s /up, that has to answer with the status you expect.

Envoryx asks the web server the same way the proxy does, with the project’s host name, so your application sees an ordinary visitor. Redirects aren’t followed, which matters: a /health that redirects to /login counts as a failure.

By default the check runs every 30 seconds (anything from 10 seconds to an hour works) and waits 5 seconds for each answer. After 3 failures in a row the application counts as down. You get one notification when it goes down and one when it’s back. Test runs the check once, even before you save it. While the project is stopped, while an operation is running on it or while its application container isn’t running, the check takes a break.

A good health path checks what your application actually depends on - database, cache, queue - and still answers quickly.