OrientDB's available memory estimate ignores cgroup (container) limits #7812
Description
OrientDB Version: 2.2.29
Java Version: OpenJDK 8
OS: Linux
When calculating the available system memory to use as a disk cache, OrientDB currently uses the formula total system memory - JVM heap size - 2GB.
This ignores the fact that OrientDB may be running on, say, a 64GB RAM system, but inside a docker container that is only allowed to use 10GB maximum. This means that OrientDB will keep using more and more memory, thinking there's plenty of capacity, until it gets terminated by the operating system's OOM killer for exceeding its allocated limit.
As a workaround until this issue can be fixed, if you run OrientDB in a container, I recommend setting the -XX:MaxDirectMemorySize option in the ORIENTDB_OPTS_MEMORY environment variable. Keep in mind that the direct memory consumption is on top of the heap and stack memory, so leave some breathing room between the sum of these three values and the container memory limit.
Expected behavior
OrientDB should be aware of any cgroup limits that might apply to it. Other projects have had similar issues in the past, and have developed fixes. OpenBLAS and YARN are two examples.
Actual behavior
OrientDB is not aware of the cgroup limits, and keeps allocating memory until it gets killed by the OS.
Steps to reproduce
- Get a 64GB machine / instance / VM.
- Start an OrientDB docker container with the
--memoryoption set to significantly less than 64GB, e.g. 6GB. - Put load on the database until OrientDB gets killed by the operating system for using too much RAM.