Java – Can the JVM max heap size be dynamic

javajvmmemory

The JVM -Xmx argument lets one set the max heap size for the JVM to some value. But, is there a way to make that value dynamic? In other words, I want to tell the JVM "look, if you need it, just keep taking RAM from the system until the system is out."

Two-part reason for asking:
First, the app in question can use a really wide range of ram depending on what the user is doing, so the conceptual min and max values are pretty far apart. Second, it would seem that the JVM reserves the max heap space from virtual memory at boot time. This particular app is run on a pretty wide variety of hardware, so picking a "one-size-fits-all" max heap space is hard since it has to be low enough to run on low-end hardware, but we'd really like to be able to take advantage of really beefy machines if they're available.

Best Answer

But, is there a way to make that value dynamic?

Literally, no. The max heap size is set at JVM launch time and cannot be increased.

In practice, you could just set the max heap size to as large as your platform will allow, and let the JVM grow the heap as it needs. There is an obvious risk in doing this; i.e. that your application will use all of the memory and cause the user's machine to grind to a halt. But that risk is implicit in your question.

EDIT

It is worth noting that there are various -XX... GC tuning options that allow you to tweak the way that the JVM expands the heap (up to the maximum).

Another possibility is to split your application into 2 parts. The first part of the application does all of the preparation necessary to determine the "size" of the problem. Then it works out an appropriate max heap size, and launches the memory hungry second part of the application in a new JVM.

This only works if the application can sensibly be partitioned as above.
This only works if it is possible to compute the problem size. In some cases, computing the problem size is tantamount to computing the result.
It is not clear that you will get better overall performance than if you just let the heap grow up to a maximum size.

Related Solutions

Java – How to deal with “java.lang.OutOfMemoryError: Java heap space” error

Ultimately you always have a finite max of heap to use no matter what platform you are running on. In Windows 32 bit this is around 2GB (not specifically heap but total amount of memory per process). It just happens that Java chooses to make the default smaller (presumably so that the programmer can't create programs that have runaway memory allocation without running into this problem and having to examine exactly what they are doing).

So this given there are several approaches you could take to either determine what amount of memory you need or to reduce the amount of memory you are using. One common mistake with garbage collected languages such as Java or C# is to keep around references to objects that you no longer are using, or allocating many objects when you could reuse them instead. As long as objects have a reference to them they will continue to use heap space as the garbage collector will not delete them.

In this case you can use a Java memory profiler to determine what methods in your program are allocating large number of objects and then determine if there is a way to make sure they are no longer referenced, or to not allocate them in the first place. One option which I have used in the past is "JMP" http://www.khelekore.org/jmp/.

If you determine that you are allocating these objects for a reason and you need to keep around references (depending on what you are doing this might be the case), you will just need to increase the max heap size when you start the program. However, once you do the memory profiling and understand how your objects are getting allocated you should have a better idea about how much memory you need.

In general if you can't guarantee that your program will run in some finite amount of memory (perhaps depending on input size) you will always run into this problem. Only after exhausting all of this will you need to look into caching objects out to disk etc. At this point you should have a very good reason to say "I need Xgb of memory" for something and you can't work around it by improving your algorithms or memory allocation patterns. Generally this will only usually be the case for algorithms operating on large datasets (like a database or some scientific analysis program) and then techniques like caching and memory mapped IO become useful.

How to determine the size of the array in C

Executive summary:

int a[17];
size_t n = sizeof(a)/sizeof(a[0]);

Full answer:

To determine the size of your array in bytes, you can use the sizeof operator:

int a[17];
size_t n = sizeof(a);

On my computer, ints are 4 bytes long, so n is 68.

To determine the number of elements in the array, we can divide the total size of the array by the size of the array element. You could do this with the type, like this:

int a[17];
size_t n = sizeof(a) / sizeof(int);

and get the proper answer (68 / 4 = 17), but if the type of a changed you would have a nasty bug if you forgot to change the sizeof(int) as well.

So the preferred divisor is sizeof(a[0]) or the equivalent sizeof(*a), the size of the first element of the array.

int a[17];
size_t n = sizeof(a) / sizeof(a[0]);

Another advantage is that you can now easily parameterize the array name in a macro and get:

#define NELEMS(x)  (sizeof(x) / sizeof((x)[0]))

int a[17];
size_t n = NELEMS(a);

Best Answer

Related Solutions

Java – How to deal with “java.lang.OutOfMemoryError: Java heap space” error

How to determine the size of the array in C

Related Topic