GC settings and buffer memory sizing

Due to the bad garbage collection performance in some cases, it was decided to change the settings for the garbage collection and to change the way how the initial buffer size is calculated.

Memory pols and GC settings

The first thing was to define a new ratio of the old and new generation space in heap memory. It was concluded that the default values for the memory pools depend if the CMR is run on the 32bit or 64bit machine. The settings are following:

The new GC settings included the definition that only after the 80% of the old generation space is full, garbage collection is activated. This percentage is very important for the buffer memory size calculation, because the buffer size can not be above 80% of old generation. Otherwise, it could happen that the garbage collection is always active. JVM parameters added for this purpose are:

XX:CMSInitiatingOccupancyFraction=80 -XX:+UseCMSInitiatingOccupancyOnly

Buffer memory sizing

After the changes of heap memory pool sizes and GC settings, it was clear that the calculation of the buffer size can not reside only on the total heap size, but has to be based on the size of old generation memory pool. It was decided that the buffer will, by default, have the size between 50-65% of old generation space, where bigger old generation size will enable bigger occupancy percentage for the buffer.

These default settings can be changed in config/buffer.properties file:

# Size of old space occupancy till which min occupancy will be active - 384MB
buffer.minOldSpaceOccupancyActiveTillOldGenSize = 402653184

# Size of old space occupancy from which max occupancy will be active - 2GB
buffer.maxOldSpaceOccupancyActiveFromOldGenSize = 2147483648

# percentage of the min old generation heap space buffer is occupying
buffer.minOldSpaceOccupancy = 0.5

# percentage of the max old generation heap space buffer is occupying
buffer.maxOldSpaceOccupancy = 0.65

Additional info

Additional info about heap size influence to the buffer size can also be found on public wiki page: Working with the in-memory storage.