Decrease memory usage of the CMR by removing ConcurrentHashMaps in indexing

Description

Migrated from NovaTec Jira. Original ticket: INSPECTIT-545

As a part of INSPECTIT-428, a custom leafs for the indexing of storage tree have been made. These leafs do not use concurrent map to hold the references to objects, but some kind of custom Hashtable, where key is primitive value.

This way we gain much less memory consumption, because the size of the tree by my calculations should be 50% less, and that's a lot. On the other hand, we will lose some querying speed because of the synchronization. Concurrent map divided the elements in segments, thus, at the same time write to two different segments was possible for example.

When I say custom Hashtable, i mean that we either need to create our own implementations, or use the primitive collection libraries that are available like Colt (http://acs.lbl.gov/software/colt/), Trove (http://trove4j.sourceforge.net/), FastUtil (http://fastutil.dsi.unimi.it/) or Cliff Click's Highly Scalable Libraries (http://sourceforge.net/projects/high-scale-lib/).

Environment

None

Activity

Show:
Stefan Siegl
July 16, 2015, 2:30 PM

2012-02-10T14:38:22.000+0100 by Ivan Senic

As agreed on the status meeting. Create a factory for generation of the concurrent maps, use properties to set up the Java one or the FastUtil.

Stefan Siegl
July 16, 2015, 2:30 PM

2012-02-14T16:55:40.000+0100 by Ivan Senic

I increased the estimate, because I'll have to also spend some time on size calculation of this maps.

Stefan Siegl
July 16, 2015, 2:30 PM

2014-11-17T10:25:19.000+0100 by Ivan Senic

Cliff's license:

Ivan Senic
July 27, 2015, 11:54 AM
Technical User
August 7, 2015, 6:19 AM

SUCCESS: Integrated in

inspectIT - Integration #51
INSPECTIT-1941: Using NonBlockingHashMapLong (high-scale-lib) in the (patrice.bouillet: c8f4e95049fe0d6fb979ed32233507b152e5d1e0)

  • CMR/src/info/novatec/inspectit/cmr/cache/AbstractObjectSizes.java

  • CommonsCS/META-INF/MANIFEST.MF

  • CommonsCS/resources/ivy/ivy.xml

  • CommonsCS/src/info/novatec/inspectit/indexing/buffer/impl/Leaf.java

  • Commons/src/info/novatec/inspectit/cmr/cache/IObjectSizes.java

  • CMR/test/info/novatec/inspectit/cmr/cache/impl/MemoryCalculationTest.java

  • THIRDPARTYLICENSE.txt

  • inspectIT/inspectIT.product

Fixed

Assignee

Ivan Senic

Reporter

Stefan Siegl

Labels

None

Integrator

Patrice Bouillet

Components

Sprint

None

Fix versions

Priority

High