Integration Tests

• move to inspectIT Confluence

• Test System
• Analysis Tool
• Test Szenario
• Troubles during the load test
• Test 1 - Baseline Test
• Test 2 - Version 1.7.5.88
• Test 3 - Version 1.7.5.88 Disruptor

Important: The CMR was also monitored with VisualVM during these integration tests. We experienced a really huge different CPU-/Memory Usage on the CMR whereas the current agent queue implementation caused a big number of element dropping on the CMR. This issue is covered on page CMR Investigations during the Integration Tests

Agent overhead was one goal of the new agent queue realization. The overhead should not be worse than the overhead from the current implementation. Therefore, the following integration tests have been done to compare the two implementations by means of their overhead performance. The goal is not to tell at the end concrete figures like the new realizations needs 0.2 % less CPU ressource but 1% more memory. Instead, the comparison should reveal if the new agent queue realization performs well enough to be a replacement for the current implementation.

Test System

Analysis Tool

Java VisualVM Version 1.8.0_77 (Build 140910)

Test Szenario

The JBoss Demo Application Ticket Monster (running on JBoss EAP 6.3) was used for the testing purposes. Gatling was used to generate load (alpha-simulation TicketMonster.scala was provided by Marius Oehler).

In total, three integration tests will be executed. The first test will be run without the inspectIT agent. This test will be used as baseline test and is used to identify the overhead of the current agent queue realization. The second and third test will be run with the agent queue realization of pre-release version 1.7.5.88 respectively the disruptor implementation based on the pre release version 1.7.5.88

Each test consists of the following phases:

• Idle Phase: After JBoss startup, no load is generated for ~ 5 minutes, only platform sensors are activated. This phase shows how the demo application behaves with no load. High resource utilization within this phase reveal performance overhead of the agent in no / low load szenario
• Load Phase: The Gatling loadtest will be run for 30 minutes. 300 Users will be ramped up in the first 5 minutes. After 30 minutes, the users will ramped down again.
• Idle Phase: After the load test, no load is generated for ~ 5 minutes. After the loadtest, resource utilization should behave similar than during the first idle phase.

Troubles during the load test

All 3 tests suffered a problem in creating and deleting bookings. This caused a lot of exception and might have reduced the constant load on the system. Nevertheless, the three tests should be comparable as all tests experienced the same issues.

Test 1 - Baseline Test

Below are screenshots from the resource consumption of the TicketMonster JVM. After the Idle Phase the load increases over 5 minutes. After that the utilization of the CPU and memory are relatively stable. CPU utilization is roughly around 7-8 % during the load phase.

...

Test 2 - Version 1.7.5.88

Below are screenshots from the resource consumption of the TicketMonster JVM with the inspectIT agent 1.7.5.88. After the Idle Phase the load increases over 5 minutes. After that the utilization of the CPU and memory are relatively stable. CPU utlization is roughly around 13-14 percent during the load phase.

...

Test 3 - Version 1.7.5.88 Disruptor

Below are screenshots from the resource consumption of the TicketMonster JVM with the inspectIT agent 1.7.5.88 disruptor version. After the Idle Phase the load increases over 5 minutes. After that the utilization of the CPU and memory are relatively stable.  CPU utlization is roughly around 12-13 percent during the load phase.

...