Explorative Testing v1.8

As the part of the release process it was agreed that prior to release one complete day is reserved for explorative testing that should prove that we don't have any critical bugs. In addition this testing should make us aware of some minor bugs that are always present. These minor bugs should be documented on JIRA and planned for next version.

The steps that are described on this page were used for explorative testing for version 1.5. Please update this page and add additional steps for versions to come.

Explorative testing should perform same steps on both Linux and Windows machines. It's in most cases duplication of work, but it happens often that bug is related to only one platform, especially for the User interface.

Steps

Perform these steps in chronological order (grouping is just for better understanding). Add the person who tests the functionality, if any vulnerability was found and the JIRA ticket if it exists:

TaskIvan Senic (Ubuntu)Marius Oehler (Win 7)Tobias Angerstein (Xubuntu)FoundTicket


Installers

Use installers to install inspectIT(tick)
(tick)

Select all components and creation of all shortcuts(tick)
(tick)

Test if shortcuts work(tick)
(tick)





UI


Welcome screen

Test all internal links(tick)
(tick)We should update the contact links. INSPECTIT-2538 - Getting issue details... STATUS
Test all external links(warning)
(warning)
  1. Links are OK, but Eclipse browser can not correctly display the new Confluence pages. However, nothing we can do. Might work in Win.

2. The mailto - link does not work properly


Prove search works(tick)
(error)

No results are displayed:


Main menuClick on every item in the main menu, prove all works as expected(tick)
(tick)


Repository Manager

Add repository(tick)
(tick)

Edit repository(tick)
(tick)

Remove repository(error)
(error)Got a NPE when removing the repository. (Tobias Angerstein +1) INSPECTIT-2539 - Getting issue details... STATUS
Double-click on offline repository(tick)
(tick)

CMRStart CMR(error)
(tick)DiagnoseIT jar is missing thus the classes can not be found. INSPECTIT-2535 - Getting issue details... STATUS
UIRepository ManagerRestart CMR(tick)
(tick)

Shutdown CMR(tick)
(tick)

CMRStart CMR again (as service in Win)-
-



UI

Label ManagementAdd new label type(tick)
(tick)

Add new label of existing type(tick)
(tick)

Remove label of existing type(tick)
(tick)

Prove it's saved correctly(tick)
(tick)

Start recording, limit time(tick)
(tick)

Stress CMR for 10-15 mins (best with Invoker sample application)(tick)
(tick)

UIStop recording manually(tick)
(tick)

Check recording storage(tick)
(warning)

Got an NullPointer while exploring the new storage. (Could not reproduce)


Start one more additional sample app (Spring Petclinic inspectIT)(tick)
(tick)

Shutdown CMR, wait a bit, start CMR(tick)
(warning)

Updating CMR properties failed ...


Verify agent is reconnected(tick)
(tick)

Restart one or more agents(tick)
(tick)

















UI

Copy buffer to storage with agent filter(tick)
(tick)

Open both agents in the Data explorer(tick)
(tick)

Make sure agent tree structure saving works as agents are switched(tick)
(tick)



Platform sensors view

Check every view(warning)
(tick)All views have Maximize active sub-view enabled in the tool-bar, when clicked it leads to displaying the InfluxDB is enabled message (although influxDB is not active at all). INSPECTIT-2540 - Getting issue details... STATUS
Test time-frame(tick)
(tick)

Test sampling rate(tick)
(tick)

Check data below graph(tick)
(tick)

Play around with Invocation browser(tick)
(tick)








Invocation sequence

Show All

Filters - all types(tick)
(tick)

Show hide columns(tick)
(tick)

Minimize / maximize view(tick)
(tick)

Change property like refresh time, close UI and check changes are remembered on restart(tick)
(tick)

Sorting(tick)
(tick)

Searching(tick)
(tick)



Call hierarchy

Expand / collapse(tick)
(tick)

Locate in Hierarchy (multiple items)(tick)
(tick)

Copy(tick)
(tick)

Navigation (multiple items)(tick)
(tick)

Details(tick)
(tick)

Parameters on context capturing (tick)




SQL / Method / Exceptions

Test raw / aggregated(tick)
(tick)

Locate in Hierarchy (one raw item, one aggregated)(tick)
(tick)

Navigation(tick)
(tick)

Copy(tick)
(tick)

SQL statistics(error)

Statistics not displayed. INSPECTIT-2541 - Getting issue details... STATUS
Combine data (select as much as possible invocations)(tick)
(tick)



SQL view

Double-click both views(tick)
(tick)

Decimal places(tick)
(tick)

Navigation(tick)
(tick)

Sorting(tick)
(tick)

Other menu options(tick)
(tick)

Save SQLs to Server with no invocations included(tick)
(tick)


Timer Data view

Display in chart(tick)
Option was not visible

Navigation(tick)
(tick)

Sorting(tick)
(tick)

Save Timers to Server with invocations(tick)
(tick)



HTTP


URI

Split method(tick)



URI RegexN/A



Display in chart(tick)
(tick)

Navigation(tick)
(tick)

Tag

Display in chart




Navigation





Exceptions

(Single and Grouped)

Double clicks(tick)
(tick)

Check stack traces(tick)
(tick)

Details(tick)
(tick)

Navigation(tick)
(tick)

Tracing















Overview





Sort(tick)
(tick)

Copy(tick)
(tick)

Search(tick)
(tick)

Details(tick)
(tick)

Save to Storage (full trace must be saved)(tick)
(tick)

Open details (double-click)(tick)
(tick)

Open details (navigate-to)(tick)
(tick)

Trace (top)



Search(tick)
(tick)

Copy complete(tick)
(tick)

Details(tick)
(tick)

Save to Storage (full trace must be saved)(tick)



Locate span using double-click(tick)



Trace (bottom)



Locate in hierarchy (SQL, Method)(tick)



Search call hierarchy(tick)
(tick)

Details(tick)
(tick)

Navigate from sub-views(warning)

Navigation includes agent information even in the aggregated SQL mode. Not sure if we can do anything until agent-less views.
Copy complete call hierarchy(tick)
(tick)

Rename few editors(tick)
(tick)

Delete an agent from Repository(tick)
(tick)









Storage

Add new(tick)
(tick)

Edit(tick)
(tick)

Delete (test both finalized and not-finalized one)(tick)
(tick)

Filters and grouping(tick)
(error)

StorageManager → ShowProperties causes an exception, when no Storage is focused.

INSPECTIT-2545 - Getting issue details... STATUS
Finalize open storage(tick)
(tick)

Add all available label types to one storage (boolean, date, number and text)(tick)
(tick)

Test changing of all label values in the table(tick)
(tick)

Remove label from storage(tick)
(tick)

Delete label that exists in storage from Label management(tick)
(tick)

Download storageCompressed(tick)
(tick)

Not-compressed(tick)
(tick)

Upload storage(tick)
(tick)

Export(tick)
(tick)

ImportLocal(tick)
(tick)

To repository(tick)
(tick)

ToolbarTest all actions(tick)
(tick)

Connect to influxDB(tick)
(tick)

Prove data is written in influxDB (Grafana or influx command)(tick)
(tick)

Prove views on the UI that can not display data that is in influx show an info banner(tick)
(tick)

Configuration Interface




















































Switch between profiles / environments(tick)
(tick)

Environments



Create(tick)
(tick)

Edit(tick)
(tick)

Export(tick)
(tick)

Delete(tick)
(tick)

Import(tick)
(tick)

Environment editor








Filter profiles(tick)
(tick)

Select / deselect profiles(tick)
(tick)

Open profile (double click)(tick)
(tick)

Open profile (menu item)(tick)
(tick)

Change strategies(tick)
(tick)

Deselect some platform sensors(tick)
(tick)

Change sensor options(tick)
(tick)

Save & close(tick)
(tick)

Reload(tick)
(tick)

Edit (F2)(tick)
(tick)

Profiles






Create different profile types(tick)
(warning)

Two errors occurred, when I tried to create a new profile, by using the right-click menu in the instrumentation manager. Unfortunately I could not reproduce the problem


Edit(tick)
(tick)

Export(tick)
(tick)

Delete(tick)
(tick)

Import(tick)
(tick)

Set as Default (create new Environment to prove)(tick)
(tick)


Deactivate (confirm by Dynamic instrumentation)(tick)
(tick)

Duplicate(tick)
(tick)

Sensor profile











Add sensor type(tick)
(tick)

Show / hide advances sensors(tick)
(tick)

Save not possible when not correct input(tick)
(tick)

One wildcard definition(tick)
(tick)

One interface/super-class definition(tick)
(tick)

One method with parameters (test add/remove/edit parameter)(tick)
(tick)

One with custom modifiers(tick)



One with context capturing (test add/remove/edit capturing)(tick)
(tick)

Remove assignment (mouse + DEL)(tick)
(tick)DEL is not working. Not creating a ticket for that. (Tobias Angerstein: works for me Ivan Senic)
Duplicate assignment(tick)
(tick)

Add second sensor type(tick)
(tick)

Save(tick)
(warning)Saving a new Profile, which contains a wildcard leads to a timeout ... However it is shown as saved in the UI
Reopen(tick)
(tick)

Exclude rule profile



Add exclude rule(tick)
(tick)

Edit exclude ruleN/A
not available

Close (with save)(tick)
(tick)

Reopen(tick)
(tick)

Remove exclude rule(tick)
(tick)

JMX profile


Add one assignment(tick)
(tick)

Set specific attributes(tick)
(tick)

Close (with save)(tick)
(tick)

Reopen(tick)
(tick)

Mappings





Deactivate mapping(tick)
(tick)

Activate mapping(tick)
(tick)

Change mapping to another environment (don't update with DI)(tick)
(tick)

Add mapping(tick)
(tick)

Save(tick)
(warning)

Same here while updating the agent mapping, lead to a timeout ... (The added profile contained a full-instrumentation) However it is shown as saved in the UI (got also the following exception on the agent:13:05:26,657 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.TcpConnection.readObject(TcpConnection.java:211)
13:05:26,657 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.Client.update(Client.java:305)
13:05:26,657 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.Client.run(Client.java:421)
13:05:26,657 ERROR [STDERR] at rocks.inspectit.agent.java.connection.impl.AgentAwareClient.run(AgentAwareClient.java:47)
13:05:26,657 ERROR [STDERR] at java.lang.Thread.run(Thread.java:745)
13:05:26,657 ERROR [STDERR] Caused by: com.esotericsoftware.kryo.KryoException: Buffer underflow.
Serialization trace:
timeStamp (rocks.inspectit.shared.all.communication.data.CpuInformationData)
updatedInstrumentationDefinitions (rocks.inspectit.shared.all.communication.message.UpdatedInstrumentationMessage)
result (rocks.inspectit.shared.all.kryonet.rmi.ObjectSpace$InvokeMethodResult)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.io.Input.require(Input.java:181)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.io.Input.readAscii_slow(Input.java:598)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.io.Input.readAscii(Input.java:576)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.io.Input.readString(Input.java:454)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:132)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:99)
13:05:26,657 ERROR [STDERR] at rocks.inspectit.shared.all.serializer.impl.CustomCompatibleFieldSerializer.read(CustomCompatibleFieldSerializer.java:207)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:116)
13:05:26,657 INFO [STDOUT] 2017-10-13 13:05:26,657: [inspectIT] 5686399 [ruptor-thread-0] WARN va.stats.AgentStatisticsLogger - Monitoring data is dropped due to buffer capacity reached or connection failure. Current count of dropped data is 11.
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:116)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
13:05:26,658 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
13:05:26,658 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
13:05:26,658 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.ExtendedSerializationImpl.read(ExtendedSerializationImpl.java:163)
13:05:26,658 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.TcpConnection.readObject(TcpConnection.java:209)
)


Reopen(tick)
(tick)

Remove mapping(tick)
(tick)

Use instrumentation dialog from Repository manager to refresh agent(tick)
(tick)

Business Context























Application










Create application(tick)
(tick)

Move application(tick)
(tick)

Add several matching rules(tick)
(tick)

Switch to advanced mode(tick)
(tick)

Add some AND & OR operator(tick)
(tick)

Add some more rules into ADD and OR(tick)
(tick)

Save & Close(tick)
(tick)

Reopen(tick)
(tick)

Delete added AND & OR(tick)
(tick)

Switch back to simple mode(tick)
(tick)

Edit (F2)(warning)
(tick)Editing application from the table discards the non saved changes in the editor.
Save(tick)
(tick)

Business transaction










Add BTs (test insert before option)(tick)
(tick)

Move BTs(tick)
(tick)

Delete BTs(tick)
(tick)

Add several matching rules(tick)
(tick)

Add some AND & OR operator(tick)
(tick)

Add some more rules into ADD and OR(tick)
(tick)

Save & Close(tick)
(tick)

Reopen(tick)
(tick)

Enable extract name dynamically (parameter value SearchForm:SearchQuery for DVD store)(tick)
(tick)

Test BTs on incoming data(tick)
(tick)

Edit (F2)(tick)
(tick)

Save & Close(warning)
(tick)Changing values in the dynamic extraction part does not get saved correctly. Furthermore it's also not displayed correctly when switched between the BTs. (Tobias Angerstein could not reproduce)
Delete created application(tick)
(tick)

Alerting





Connect to email service (NT-VPN host: smtp or your google account)(tick)
(tick)

Alert




Create(tick)



Use suggestions from influxDB(tick)



Add mail address(tick)



Test with performance problems (Slow Search)(tick)



Use alert ID in the UI (toolbar)(tick)



Create one with lower bound




Delete(tick)



End User MonitoringConfigurationActivate EUM in the environment(tick)



Check JS agent is delivered(tick)



Check scripts properties are same as EUM configuration (relevancy threshold, insturment listeners, etc)(tick)



Add all modules (plugins)(tick)



Check agent JS link has included modules(tick)



TracingCheck correlation between client and server page load requests(tick)



Check resource loading is connect(tick)



Perform other Tracing tasks from the table above if not already(tick)



influxDBMake sure data EUM data is written to influxDB(tick)



Verify dashboard is working(tick)



DataBrowser / OS(tick)



Listeners(tick)



Resources(tick)



Page requests(tick)



Speed index(tick)



DiagnosisConfigurationActivate diagnosis(tick)



Adapt baseline if needed(tick)



Create some slow requests(warning)

CMR log contains errors. INSPECTIT-2543 - Getting issue details... STATUS
influxDBMake sure data is written to influx(tick)



Verify baseline is respected(tick)



Verify dashboard is working(tick)

Bottom table of the dashboard should be more clear. We should add trace and problem context durations as well.
Turn off influxDB and make sure diagnosis will stop(tick)



Remote technologiesApache HTTP client (latest)(tick)



Async Apache HTTP client (latest)(tick)



Spring Rest Template (latest)(tick)



Jetty Http Client (8.latest)(tick)



Url Connection (java 8)(tick)



Url Connection (java 9)(error) (tick)

Agent can not be initialized with Java 9. Spring failure. Works with Java 9 fix. INSPECTIT-2544 - Getting issue details... STATUS
Executors (java 8)(tick)



Executors (java 9)(error) (tick)

Agent can not be initialized with Java 9. Spring failure. Works with Java 9 fix. INSPECTIT-2544 - Getting issue details... STATUS
JMS(tick)



Manual remote server sensor(tick)



























RESTful API














CMRhttp://localhost:8182/rest/cmr/version(tick)



http://localhost:8182/rest/cmr/status(error)

Error on the end-point.

INSPECTIT-2542 - Getting issue details... STATUS

Agents

http://localhost:8182/rest/data/agents (tick)



http://localhost:8182/rest/data/agents/{id}/methods(tick)



http://localhost:8182/rest/data/agents/{id}/sensors(tick)



Business Contexthttp://localhost:8182/rest/bc/app(tick)



http://localhost:8182/rest/bc/app/{id}/btx(tick)



Business Context Definition


http://localhost:8182/rest/bc/definition/app(tick)



http://localhost:8182/rest/bc/definition/app/{id}(tick)



http://localhost:8182/rest/bc/definition/app/{id}/btx(tick)



http://localhost:8182/rest/bc/definition/app/{id}/btx/{id}(tick)



Alertinghttp://localhost:8182/rest/alerts(tick)



http://localhost:8182/rest/alerts/closed

(tick)



Invocations

http://localhost:8182/rest/data/invocations(tick)



http://localhost:8182/rest/data/invocations?minDuration=100(tick)



http://localhost:8182/rest/data/invocations/{id}(tick)



Stop CMR service (Win only)-



UninstallersRun uninstaller(tick)



AgentPropertyViewSelect agent in repository manager

(warning)AgentPropertyView throws NullPointer - possibly caused by a race condition (I cannot reproduce the error, yet)