Explorative Testing v1.8

As the part of the release process it was agreed that prior to release one complete day is reserved for explorative testing that should prove that we don't have any critical bugs. In addition this testing should make us aware of some minor bugs that are always present. These minor bugs should be documented on JIRA and planned for next version.

The steps that are described on this page were used for explorative testing for version 1.5. Please update this page and add additional steps for versions to come.

Explorative testing should perform same steps on both Linux and Windows machines. It's in most cases duplication of work, but it happens often that bug is related to only one platform, especially for the User interface.

Steps

Perform these steps in chronological order (grouping is just for better understanding). Add the person who tests the functionality, if any vulnerability was found and the JIRA ticket if it exists:

Task				Ivan Senic (Ubuntu)	Marius Oehler (Win 7)	Tobias Angerstein (Xubuntu)	Found	Ticket
Installers	Use installers to install inspectIT
	Select all components and creation of all shortcuts
	Test if shortcuts work
UI	Welcome screen	Test all internal links					We should update the contact links.	INSPECTIT-2538 - Getting issue details... STATUS
		Test all external links					Links are OK, but Eclipse browser can not correctly display the new Confluence pages. However, nothing we can do. Might work in Win. 2. The mailto - link does not work properly
		Prove search works					No results are displayed:
	Main menu	Click on every item in the main menu, prove all works as expected
	Repository Manager	Add repository
		Edit repository
		Remove repository					Got a NPE when removing the repository. (Tobias Angerstein +1)	INSPECTIT-2539 - Getting issue details... STATUS
		Double-click on offline repository
CMR	Start CMR						DiagnoseIT jar is missing thus the classes can not be found.	INSPECTIT-2535 - Getting issue details... STATUS
UI	Repository Manager	Restart CMR
UI	Repository Manager	Shutdown CMR
CMR	Start CMR again (as service in Win)			-		-
UI	Label Management	Add new label type
		Add new label of existing type
		Remove label of existing type
		Prove it's saved correctly
	Start recording, limit time
Stress CMR for 10-15 mins (best with Invoker sample application)
UI	Stop recording manually
UI	Check recording storage						Got an NullPointer while exploring the new storage. (Could not reproduce)
Start one more additional sample app (Spring Petclinic inspectIT)
Shutdown CMR, wait a bit, start CMR							Updating CMR properties failed ...
Verify agent is reconnected
Restart one or more agents
UI	Copy buffer to storage with agent filter
	Open both agents in the Data explorer
	Make sure agent tree structure saving works as agents are switched
	Platform sensors view	Check every view					All views have Maximize active sub-view enabled in the tool-bar, when clicked it leads to displaying the InfluxDB is enabled message (although influxDB is not active at all).	INSPECTIT-2540 - Getting issue details... STATUS
		Test time-frame
		Test sampling rate
		Check data below graph
	Play around with Invocation browser
	Invocation sequence Show All	Filters - all types
		Show hide columns
		Minimize / maximize view
		Change property like refresh time, close UI and check changes are remembered on restart
		Sorting
		Searching
		Call hierarchy	Expand / collapse
			Locate in Hierarchy (multiple items)
			Copy
			Navigation (multiple items)
			Details
			Parameters on context capturing
		SQL / Method / Exceptions	Test raw / aggregated
			Locate in Hierarchy (one raw item, one aggregated)
			Navigation
			Copy
			SQL statistics				Statistics not displayed.	INSPECTIT-2541 - Getting issue details... STATUS
		Combine data (select as much as possible invocations)
	SQL view	Double-click both views
		Decimal places
		Navigation
		Sorting
		Other menu options
		Save SQLs to Server with no invocations included
	Timer Data view	Display in chart				Option was not visible
		Navigation
		Sorting
		Save Timers to Server with invocations
	HTTP	URI	Split method
			URI Regex	N/A
			Display in chart
			Navigation
		Tag	Display in chart
		Tag	Navigation
	Exceptions (Single and Grouped)	Double clicks
		Check stack traces
		Details
		Navigation
	Tracing	Overview	Sort
			Copy
			Search
			Details
			Save to Storage (full trace must be saved)
			Open details (double-click)
			Open details (navigate-to)
		Trace (top)	Search
			Copy complete
			Details
			Save to Storage (full trace must be saved)
			Locate span using double-click
		Trace (bottom)	Locate in hierarchy (SQL, Method)
			Search call hierarchy
			Details
			Navigate from sub-views				Navigation includes agent information even in the aggregated SQL mode. Not sure if we can do anything until agent-less views.
			Copy complete call hierarchy
	Rename few editors
Delete an agent from Repository
Storage	Add new
	Edit
	Delete (test both finalized and not-finalized one)
	Filters and grouping					StorageManager → ShowProperties causes an exception, when no Storage is focused.	INSPECTIT-2545 - Getting issue details... STATUS
	Finalize open storage
	Add all available label types to one storage (boolean, date, number and text)
	Test changing of all label values in the table
	Remove label from storage
	Delete label that exists in storage from Label management
	Download storage	Compressed
	Download storage	Not-compressed
	Upload storage
	Export
	Import	Local
	Import	To repository
Toolbar	Test all actions
Connect to influxDB
Prove data is written in influxDB (Grafana or influx command)
Prove views on the UI that can not display data that is in influx show an info banner
Configuration Interface	Switch between profiles / environments
	Environments	Create
		Edit
		Export
		Delete
		Import
	Environment editor	Filter profiles
		Select / deselect profiles
		Open profile (double click)
		Open profile (menu item)
		Change strategies
		Deselect some platform sensors
		Change sensor options
		Save & close
		Reload
		Edit (F2)
	Profiles	Create different profile types				Two errors occurred, when I tried to create a new profile, by using the right-click menu in the instrumentation manager. Unfortunately I could not reproduce the problem
		Edit
		Export
		Delete
		Import
		Set as Default (create new Environment to prove)
		Deactivate (confirm by Dynamic instrumentation)
		Duplicate
	Sensor profile	Add sensor type
		Show / hide advances sensors
		Save not possible when not correct input
		One wildcard definition
		One interface/super-class definition
		One method with parameters (test add/remove/edit parameter)
		One with custom modifiers
		One with context capturing (test add/remove/edit capturing)
		Remove assignment (mouse + DEL)				DEL is not working. Not creating a ticket for that. (Tobias Angerstein: works for me Ivan Senic)
		Duplicate assignment
		Add second sensor type
		Save				Saving a new Profile, which contains a wildcard leads to a timeout ... However it is shown as saved in the UI
		Reopen
	Exclude rule profile	Add exclude rule
		Edit exclude rule	N/A		not available
		Close (with save)
		Reopen
		Remove exclude rule
	JMX profile	Add one assignment
		Set specific attributes
		Close (with save)
		Reopen
	Mappings	Deactivate mapping
		Activate mapping
		Change mapping to another environment (don't update with DI)
		Add mapping
		Save				Same here while updating the agent mapping, lead to a timeout ... (The added profile contained a full-instrumentation) However it is shown as saved in the UI (got also the following exception on the agent:13:05:26,657 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.TcpConnection.readObject(TcpConnection.java:211) 13:05:26,657 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.Client.update(Client.java:305) 13:05:26,657 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.Client.run(Client.java:421) 13:05:26,657 ERROR [STDERR] at rocks.inspectit.agent.java.connection.impl.AgentAwareClient.run(AgentAwareClient.java:47) 13:05:26,657 ERROR [STDERR] at java.lang.Thread.run(Thread.java:745) 13:05:26,657 ERROR [STDERR] Caused by: com.esotericsoftware.kryo.KryoException: Buffer underflow. Serialization trace: timeStamp (rocks.inspectit.shared.all.communication.data.CpuInformationData) updatedInstrumentationDefinitions (rocks.inspectit.shared.all.communication.message.UpdatedInstrumentationMessage) result (rocks.inspectit.shared.all.kryonet.rmi.ObjectSpace$InvokeMethodResult) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.io.Input.require(Input.java:181) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.io.Input.readAscii_slow(Input.java:598) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.io.Input.readAscii(Input.java:576) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.io.Input.readString(Input.java:454) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:132) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:99) 13:05:26,657 ERROR [STDERR] at rocks.inspectit.shared.all.serializer.impl.CustomCompatibleFieldSerializer.read(CustomCompatibleFieldSerializer.java:207) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:116) 13:05:26,657 INFO [STDOUT] 2017-10-13 13:05:26,657: [inspectIT] 5686399 [ruptor-thread-0] WARN va.stats.AgentStatisticsLogger - Monitoring data is dropped due to buffer capacity reached or connection failure. Current count of dropped data is 11. 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:116) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) 13:05:26,657 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) 13:05:26,658 ERROR [STDERR] at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) 13:05:26,658 ERROR [STDERR] at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) 13:05:26,658 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.ExtendedSerializationImpl.read(ExtendedSerializationImpl.java:163) 13:05:26,658 ERROR [STDERR] at rocks.inspectit.shared.all.kryonet.TcpConnection.readObject(TcpConnection.java:209) )
		Reopen
		Remove mapping
	Use instrumentation dialog from Repository manager to refresh agent
Business Context	Application	Create application
		Move application
		Add several matching rules
		Switch to advanced mode
		Add some AND & OR operator
		Add some more rules into ADD and OR
		Save & Close
		Reopen
		Delete added AND & OR
		Switch back to simple mode
		Edit (F2)				Editing application from the table discards the non saved changes in the editor.
		Save
	Business transaction	Add BTs (test insert before option)
		Move BTs
		Delete BTs
		Add several matching rules
		Add some AND & OR operator
		Add some more rules into ADD and OR
		Save & Close
		Reopen
		Enable extract name dynamically (parameter value SearchForm:SearchQuery for DVD store)
		Test BTs on incoming data
		Edit (F2)
		Save & Close				Changing values in the dynamic extraction part does not get saved correctly. Furthermore it's also not displayed correctly when switched between the BTs. (Tobias Angerstein could not reproduce)
	Delete created application
Alerting	Connect to email service (NT-VPN host: smtp or your google account)
	Alert	Create
		Use suggestions from influxDB
		Add mail address
		Test with performance problems (Slow Search)
		Use alert ID in the UI (toolbar)
		Create one with lower bound
		Delete
End User Monitoring	Configuration	Activate EUM in the environment
		Check JS agent is delivered
		Check scripts properties are same as EUM configuration (relevancy threshold, insturment listeners, etc)
		Add all modules (plugins)
		Check agent JS link has included modules
	Tracing	Check correlation between client and server page load requests
		Check resource loading is connect
		Perform other Tracing tasks from the table above if not already
	influxDB	Make sure data EUM data is written to influxDB
	influxDB	Verify dashboard is working
	Data	Browser / OS
		Listeners
		Resources
		Page requests
		Speed index
Diagnosis	Configuration	Activate diagnosis
	Configuration	Adapt baseline if needed
	Create some slow requests					CMR log contains errors.	INSPECTIT-2543 - Getting issue details... STATUS
	influxDB	Make sure data is written to influx
		Verify baseline is respected
		Verify dashboard is working				Bottom table of the dashboard should be more clear. We should add trace and problem context durations as well.
	Turn off influxDB and make sure diagnosis will stop
Remote technologies	Apache HTTP client (latest)
	Async Apache HTTP client (latest)
	Spring Rest Template (latest)
	Jetty Http Client (8.latest)
	Url Connection (java 8)
	Url Connection (java 9)					Agent can not be initialized with Java 9. Spring failure. Works with Java 9 fix.	INSPECTIT-2544 - Getting issue details... STATUS
	Executors (java 8)
	Executors (java 9)					Agent can not be initialized with Java 9. Spring failure. Works with Java 9 fix.	INSPECTIT-2544 - Getting issue details... STATUS
	JMS
	Manual remote server sensor



RESTful API	CMR	http://localhost:8182/rest/cmr/version
	CMR	http://localhost:8182/rest/cmr/status					Error on the end-point.	INSPECTIT-2542 - Getting issue details... STATUS
	Agents	http://localhost:8182/rest/data/agents
		http://localhost:8182/rest/data/agents/{id}/methods
		http://localhost:8182/rest/data/agents/{id}/sensors
	Business Context	http://localhost:8182/rest/bc/app
	Business Context	http://localhost:8182/rest/bc/app/{id}/btx
	Business Context Definition	http://localhost:8182/rest/bc/definition/app
		http://localhost:8182/rest/bc/definition/app/{id}
		http://localhost:8182/rest/bc/definition/app/{id}/btx
		http://localhost:8182/rest/bc/definition/app/{id}/btx/{id}
	Alerting	http://localhost:8182/rest/alerts
	Alerting	http://localhost:8182/rest/alerts/closed
	Invocations	http://localhost:8182/rest/data/invocations
		http://localhost:8182/rest/data/invocations?minDuration=100
		http://localhost:8182/rest/data/invocations/{id}
Stop CMR service (Win only)				-
Uninstallers	Run uninstaller
AgentPropertyView	Select agent in repository manager						AgentPropertyView throws NullPointer - possibly caused by a race condition (I cannot reproduce the error, yet)