Explorative Testing
- Stefan Siegl
- Ivan Senic
- Isabel Vico Peinado
Owned by Stefan Siegl
As the part of the release process it was agreed that prior to release one complete day is reserved for explorative testing that should prove that we don't have any critical bugs. In addition this testing should make us aware of some minor bugs that are always present. These minor bugs should be documented on JIRA and planned for next version.
The steps that are described on this page were used for explorative testing for version 1.5. Please update this page and add additional steps for versions to come.
Explorative testing should perform same steps on both Linux and Windows machines. It's in most cases duplication of work, but it happens often that bug is related to only one platform, especially for the User interface.
Steps
Perform these steps in chronological order (grouping is just for better understanding). Add the person who tests the functionality, if any vulnerability was found and the JIRA ticket if it exists:
| Task | Tester | Found | Ticket | |||
|---|---|---|---|---|---|---|
Installers | Use installers to install inspectIT | |||||
| Select all components and creation of all shortcuts | ||||||
| Test if shortcuts work | ||||||
UI | Welcome screen | Test all internal links | ||||
| Test all external links | ||||||
| Prove search works | ||||||
| Main menu | Click on every item in the main menu, prove all works as expected | |||||
Repository Manager | Add repository | |||||
| Edit repository | ||||||
| Remove repository | ||||||
| Double-click on offline repository | ||||||
| CMR | Start CMR | |||||
| UI | Repository Manager | Restart CMR | ||||
| Shutdown CMR | ||||||
| CMR | Start CMR again (as service in Win) | |||||
UI | Label Management | Add new label type | ||||
| Add new label of existing type | ||||||
| Remove label of existing type | ||||||
| Prove it's saved correctly | ||||||
| Start recording, limit time | ||||||
| Stress CMR for 10-15 mins (best with Invoker sample application) | ||||||
| UI | Stop recording manually | |||||
| Check recording storage | ||||||
| Start one more additional sample app (Spring Petclinic inspectIT) | ||||||
| Shutdown CMR, wait a bit, start CMR | ||||||
| Verify agent is reconnected | ||||||
| Restart agent | ||||||
UI | Copy buffer to storage with agent filter | |||||
| Open both agents in the Data explorer | ||||||
| Make sure agent tree structure saving works as agents are switched | ||||||
Platform sensors view | Check every view | |||||
| Test time-frame | ||||||
| Test sampling rate | ||||||
| Check data below graph | ||||||
| Play around with Invocation browser | ||||||
Invocation sequence Show All | Filters - all types | |||||
| Show hide columns | ||||||
| Minimize / maximize view | ||||||
| Change property like refresh time, close UI and check changes are remembered on restart | ||||||
| Sorting | ||||||
| Searching | ||||||
Call hierarchy | Expand / collapse | |||||
| Locate in Hierarchy (multiple items) | ||||||
| Copy | ||||||
| Navigation (multiple items) | ||||||
| Details | ||||||
| Parameters on context capturing | ||||||
SQL / Method / Exceptions | Test raw / aggregated | |||||
| Locate in Hierarchy (one raw item, one aggregated) | ||||||
| Navigation | ||||||
| Copy | ||||||
| Combine data (select as much as possible invocations) | ||||||
SQL view | Double-click both views | |||||
| Decimal places | ||||||
| Navigation | ||||||
| Sorting | ||||||
| Other menu options | ||||||
| Save SQLs to Server with no invocations included | ||||||
Timer Data view | Display in chart | |||||
| Navigation | ||||||
| Sorting | ||||||
| Save Timers to Server with invocations | ||||||
HTTP | URI | Split method | ||||
| URI Regex | ||||||
| Display in chart | ||||||
| Navigation | ||||||
Tag | Display in chart | |||||
| Navigation | ||||||
Exceptions (Single and Grouped) | Double clicks | |||||
| Check stack traces | ||||||
| Details | ||||||
| Navigation | ||||||
| Tracing | Overview | Sort | ||||
| Copy | ||||||
| Search | ||||||
| Details | ||||||
| Save to Storage (full trace must be saved) | ||||||
| Open details (double-click) | ||||||
| Open details (navigate-to) | ||||||
| Trace (top) | Search | |||||
| Copy complete | ||||||
| Details | ||||||
| Save to Storage (full trace must be saved) | ||||||
| Locate span using double-click | ||||||
| Trace (bottom) | Locate in hierarchy (SQL, Method) | |||||
| Search call hierarchy | ||||||
| Details | ||||||
| Navigate from sub-views | ||||||
| Copy complete call hierarchy | ||||||
| Rename few editors | ||||||
| Delete an agent from Repository | ||||||
Storage | Add new | |||||
| Edit | ||||||
| Delete (test both finalized and not-finalized one) | ||||||
| Filters and grouping | ||||||
| Finalize open storage | ||||||
| Add all available label types to one storage (boolean, date, number and text) | ||||||
| Test changing of all label values in the table | ||||||
| Remove label from storage | ||||||
| Delete label that exists in storage from Label management | ||||||
| Download storage | Compressed | |||||
| Not-compressed | ||||||
| Upload storage | ||||||
| Export | ||||||
| Import | Local | |||||
| To repository | ||||||
| Toolbar | Test all actions | |||||
| Connect to influxDB | ||||||
| Prove data is written in influxDB (Grafana or influx command) | ||||||
| Prove views on the UI that can not display data that is in influx show an info banner | ||||||
| Configuration Interface | Switch between profiles / environments | |||||
| Environments | Create | |||||
| Edit | ||||||
| Export | ||||||
| Delete | ||||||
| Import | ||||||
| Environment editor | Filter profiles | |||||
| Select / deselect profiles | ||||||
| Open profile (double click) | ||||||
| Open profile (menu item) | ||||||
| Change strategies | ||||||
| Deselect some platform sensors | ||||||
| Change sensor options | ||||||
| Save & close | ||||||
| Reload | ||||||
| Edit (F2) | ||||||
| Profiles | Create different profile types | |||||
| Edit | ||||||
| Export | ||||||
| Delete | ||||||
| Import | ||||||
| Set as Default (create new Environment to prove) | ||||||
| Deactivate (confirm by Dynamic instrumentation) | ||||||
| Duplicate | ||||||
| Sensor profile | Add sensor type | |||||
| Show / hide advances sensors | ||||||
| Save not possible when not correct input | ||||||
| One wildcard definition | ||||||
| One interface/super-class definition | ||||||
| One method with parameters (test add/remove/edit parameter) | ||||||
| One with custom modifiers | ||||||
| One with context capturing (test add/remove/edit capturing) | ||||||
| Remove assignment (mouse + DEL) | ||||||
| Duplicate assignment | ||||||
| Add second sensor type | ||||||
| Save | ||||||
| Reopen | ||||||
| Exclude rule profile | Add exclude rule | |||||
| Edit exclude rule | ||||||
| Close (with save) | ||||||
| Reopen | ||||||
| Remove exclude rule | ||||||
| JMX profile | Add one assignment | |||||
| Set specific attributes | ||||||
| Close (with save) | ||||||
| Reopen | ||||||
| Mappings | Deactivate mapping | |||||
| Activate mapping | ||||||
| Change mapping to another environment (don't update with DI) | ||||||
| Add mapping | ||||||
| Save | ||||||
| Reopen | ||||||
| Remove mapping | ||||||
| Use instrumentation dialog from Repository manager to refresh agent | ||||||
| Business Context | Application | Create application | ||||
| Move application | ||||||
| Add several matching rules | ||||||
| Switch to advanced mode | ||||||
| Add some AND & OR operator | ||||||
| Add some more rules into ADD and OR | ||||||
| Save & Close | ||||||
| Reopen | ||||||
| Delete added AND & OR | ||||||
| Switch back to simple mode | ||||||
| Edit (F2) | ||||||
| Save | ||||||
| Business transaction | Add BTs (test insert before option) | |||||
| Move BTs | ||||||
| Delete BTs | ||||||
| Add several matching rules | ||||||
| Add some AND & OR operator | ||||||
| Add some more rules into ADD and OR | ||||||
| Save & Close | ||||||
| Reopen | ||||||
| Enable extract name dynamically (parameter value SearchForm:SearchQuery for DVD store) | ||||||
| Test BTs on incoming data | ||||||
| Edit (F2) | ||||||
| Save & Close | ||||||
| Delete created application | ||||||
| Alerting | Connect to email service (NT-VPN host: stmp or your google account) | |||||
| Alert | Create | |||||
| Use suggestions from influxDB | ||||||
| Add mail address | ||||||
| Test with performance problems (Slow Search) | ||||||
| Create one with lower bound | ||||||
| Delete | ||||||
| End User Monitoring | Configuration | Activate EUM in the environment | ||||
| Check JS agent is delivered | ||||||
| Check scripts properties are same as EUM configuration (relevancy threshold, insturment listeners, etc) | ||||||
| Add all modules (plugins) | ||||||
| Check agent JS link has included modules | ||||||
| Tracing | Check correlation between client and server page load requests | |||||
| Check resource loading is connect | ||||||
| Perform other Tracing tasks from the table above if not already | ||||||
| influxDB | Make sure data EUM data is written to influxDB | |||||
| Verify dashboard is working | ||||||
| Data | Browser / OS | |||||
| Listeners | ||||||
| Resources | ||||||
| Page requests | ||||||
| Speed index | ||||||
| Diagnosis | Configuration | Activate diagnosis | ||||
| Adapt baseline if needed | ||||||
| Create some slow requests | ||||||
| influxDB | Make sure data is written to influx | |||||
| Verify baseline is respected | ||||||
| Verify dashboard is working | ||||||
| Turn off influxDB and make sure diagnosis will stop | ||||||
| Remote technologies | Apache HTTP client (latest) | |||||
| Async Apache HTTP client (latest) | ||||||
| Spring Rest Template (latest) | ||||||
| Jetty Http Client (8.latest) | ||||||
| Url Cconnection (java 8) | ||||||
| Url Cconnection (java 9) | ||||||
| Executors (java 8) | ||||||
| Executors (java 9) | ||||||
| JMS | ||||||
| Manual remote server sensor | ||||||
| RESTful API | CMR | http://localhost:8182/rest/cmr/version | ||||
| http://localhost:8182/rest/cmr/status | ||||||
| Agents | http://localhost:8182/rest/data/agents | |||||
| http://localhost:8182/rest/data/agents/{id}/methods | ||||||
| http://localhost:8182/rest/data/agents/{id}/sensors | ||||||
| Business Context | http://localhost:8182/rest/bc/app | |||||
| http://localhost:8182/rest/bc/app/{id}/btx | ||||||
| Business Context Definition | http://localhost:8182/rest/bc/definition/app | |||||
| http://localhost:8182/rest/bc/definition/app/{id} | ||||||
| http://localhost:8182/rest/bc/definition/app/{id}/btx | ||||||
| http://localhost:8182/rest/bc/definition/app/{id}/btx/{id} | ||||||
| Alerting | http://localhost:8182/rest/alerts | |||||
| Invocations | http://localhost:8182/rest/data/invocations | |||||
| http://localhost:8182/rest/data/invocations?minDuration=100 | ||||||
| http://localhost:8182/rest/data/invocations/{id} | ||||||
| Stop CMR service (Win only) | ||||||
| Uninstallers | Run uninstaller | |||||