2014-04-14 Goals of End User Experience - Workshop

These are the results of the initial workshop on End-User Experience Monitoring. We set the goal of the workshop to get a better understanding of what we expect of End-User Monitoring in inspectIT. We wanted to identify the topics that we need to address and prioritize them. The workshop was done from 13.04.2014 to 14.04.2014. Matthias, Ivan and Stefan were present.

The document will divide in several sections that main topic groups that were discussed. In each of these sections the concrete ideas and functionality will be presented with the prioritized number. Since for each "feature" we voted with 0, 2 and 5 points, the priority number can span from 0-15.

Trace

The trace section relates on how should the different end-user monitoring metrics be grouped and organized. Here we recognized three main "containers":

1. User interaction - As we defined it a user interaction is group of events (JS, AJAX calls, HTTP requests, resource loading, etc) that occurred after user has made an interaction as mouse click or key pressed. One user interaction contains all events that happened until the next interaction occurs. Here is important to define that we would like to have a duration of user interaction defined as the time spent from initializing the interaction until all the events have finished. Time spend from all events finished till the next interaction can be in some way called "user think time", but we did not defined if we would like to use this at all for something.
2. Session - This is basically one user session. It contains several user interactions.
3. Visit - Visit is a collection of several sessions performed by the same user. Here user is not defined by the session ID, but with username, e-mail details or something else. 

In addition we want to have the trace of all the things happening in the browser (for example JS1() functions calls JS2() and afterwards loads two images).

Metrics

In this group we want to define what kind of metrics we would like to collect. We recognized two main groups:

1. Timing metrics or events - events that influence the time spend
2. Demographics metrics - metric that give us more information about the user and environment

Diagnosis

Diagnosis should serve as a possibility to explore in more detail a possible performance problem. Here we defined that we would like to have the navigation possibilities and to be able to aggregate and filter the set of information. One example here was finding a slow JS function in a user session, navigating to aggregated information about same function and then for example aggregating on the Browser Type bases, so that we can maybe check if it s slow in all browsers or only one. Or if we have slow HTTP request, check if there is difference in from which location was the request fired.

Monitoring

We also discussed on basic monitoring support that we would like to have. The main discussion here was going in the direction that to support monitoring we need to store data long-term. We also discussed what would be most important to monitor and what opportunities should be given to the user.

Analytics 

The commercial tools have additionally possibility to do do business related analytic with the help of the end-user monitoring. We concluded that this is not of greatest interest to us and should not have the high priority. Still we selected two topics that are nice to have:

1. Business section - we defined this as a critical part of work user has to do on the application, like for example the check-out process. These sections are usually the same. It would consist of one or more user-interactions, but does not have to be big as the complete session, but more just one part of it.
2. Conversion rate - Possibility to define when user is converted. Like reached the page offerProcessed.html or clicked on "Register for Newsletter". 

Expert knowledge

Expert knowledge is imaged as the place where we could suggest the user what should be improved and how much time could be saved. This is mainly taken from the Ajax Edition of Dynatrace.