End User Monitoring 101: Tools Part 1
The first aspect of Front End Optimization (FEO) practice in Operations is understanding the outturn performance to external end points (usually end users). This is achieved through monitoring, that is, obtaining an objective understanding of transaction, page, or page component response from replicate tests in known conditions, or of site visitors over time.
Monitoring provides information relative to patterns of response of the target site or application, both absolute and relative to key competitors or other comparators.Tools for monitoring of external performance fall into two distinct types: active or passive.
Active (also called Synthetic) monitoring involves replicate testing from known external locations. Data captured is essentially based on reporting on the network interactions between the test node and the target site.
i) Understanding the availability of the target site.
ii) Understanding site response/patterns in consistent test conditions – for example to determine long term trends, the effect of visitor traffic load, performance in low traffic periods, or objective comparison with competitor (or other comparator) sites.
iii) Understanding response/patterns of individual page components. These can be variations in the response of the various elements of the object delivery chain – DNS resolution, Initial connection, First byte (i.e. the dwell time between the connection handshake and the commencement of data transfer over the connection – a measure of infrastructure latency), and content delivery time. Alternatively, the objective may be to understand the variation in total response time of a specific element, for example 3rd Party content (useful for Service Level Agreement management).
Increasingly, modern Application Performance Management (APM) tools offer a synthetic monitoring option. These tend to be useful in the context of the APM – i.e. holistic, ongoing performance understanding, but more limited in terms of control of test conditions and specific granular aspects of FEO point analysis such as Single Point Of Failure (SPOF) testing of third party content.
In brief, key aspects of such tooling for FEO analysis are:
■ Range of external locations – geography and type
- e.g. Tier 1 ISP/LINX test locations; end user locations; private peer (i.e. specific known test source)
- PC and mobile (the latter increasingly important)
■ Control of connection conditions – hardwired vs wireless; connection bandwidth
■ Ease & sophistication of transaction scripting – introducing cookies, filtering content, coping with dynamic content (popups etc.)
■ Control of recorded page load end point
As a rule of thumb, the more control the better. However, a good compromise position is to take whatever is on offer from the APM vendor – provided you are clear as to exactly what is being captured; and supplement this with a ‘full fat' tool that is more analysis-centric – WebPageTest being a popular, open source choice – though beware variable test node environments if using the public network.
A final word on page load end points. "Traditional" synthetic tools (such as Gomez/dynaTrace synthetic in the above example) relied on the page onload navigation marker. It really is essential to define an end point more closely based on end user experience – i.e. browser fill time. With older tools this needs to be done by introducing a flag to the page. This can either be existing content such as an image appearing at the base of the page (at a given screen resolution), or by introducing such content at the appropriate point. This marker can then be recorded by modification of the test script.
Note that, given the dynamic nature of many sites, attempting to time to a particular visual component can be a short lived gambit. Introducing your own marker, assuming that you have access to the code, is a more robust intervention.
Some modern tooling have introduced this as a standard feature. It is likely that competitors will follow suit. Use of the onload marker will produce results that do not bear any meaningful relationship to end user experience, particularly in sites with high affiliate content loads.
Modifications of standard testing to meet the requirements/manage misleading results in specific cases e.g. server push, Single Page Applications, will be covered in a subsequent post.
Larry Haig is Senior Consultant at Intechnica.