Load Test Statistics
Load Tester collects an extensive measurements during a load test. This data is collected in samples. Each sample has multiple measurements associated with it. Depending on the measurement type, some measurements may not ba applicable during each sample and may therefore appear empty in some displays, such as the Statistics View. Unless otherwise noted, the value of each measurement in a sample pertains only to events that completed within that sample period.
While the term sample is used for our data storage units, it is important to understand that Load Tester does not rely on sampled data as some other load-testing products do. Unlike those products, Load Tester measures the response time of every single page and transaction executed. For instance, every web page duration is measured and the page duration statistics reported reflect every page completed during the test.
Test statistics
Test statistics are collected by the Virtual Users as the testcases are executed. The data is collected at four levels: Test Summary, Testcase, Web Page and Transaction. These levels represent narrowing scopes of data - for example: web pages may contain many transactions and a testcase may contain many web pages. In many cases, the higher levels are not simply aggregations of lower levels but measurements tailored specifically for each level.
Average Duration - The average duration for the item. For transactions, the measured duration starts when the first byte of the request is sent and ends when the final byte of the response is received. If a new connection was initiated for the transaction, the connect time is included as well. For pages, the measured duration starts when the first transaction on the page begins and ends when the final transaction completes. For testcases, the measured duration starts when the first page starts and ends when the final page completes.
Average Page Duration - The average page duration for all pages.
Average Page Wait Time - The average amount of time that users have waiting for a web page to complete. This counts all users that are actively working on a web page at the end of the sample period. Because page durations that are used in the min/max/avg page duration measurements are not reported until the page completes, this measurement can be useful in determining backlog on the server as the performance degrades.
Average Speed - The average data transfer rate of the item. For testcases, this measurement includes the total bytes transferred for all web pages and transactions and the total duration of the testcase, including think time but not repeat delay. For web pages, this includes the bytes transferred for all transactions and the total duration of the page, not including think time. For transactions, this includes bytes sent and received (the total of the size of the HTTP request and response messages) and the duration of the transaction.
Average Wait Time - The average amount of time a user has been waiting for the item to complete. This is similar to average page wait time, but for the individual item (testcase, web page or transaction).
Average TTFB - The average TTFB for the transaction. The Time To First Byte measures the time from the beginning of the request to the beginning of the response. On large or slow transactions in some systems, this measurement can be useful to separate the time required for the server to service a request from the time required to transfer the response over the network.
Bytes/sec - The average rate at which data was sent and received. It represents all bytes sent/received during the sample period.
Current Page Users - The number of users working on a web page at the end of the sample period. Note that working on a web page refers to the process of requesting and receiving the contents of a URL. Users that are not working on pages will either be executing think time or the repeat delay.
Errors - The number of error recorded. At each level, the number of an errors is an aggregation of all lower levels.
Hits/sec - The number of transactions completed per second. "Transactions/sec" would be more technically accurate but is not used for two reasons: (1) hits/sec is a standard industry term and (2) the term transaction is already applied to many other levels of interaction in business applications. Note that a completed transaction implies that there is a complete response received from the server. Therefore, transactions which are prematurely terminated due (e.g. due to network errors) are not counted as a hit. Transactions which complete successfully but result in an error due to transaction validators are counted as a hit. Thus you can have errors when hits/sec is zero. This measurement is reported only at the test summary level.
Maximum Duration - The highest duration for the item. See Average Duration for more details.
Maximum Page Duration - The highest page duration. See Average Page Duration for more details.
Maximum TTFB - The highest TTFB for the transaction. See Average TTFB for more details.
Minimum Duration - The lowest duration for the item. See Average Duration for more details.
Minimum Page Duration - The lowest page duration. See Average Page Duration for more details.
Minimum TTFB - The lowest TTFB for the transaction. See Average TTFB for more details.
Pages/sec - The average rate at which pages were completed.
Repeats - The number of times that the item was completed during the sample period. This applies to Testcases, Web Pages and Transactions.
Time - The time at which the sample period ended. This is expressed as an elapsed time since the start of the test.
Total Size - The total number of bytes transferred for this item. Includes both request and response. Note that the bytes transferred includes only the bytes that make up the HTTP transactions. Other bytes transferred at lower protocol levels, such as those required for TCP/IP overhead, are not included.
Users - The number of users currently being simulated. At the test summary level, this is the total number of users in the test. At the Testcase level, this is the number of users simulating the testcase.
Load Engine statistics
Load Engine statistics are collected by the load engines in the process of monitoring its own performance.
Time - (see above)
CPU % - The total CPU utilization as reported by the OS.
Memory % - The amount of VM memory that is available. Note that the load testing and/or the load engine, by default, is configured, by default, to use 200M of RAM. There may be more memory available. If so, see the Configuring Memory Usage page for configuration instructions.
Users - The number of VUs running in this load engine.
Total Capacity - The estimated number of VUs that the load engine can handle. Note that this estimate is very conservative and can be very innaccurate at low numbers. It is not unusual for a load engine to report a total capacity of 15 VUs when 1 VU is running, 80 when 10 are running and 500 when 400 are running.
Engine Status - The state of the load engine (Idle, Initializing, Running, Stopping, Error or Off-line).
Server statistics
Server statistics are collected by the Server Agents to help assess the performance of the server OS and application servers during a load test.
Operating System
CPU % - The total CPU utilization as reported by the OS.
Memory % - The total memory utilization as reported by the OS.
Context Switches / sec - The rate at which the OS kernel switches from one thread to another.
Process Queue Length - The number of system threads in the process queue.
Page reads/sec - Rate of major memory faults resulting in page reads.
Page writes/sec - Rate at which pages are moved to disk to free space in physical memory.
Disk I/O Time % - Elapsed time during which a disk was servicing read or write requests.
Avg Disk Service Time - Average amount of time for each disk I/O transfer (ms)
Disk Queue Length - Number of queued disk operations.
Disk Reads/sec - Rate of disk read operations.
Disk Writes/sec - Rate of disk write operations.
Network bytes sent - Network outbound throughput.
Network bytes received - Network inbound throughput.
Network packets received / sec - Number of network packets received per second.
Network packets sent / sec - Number of network packets sent per second.
Network Packets Received Errors - Number of network packets received with errors.
Network Packets Sent Errors - Number of network packets sent with errors.
Network Collisions - Number of network packet transmission attempts that resulted in a collision.
TCP Connections Established - Number of open TCP connections.
TCP Connection Failures - Number of failed TCP connections.
TCP Segments Retransmitted / sec - Rate of previously transmitted TCP segments being retransmitted.
ASP.NET (Active Server Pages)
ASP.NET Active Sessions - The number of sessions active.
ASP.NET Application Restarts - Number of times the application has been restarted during the sample period
ASP.NET Cache Entries - Total number of entries within the ASP.NET application cache (both internal and user added)
ASP.NET Cache Hit Ratio - Ratio of hits from all ASP.NET application cache calls. Note that the Windows counter calculates this value as a historical average since the last restart, but Load Tester does not.
ASP.NET Cache Hits - Number of hits from all ASP.NET application cache calls.
ASP.NET Cache Misses - Number of misses from all ASP.NET application cache calls.
ASP.NET Cache Turnover Rate - Number of additions and removals to the total ASP.NET application cache per second.
ASP.NET Compilations - Number of .asax, .ascx, .ashx, .asmx, or .aspx source files dynamically compiled.
ASP.NET Current Requests - The current number of ASP.NET requests, including those that are queued, currently executing, or waiting to be written to the client. Under the ASP.NET process model, when this counter exceeds the requestQueueLimit defined in the processModel configuration section, ASP.NET will begin rejecting requests.
ASP.NET Disconnected Requests - The number of requests that were disconnected due to communication failure.
ASP.NET Errors/sec - Rate of errors occurred.
ASP.NET Last Request Execution Time - The amount of time that it took to execute the most recent ASP.NET request.
ASP.NET Pipeline Instance count - Number of active ASP.NET application pipeline instances.
ASP.NET Queued Requests - The number of ASP.NET requests waiting to be processed.
ASP.NET Rejected Requests - The number of ASP.NET requests rejected because the request queue was full.
ASP.NET Request Wait Time - The amount of time the most recent request was waiting in the queue.
ASP.NET Requests Executing - The number of ASP.NET requests currently executing
ASP.NET Requests Not Found - The number of ASP.NET requests for resources that were not found.
ASP.NET Requests Not Authorized - Number of requests failed due to unauthorized access.
ASP.NET Requests Queue Length - The number of ASP.NET requests in the application request queue.
ASP.NET Requests Succeeded - The number of requests that executed successfully during the sample period.
ASP.NET Requests Timed Out - The number of requests that timed out.
ASP.NET Requests/sec - The number of ASP.NET requests executed per second.
ASP.NET Sessions Timed Out - The number of sessions that have been closed due to their idle time exceeding the AutoDisconnect parameter for the server. Shows whether the AutoDisconnect setting is helping to conserve resources.
ASP.NET Sessions Abandoned - The number of sessions that have been explicitly abandoned.
ASP.NET Unhandled Execution Errors/sec - Rate of unhandled errors.
.NET CLR (Common Language Runtime)
.NET CLR Class loader Heap Size - The current size (in bytes) of the memory committed by the class loader across all AppDomains. (Committed memory is the physical memory for which space has been reserved on the disk paging file.)
.NET CLR Current AppDomains - The current number of AppDomains loaded in this application. AppDomains (application domains) provide a secure and versatile unit of processing that the CLR can use to provide isolation between applications running in the same process.
.NET CLR Current Assemblies - The current number of Assemblies loaded across all AppDomains in this application. If the Assembly is loaded as domain-neutral from multiple AppDomains then this counter is incremented once only. Assemblies can be loaded as domain-neutral when their code can be shared by all AppDomains or they can be loaded as domain-specific when their code is private to the AppDomain.
.NET CLR Exceptions / sec - The number of exceptions thrown per second. These include both .NET exceptions and unmanaged exceptions that get converted into .NET exceptions e.g. null pointer reference exception in unmanaged code would get re-thrown in managed code as a .NET System.NullReferenceException; this counter includes both handled and unhandled exceptions. Exceptions should only occur in rare situations and not in the normal control flow of the program; this counter was designed as an indicator of potential performance problems due to large (>100/s) rate of exceptions thrown.
.NET CLR Heap Size - The sum of four other counters; Gen 0 Heap Size; Gen 1 Heap Size; Gen 2 Heap Size and the Large Object Heap Size. This counter indicates the current memory allocated in bytes on the GC Heaps.
.NET CLR Generation 0 Collections - The number of times the generation 0 objects (youngest; most recently allocated) are garbage collected during the sample period. Gen 0 GC occurs when the available memory in generation 0 is not sufficient to satisfy an allocation request. Higher generation GCs include all lower generation GCs. This counter is explicitly incremented when a higher generation (Gen 1 or Gen 2) GC occurs.
.NET CLR Generation 1 Collections - The number of times the generation 1 objects are garbage collected. Higher generation GCs include all lower generation GCs.
.NET CLR Generation 2 Collections - The number of times the generation 2 objects (older) are garbage collected. The counter is incremented at the end of a Gen 2 GC (also called full GC).
.NET CLR Lock Contention Queue Length - The number of threads in the runtime currently waiting to acquire a lock.
.NET CLR Lock Contention / sec - Rate at which threads in the runtime attempt to acquire a managed lock unsuccessfully. Managed locks can be acquired in many ways; by the "lock" statement in C# or by calling System.Monitor.Enter or by using MethodImplOptions.Synchronized custom attribute.
.NET CLR Lock Contentions - The total number of times threads in the CLR have attempted to acquire a managed lock unsuccessfully during the sample period. Managed locks can be acquired in many ways; by the "lock" statement in C# or by calling System.Monitor.Enter or by using MethodImplOptions.Synchronized custom attribute.
.NET CLR Logical Thread Count - The number of current .NET thread objects in the application. A .NET thread object is created either by new System.Threading.Thread or when an unmanaged thread enters the managed environment. This counters maintains the count of both running and stopped threads.
.NET CLR Percent Time in GC - The percentage of elapsed time that was spent in performing a garbage collection (GC) since the last GC cycle. This counter is usually an indicator of the work done by the Garbage Collector on behalf of the application to collect and compact memory.
.NET CLR Physical Thread Count - The number of native OS threads created and owned by the CLR to act as underlying threads for .NET thread objects. This does not include the threads used by the CLR in its internal operations; it is a subset of the threads in the OS process.
.NET CLR Recognized Thread Count - Displays the number of threads that are currently recognized by the CLR; they have a corresponding .NET thread object associated with them. These threads are not created by the CLR; they are created outside the CLR but have since run inside the CLR at least once. Only unique threads are tracked; threads with same thread ID re-entering the CLR or recreated after thread exit are not counted twice.
IIS (Internet Information Server)
IIS CGI Requests/sec - The rate CGI requests are received by the Web service.
IIS Connection Attempts/sec - The rate that connections to the Web service are being attempted
IIS Current Connections - Current Connections is the current number of connections established with the Web service.
IIS File Cache Entries - The number of entries in the File Cache.
IIS File Cache Hits - The number of successful lookups in the file cache.
IIS File Cache Hit % - The ratio of file cache hits to total cache requests. Note that the Windows counter calculates this value as a historical average since the last restart, but Load Tester does not.
IIS File Cache Misses - The number of failed lookups in the file cache.
IIS ISAPI Extension Requests/sec - The rate that ISAPI Extension requests are received by the Web service.
IIS Kernel URI Cache Entries - The number of entries in the Kernel URI Cache.
IIS Kernel URI Cache Hits - The number of successful lookups in the Kernel URI cache.
IIS Kernel URI Cache Hit % - The ratio of Kernel URI cache hits to total cache requests. Note that the Windows counter calculates this value as a historical average since the last restart, but Load Tester does not.
IIS Kernel URI Cache Misses - The number of failed lookups in the Kernel URI cache.
IIS Requests/sec - The rate HTTP requests are received by the web server.
IIS URI Cache Entries - The number of entries in the URI Cache.
IIS URI Cache Hits - The number of successful lookups in the URI cache.
IIS URI Cache Hit % - The ratio of URI cache hits to total cache requests. Note that the Windows counter calculates this value as a historical average since the last restart, but Load Tester does not.
IIS URI Cache Misses - The number of failed lookups in the URI cache.
IIS+App
These statistics represent the combined processes for IIS (inetinfo), ASP.NET (aspnet_wp) and the Application Pool Process (w3wp).
IIS+App Handle Count - The number of handles held by known web server processes.
IIS+App Private Bytes - The current size of committed memory owned by known web server processes.
IIS+App Thread Count - The current number of threads active for known web server processes.
IIS+App Virtual Bytes - The current size of virtual address space for known web server processes.
IIS+App % Processor Time - The percentage of elapsed time that threads from known webserver processes used the processor to execution instructions..
http://www.webperformanceinc.com/support/suite/manual/Load_Test_Statistics.htm