Eye On Performance: A Load Of Stress
By Jack Shirazi & Kirk Pepperdine
2004-01-22
Reader Rating: 
A rich feature set
Now that we've looked at the basic functionality of stress testing tools, what additional functionality would be nice to have? How do the various load testing tools differ? Your primary requirement must, of course, be that the tool can simulate your application clients. That may rule out a number of candidates if your application uses some unusual combination of browser features or other non-standard client technology. Beyond that, there are other features that make the load testing more productive. The following list may be a useful starting point for making a decision on the appropriate load testing tool for your project:
Simulates your clients
The primary requirement must be that the load tester handles the features and protocols that your application uses.
Run multiple simulated clients
This is the most basic functionality of a load tester, and it helps to determine what is a load tester and what isn't (some frameworks try to masquerade as load testers.).
Scripted execution with ability to edit scripts
If you can't script the interaction between the client and the server, then you cannot handle anything except the most simple clients. The ability to edit the scripts is essential -- minor changes shouldn't require you to go through the process of re-generating a script.
Supports sessions
If a load tester cannot support sessions or cookies, it isn't really much of a load tester, and it won't be able to load-test most J2EE applications.
Configurable numbers of users
The tester should let you specify how many simulated users are running each script, including letting you vary the number of simulated users over time, as many load tests should start with a small number of users and ramp up slowly to higher numbers of users.
Report success, errors, and failures
Each script must have a defined way to identify a successful interaction, as well as failure and error modes (an error would be getting no page back at all, whereas a failure might be getting the wrong data back on the page.)
Page display
It is useful for the load tester to allow you to inspect some of the pages that are being sent to the simulated users, so that you can be confident the test is working correctly.
Export results
You often want to be able to analyze test results with various tools, including spreadsheets and custom analysis scripts that can process the data. While many load-test tools include extensive analysis functionality, being able to export the data gives you more flexibility to analyze and catalog the data in arbitrary ways.
Think times
Real-world users do not request one page immediately after another -- there are generally delays between viewing one page and the next. Think time is the standard term to express adding a delay into a script to more realistically simulate user behavior. Most load testers support randomly generating think times based on a statistical distribution.
Client selection of data from lists
Users don't tend to work with the same set of data; each is usually running a different interaction with the server. The simulated users should also be doing this, and it is easier to make your simulated users appear to be working with varied data if the scripts can select data from lists of data at strategic points in the interaction.
Recording scripts from manually executed sessions
Rather than writing scripts, it is so much easier to just manually run through a session with your browser and have that session recorded for later editing.
JavaScript
Some applications use JavaScript extensively and need it to be supported by the simulated clients. However, the use of client-side JavaScript may increase the requirements for system resources on the test systems.
Analysis tools
Measuring performance is half the story. The other half is analyzing the performance data. Who better to build those analysis tools than the people who built the measurement tools? Well, at least that's the theory. In any case, the more analysis tools your toolkit provides, the better off you are.
Measure server-side statistics
The basic load tester measures client-based response times from client/server interactions. It would also be nice to gather other statistics, such as CPU utilization or page fault rates, as well. The more statistics you can get, the more you can do with your load testing system. If you have this data, you can then do useful things like view client response times in the context of server load and throughput statistics.
First published by IBM developerWorks
If you found this article interesting, you may want to read these as well:
» Build and Implement A Single Sign-On Solution
» Scheduling Recurring Tasks In Java Applications
» A Brief History Of Garbage Collection
|
