...
- Using The Grinder
- Using Apache JMeter (Call for Participation)
- Bamboo
- Compute and storage infrastructure
- Performance tests (code)
- Reference test data
- Report publishing facility
Test Resources
To be valid, a performance test must minimize or isolate external influences. This can be done with a dedicated platform (and network if a multi-host configuration is used). Unfortunately, cloud platforms are shared and the performance allocated to any given set of processes varies with time. To utilize a ahared platform, you must monitor it and factor its load into the performance calculations. This can be complicated since there are so many potential factors to consider but "rough order of magnitude" results are still quite useful. This is still useful and may be the most practical approach for interim performance test.
- Best Accuracy - Isolated Platform
- Must be provided by one or more hosts
- Also good if on isolated LAN segment
- Dedicated disks particularly helpful and can test various storage subsystems
- Separate machine for database and/or resource index
- Separate machine for load injection
- Costs system admin time to set up but the installation can be reused
- Likely only one or a small number of configurations can be supported
- Some costs in moving test data sets around
- It would be nice to permit a Bamboo agent to kick off tests
- Reports can be sent to a common location
- Most Convenience - Cloud Platform
- Costs cash money but only during testing
- Difficult to eliminate variables so results will not be as accurate
- Relatively easy to provide a number of configurations (but there is setup time)
- AMI or equivalent must be built (though some of this can be done automatically)
- Shared LAN/WAN only choice
- Shared disk only choice
- Easy to incorporate Bamboo
- Reports can be sent to a common location
There can be any combination of the above and it can start simple, built incrementally.
What is the minimal useful starting point:
Configuration 1 -
- Single server-grade host with direct attached disks
- Load injector on server