SOAPUI AND LOADUI TDE (Test Driven Engineering)

I've been using Eviware's SOAP and LOADUI this last month as part of System testing on a project I've been working on this year.

I started very confidently as I had used SOAPUI on a more focused level on software development tasks in the past, but then I started to hit a number of obstacles:

• Complexity of set-up and tear down activities limited the scope of what I could achieve in the time available.
• Only a third of the system was exposed as web services, the rest of it was a combination Java Api's, JMS, etc,..
• Concurrency requirement were very vague.
• Throughput requirements were not related to web service operations

This experience has caused me to reflect on the value and level of detail required in Non Functional performance requirements for each system interface.

The biggest lesson learnt is the combining of different aspects of performance into a integrated model of performance: in which different measures of performance are combined and tested together (Responsiveness, Throughput, System load, Growth).

For example Account creation interface should complete in 4 Seconds, 99.95% of the time.
Account creation will be invoked at an Average throughput of 6000 request per hour, Peak throughput per hour is 12,000 requests.
Account creation will be invoked at an Average throughput of 1000 request per minute, peak throughput per minute is 4,000 requests.
The system will be loaded day one with 3.4 million Accounts.
The system will be loaded at the end of year one with 365x250,000 Accounts.