What to Test

From Training Material
Jump to navigation Jump to search

Performance Testing Requirements

  • Non functional requirements come from the business strategy or contract with clients
    • e.g. If the strategy is to have few rich visitors test should focus on reliability rather than expect billions of visitors
  • Subset of functional testing (Use Cases, User Stories and their Scenarios)
  • Most frequent scenarios
  • Scenario which potentially impacts performance
  • Prioritized by business people (managers) not testers
  • Not everything should be tested
  • Setting goals and scope of test is crucial (otherwise it is never done)
  • Testing can be a continuous process

Performance Requirements

  • Specific Hardware (8 Athlon Processors, 512MB RAM, SSD, etc..)
  • Software Confiugration
  • Max no of records in the database (e.g. 100 000 000)
    • Can be express in business terms (number of clients, contacts, invoices, enquiries, etc...)
  • Max database size (e.g. 100 GB)
  • Maximum number of registered user (e.g. 4500)
  • Distribution of scenarios: scenarios 1 (20%), scenario 2 (30%), scenario 6 (70%)
    • Average Response Time (e.g. 1500ms)
    • Maximum page load time (e.g. 10 sec)
    • Minimum throughput (e.g. 2400 pages/minute)
    • Concurrency users (300 users working concurrently)

Performance Test results

Client side:

  • Response Time: distribution shape, min, max, average, standard deviation, median, skewness
  • Throughput (scenarios/min)
  • Error Level, error responses in details

Servers side:

  • Memory utilization
  • CPU utilization
  • Network throughput
  • IO waiting time
  • Storage free space

Performance Testing Process

  • How to identify your testing needs
  • Defining the steps of your test
    • Gathering requirements (use cases, user stories)
    • Scoping (choosing and prioritize scenarios)
    • Designing tests
    • Preparing data
    • Preparing environment
    • Recording tests
    • Perform the test
    • Analysis and reports
    • Conclusions and improvement