Load testing only on stable code?



  • There's someone who oversees QA - let's call him Joe. During 2 years of development we haven't had real load tests. Now release is getting close and QA is responsible for preparing and executing load tests. We (dev team) have suggested, that the load tests should be prepared and executed ASAP, before UAT starts and re-executed at least once before release.

    Our proposal has been refused by Joe. The reason was that code is still changing and every code change could possibly break test scripts and require them to be updated. Joe explained that load tests should happen after code freeze, preferably after everything is already approved by the business. It was also refused by head of QA for the same reasons.

    I'm not really into testing, so - is this really the way to go? Is it common practice to load test just before the release? Or are they all insane?



  • If the system needs to scale to many concurrent users or work with lots of data, I would say load testing should be started as early as possible. This way possible problems in the application architecture can be found in a phase where it is still possible to fix them.

    Couple examples:

    • Let's say that the database schema is such that some queries will be slow with a large number of rows. Computers are nowadays so quick that nearly any query is fast with small amount of data and problems start only with a larger datasets. You really don't want to change database structure just before release. Load testing can be started as soon as database structure is decided, no need to wait for GUI to be ready. If the structure changes, earlier performance results can be used to evaluate the impact of the change.

    • Similarly problems in GUI may occur only with load. Maybe some third party library seems fine, but actually sorts stuff with bubble sort. Maybe another is not thread-safe but crashes or returns wrong data with certain timings. Maybe separate parts accidentally use same mutexes and cause deadlocks.

    • Maybe the system is designed for 1000 concurrent users. How it behaves with 2000 users? Does it crash, became unusable slow or gracefully serve some users and ask others to try again later?

    It's not really possible to find these problems without testing. In my experience fixing this kind problems take time. If they are tested only just before release, any problem can cause delay in release schedule. As with nearly everything, test as early as possible, even if it's just a part of the whole system.



Suggested Topics

  • 3
  • 2
  • 2
  • 2
  • 2
  • 2
  • 3
  • 2