How to determine breaking point of my application when performance testing from jMeter
I am doing some performance testing with jMeter on my application. I've check marked
Retrieve all Embedded Resourcesand
Clear cache on each iteration.
For the thread group I've got: Number of threads 10, Ramp up period: 60, Loop Count 3.
- How can I best determine the breaking point of my application? (i.e. the point at which it becomes unresponsive)
- Which listener will best be able to tell me the breaking point of my application (i.e. how many concurrent users it can handle)
First you need to define what the "breaking point" is. There are a number of possibilities. The obvious problem would be if the web server itself crashes and stops responding, but there are a number of problems that can occur well before that point that in most cases would still be considered "broken".
Some other problems that could tell you your application is "broken":
- Response times become unreasonably long. A general rule of thumb is if a web page takes longer than 6 seconds to load, from a user perspective, they will lose interest and become impatient.
- Some (but not all) requests begin returning non 200 response codes. You will need to set up validations for your responses in jmeter to watch for non 200 response codes.
- The web server crashes and restarts automatically causing a few requests to fail, but all existing sessions to immediately expire.
None of the Jmeter listeners will tell you anything about the health of your application, it just tells you about some of the symptoms such as slow response times, or error response codes. In order to actually monitor the health of the web server, you need to monitor the system it is running on to watch memory usage, cpu usage, network usage, disk IO, etc. Many tools like Tomcat have some performance counters you can tap into and get more information as well.
To actually determine where you application breaks is a blurry line as well. If you immediately pound a server with 1000 requests, it could break, but if you ramp up slowly to 1000 threads it may handle it. In addition, you may be able to run 100 threads just fine, but if you run 100 threads for an hour straight, it may break. Again, the most obvious and the most common scenario is that you run a certain number of threads and it breaks when you reach some threshold.
To determine where that threshold is, you can often use a step load pattern where you start at some number of threads, and every X time period increase the threads by Y. So for example, every 10 minutes I might increase threads by 5, and I might run that test overnight and come back in the morning to see how far it got before the system "broke". The closest out of the box functonality in Jmeter to do this kind of load, would be the "ramp up". You would need to put the number of threads up to something very large (number depends on the project), maybe 1000+ and change the ramp up time to something very large like 43200 (12 hours) so that it continually ramps up until the application breaks.
Another option would be do multiple runs where you manually increase the number of threads for each run. For Jmeter, I would actually prefer creating or using a plugin to do the step up load since their out of the box functionality does not provide an ideal solution.