Performance Testing vs Unit Testing

  • I'm reading Osherove's "The Art of Unit Testing," and though I've not yet seen him say anything about performance testing, two thoughts still cross my mind:

    • Performance tests generally can't be unit tests, because performance tests generally need to run for long periods of time.
    • Performance tests generally can't be unit tests, because performance issues too often manifest at an integration or system level (or at least the logic of a single unit test needed to re-create the performance of the integration environment would be too involved to be a unit test).

    Particularly for the first reason stated above, I doubt it makes sense for performance tests to be handled by a unit testing framework (such as NUnit).

    My question is: do my findings / leanings correspond with the thoughts of the community?

  • I agree with your findings/learnings. True unit tests only test a portion of the system, ignoring, mocking or faking the rest as necessary. Integration tests (or regression tests) test most or all of the units working together, and that is the true measure of performance.

  • In some situations you can use unit tests to make sure that an operation finishes within a certain time period. If you want to add more features to your operation, but you don't want to sacrifice performance you can use unit tests to assert that. Of course, these kind of unit tests are machine dependent, but you can throw some additional variables or configuration to the equation.

  • I agree that performance tests cannot be unit tests but there is no reason we cannot have another set of tests called performance tests. Broadly the tests fall under two categories

    a) Unit tests

    b) Integration tests

    We run integration tests again the real database (instead of in memory) to ensure the sql scripts, the hibernate repositories work as expected

    My idea is we can add another set of tests called performance tests which are a part of nightly build which tests for performance of certain functions. This is important to track down the statistics after a code re factoring or to evaluate if changes to one part of application can have unintended consequence on another.

    I have come across JunitPerf which might help me to achieve this objective.

Suggested Topics

  • 2
  • 2
  • 2
  • 3
  • 2