The scope of automated API testing for a SQA tester



  • My situation:

    I am creating automated tests for testing a server's API endpoints. It is connected to a database and making requests to its API endpoints can result in it grabbing data from the database and, possibly, processing said data before returning it.

    Let's say, for example, that for one of its endpoints which accepts a POST request, the general flow is as follows:

    1. Send a configuration to the endpoint
    2. The configuration is stored in the database
    3. The server processes and returns data based on the passed-in configuration

    The POSTed configuration depends on an "instance" to already exist in the database. This dependency can be met by using the server's other API endpoints, or by prepopulating the database with my own queries.

    I want to avoid mocked unit level tests, which are the responsibility of the Software Engineers. That being said, how can I go about testing this endpoint at the scope of a SQA tester?

    My solution:

    There are 2 obvious ways that I see I could go about testing this endpoint:

    1. Directly -- Prepopulate the database using my own queries and avoid using the other API endpoints. Then call the specific API endpoint that I wish to test, and verify that its output is expected given the prepopulated database input and the configuration I created.
    2. Indirectly -- Use the server's other API endpoints to create a "flow test", testing this particular endpoint in a group of endpoints.

    Explanation:

    I believe both methods above are valid testing strategies that should be covered. The former is a bit involved in the sense that I must understand the database schema and have a general understanding of how it's being used, which results in a gray-box kind of testing. But prepopulating data also allows me to avoid test chaining and allows me to test the endpoints at a more granular level, but still not at a mocked unit level, as desired. It can also be argued that I am doing integration level testing, testing the integration of the server with the database, a kind of sandwich/hybrid integration test. The latter is only slightly less involved and tests at an integration level, which is obviously valuable for me as a SQA tester.

    Edit:

    Many responses mentioned that if I rely on prepopulating the database, my tests become prone to breaking due to database schema changes. I agree. However, in order to have complete test coverage of the server and its functionality, I've already created automated tests for the database schema. In fact, in my API endpoint tests, I am using some of the functionality from my schema tests to prepopulate my database. This is Bottom-Up testing. And, in an ideal world, my automated schema tests should be broken long before my automated API tests break.



  • I would personally go for approach 2 wherever possible.

    If the API itself is capable of making its own state changes, then it's a safer and more accepted way of doing things. If you make direct changes to the database, any bugs that you find as a result will inevitably be second guessed by the developers.

    The first approach is also more prone to breaking due to code changes. What if they rename a field in the database? You'd have to figure that out when your tests start failing.

    The first also creates a situation where you may not be sure that what you're testing is the reality of the API. What if the API starts storing data in a different set of database fields during a create event, but during a retrieval event it still accesses the old data location? That would lead to a false negative, where you think the API is working as intended, but it only works based on you directly inserting information into the database.


Log in to reply
 

Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2