Automation data setup: via SQL or via APIs?
irl last edited by
I've had this discussion recently and would appreciate some more opinions and experiences of this community.
When setting up data for integration tests (API or UI), is the favored approach via SQL or via API calls?
Pros for doing API calls:
- if business logic or database changes, data will be treated accordingly and still be correctly inserted
- no need to maintain separate scripts
- realistic as this is how data is added in production as well
Pros for using SQL scripts:
- if the API contains bugs, you won't be blocked during setup already (no dependency on other API endpoints)
- easier to get data into a specific state
- full control over which data is inserted
Additional question When using APIs, I'm kind of worried that tests will inevitably become end-to-end (rather than integration) with too many points of failure. Let me clarify with an example, supposing we do setup via API calls.
- Create customer test: calls CreateCustomer route
- Create order test: calls CreateCustomer > CreateOrder routes
- Create invoice test: calls CreateCustomer > CreateOrder > CreateInvoice routes
... And so on. If CreateCustomer fails, then all tests fail while they shouldn't (as merely used as setup step in most).
What if the datastore changes and the API stays the same? Tests should help developers to refactor safely. This includes changing data storage types, maybe even to flat files or another API for storage. I prefer to use the least implementation details as possible for this reason.
The API the test-fixtures use do not have to be an web-service, but could also be a create user class used by the web-api.
When using APIs, I'm kind of worried that tests will inevitably become end-to-end (rather than integration) with too many points of failure.
Using the database means it is sorta end-to-end, you need a fully deployed and working system for your test. Else you could mock the database for example.
These points of failure should be fixed asap, for example by a rollback if broken. Isn't it the general idea that tests detect issues in API's as soon as possible?