Is there a better way of carrying out basic text and form validation instead of doing it at E2E?
carriann last edited by
My understanding of E2E tests is that we should be carrying out tests that run through the whole system to check if everything works together via Selenium (for a web application). However, I've seen places where they would consider the following test as part of E2E:
- Navigate to a certain page (which would take 1/2 minutes to execute) then check to see if the page has the correct title, paragraph, form titles etc.
- Navigate to a certain page (which would take 1/2 minutes to execute) then check to see if a certain form shows/hides correct validation messages e.g. entering a character in the DOB field.
My understanding is that, as the application grows this will become very expensive to maintain and execute. It almost seems like we are shifting right, and pushing all our tests to the GUI level when they can be executed lower down the stack.
- Is my understanding correct in that these tests should not be part of E2E?
- Can these tests be executed lower down the stack, is so at what level? For example, surely the developers can mock the data needed to load a specific page, and even then they could use selenium if they really wanted to check if the form was working, or if the page was displaying the correct data. Otherwise can such tests listed in my example be tested at unit level?
The perfect answer will help clear my understanding on where these tests should be executed and how. Any examples related to web applications written in Angular with a choice of potential tools will be perfect. Thank you.
Let's suppose you have the situation where you have a page that loads a list of items names created in the last 24 hours, on the DOM tag ul#recent_items (ul with id recent_items).
It will probably be implemented like:
- (1) There is a component that represents this list, which makes a call to a service, organizes the data, and renders it nicely using the frontend framework (React, Ember, etc...) methods.
- (2) A reference to this (1) component on the page, placing it with the #recent_items id.
- (3) There is a endpoint
/items?filter=recentwhich returns the items filtering only the recent ones.
This can be broken in many types of checks:
- For (1), you can mock the service and render only this particular component, checking if it handles the data you provided on the test correctly, UI-wise. Example here.
- For (2), you can build your frontend app locally and visit directly the page, mocking anything service call that is necessary. Here, you will check only that the component at #recent_items is there - no UI check. Example here.
- For (3), you can make a HTTP call to the endpoint and check what is the data returned. Example here. This will depend on how this endpoint is coded, of course. Database setup, unit tests, etc, apply here in order to break this big service test further down.
- Additionally, it is necessary to inform the service how it is expected to be used. This is call Consumer-Driven Contract Testing. Basically the unit tests of the consumers (in this case the frontend app) will be registered and the provider (the service) will check how itself behaves with this usage. For instance, the frontend unit tests could mock a call to
/items?filter=recent24hours- and if the service only provides
/items?filter=recent, it will not integrate well. The Contract Test would fail and the service team would decide how to handle this - either changing the endpoint or talking with the developers of the frontend app.
The E2E checks would serve to verify if the building process was acceptable. The four checks above are local checks, the system are not integrated during the test (although the Contract Test checks the integration, functionality-wise). If the build pipeline indicates that the frontend will call the service at the IP XYZ, but the service is at TVX, it will not work.
Now, talking about more software engineering-wise, understanding the necessary checks after the implementation is quite difficult, because the bundle is already done and integrated. E2E seems very tempting, because it is straightforward. So, it would be easier to build these smaller checks while you build the components themselves (the frontend, the services, etc), rather than after the whole feature is done.