How should I perform an E2E test for a report that updates only once per day?
I would like to perform an end-to-end test where I create some data via a UI and then verify it appears in the appropriate report the following day. The challenge is that the report only updates once per day at any point in time between 12AM and 8AM, so in the worst case I need to wait up to 8 hours from the time I create the data before it shows up on the report.
The options I see are:
- Schedule the test to run just before 12AM and let it run for up to 8 hours polling the report for the data. This is the most straightforward approach but it doesn't fit in well with our CI framework which is geared towards short jobs. I'll need to override a couple timeouts just to make this one work, and I'm not totally comfortable with it because it is such a long period of time for a single process to run a test.
- Run the test in two distinct steps. Step 1 completes after the data is created. Step 2 runs at 8AM and verifies the data from Step 1 appeared. This seems reasonable to me but it will now require my test process to manage durable state. I can imagine several ways to achieve this (a durable queue, file, etc) but they all introduce significant complexity. Do I just need to accept that complexity and store state for tests of this sort? What are some best practices or libraries that tend to be used for this?
Neither of these feel ideal to me, but I do not have a lot of experience in testing of this sort. Are there any better approaches?
We have a lot of this kind of report where I work. Our developers provide the following in order to test them on Dev/test servers:
A webpage that allows the report update/generation task to be run. For us, this means individual buttons that trigger Cron jobs that run the reports.
Another web page that allows the time to be changed for each individual service. This means we can tell the reporting service that it's tomorrow, so we don't have to wait until the scope of the report is correct for our data to show up.