Do you need to re-verify all bugs in the staging environment?



  • At my company the QA department will verify all of the bugs fixed in builds deployed to our QA environment. We might deploy like 10 or so builds over a few weeks and they verify that the bug fixes are in fact fixed. All of this seems fine.

    When we deploy the final build to staging they go and re-verify all of the bugs again. I'm not convinced that is necessary. It's the exact same code they tested in the QA environment. Maybe poke around to make sure everything installed correctly but why re-verify every single bug again? I repeat - it's the same code.

    Someone help me understand.



  • As a general rule, retesting in the staging environment is done more as an integration/sanity test than a full retest because the QA environment will likely have different code (due to other changes that aren't being pushed to the staging environment yet).

    The usual considerations are:

    • The staging environment is kept as close to production as possible, and should differ only when code has been pushed to staging in preparation for deployment.
    • The QA environment has code from multiple change sets, not all of which will be targeted for deployment at any particular time (particularly when a change is part of a larger project that has to be deployed as a single chunk).
    • There is a small but not negligible chance that the difference between the two code bases will cause integration problems.
    • There is a small but not negligible chance that there are changes which are not in the change control system (This happens a lot where I work - there are data updates which happen regularly, and change control is not designed to handle things like what data is contained in which tables. Sometimes the data updates are handled as a script, but sometimes they aren't - things like bulk updates to state tax rates are generally dealt with as a manual data pump because the fields that change are different every time).
    • The continuous integration system might not handle database schema updates or stored procedure/function/user defined data type changes. These updates can be handled as SQL scripts stored in the version control system easily enough, but applying the database changes isn't something every CI system does.
    • I've yet to meet a test person who wasn't at least a little bit paranoid about bugs escaping into the wild. If there's anything different, most testers are going to want to recheck it to make sure nothing's broken.

    In my experience the testing that happens in the staging environment is pretty cursory compared to what happens in the QA/Test environment, but it's never skipped.



Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2