Is it necessary to test third party tools?
I have a question about best practices for writing test cases. At my job, I am using a closed source, third party tool that parses files into a database. Since this is a third party tool, is it necessary to write a test suite for it? Should we limit our tests to only testing our own code or should we also test third party code in an attempt to increase reliability. We are testing the data for accuracy before it hits the parser, and the data is then tested subsequently once it is pulled out of the database and used in other parts of the system. We have been using the tool in development for about a month with no unexpected results, so we have the smoke test covered, now I'm wondering about the need for more formal testing.
At some point you have to trust the third-party software you work with. That's not to say the software won't have bugs; of course it will. But you can never test everything. For example, I suspect you do not test the operating system, compilers, text editors, router firmware, printer drivers, and web browsers that you depend upon. (At a previous job, our source code was occasionally corrupted by a faulty network stack. Essentially, the act of checking code in could cause bugs (or compile errors) to creep in. Fortunately that was a long time ago.) If you have a history of problems with the tool, or if you believe you use the tool in an unusual way that may not have been covered by the vendor, then it's worth considering some kind of acceptance test of those specific features for each new version of the tool. An acceptance test may also be justified if you suspect the tool could cause otherwise undetectable errors. Otherwise, I would trust the tool.