Verifying an unlimited number of combinations



  • We are in the process of changing our barcode format to Code 128 and use all kinds of "special" ASCII characters in the barcodes to maximize the data we can encode in it. A barcode scanner works like a keyboard device and sends the string as key-presses to the application. The application itself can see the difference between normal key-presses and those of the barcode scanner and handles it accordingly and uses the data to fire an action.

    During exploratory (manual) testing I found two combinations that would scan incorrectly, they have been fixed. But now I am afraid that more combinations might give trouble in the future. My developers tell we have around 9.333 × 10^157 possible combinations. This is impossible to test manually. Testing a barcode means you need to print, scan it and verify the result.

    If we release this new format and a client finds a combination that doesn't work, it could lead to a fix that renders all (or a lot of) previous bar-codes invalid.

    Question: How should I handle a possible unlimited combinations like this, how far should I go in testing to feel comfortable enough to release a new feature like our new barcode format?

    Update:

    • The found issue was a Operating System keyboard setting issue ' + e leads to é instead of 'e
    • In the end we have automated with AutoIt and tested all up-to four character combinations after each other. And the Min/Max value boundaries.

    Learnings



  • If at all possible, this would be an ideal test to automate. If you decide to do that, below is an article by James Bach and Patrick Schroeder that is illuminating. The core idea of the article is that selection of a good pairing algorithm is not straightforward and a simple randomization approach is a good technique for getting effective coverage, whether doing manual or automated testing.

    Good luck! I hope this helps.

    http://www.testingeducation.org/wtst5/PairwisePNSQC2004.pdf



Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2