Specification-based testing using cause-effect graphs

Abstract

In this paper we discuss the advantages and limitations of a specification-based software testing technique we call CEG-BOR. There are two phases in this approach. First, informal software specifications are converted into cause-effect graphs (CEG). Then, the Boolean OperatoR (BOR) strategy is applied to design and select test cases. The conversion of an informal specification into a CEG helps detect ambiguities and inconsistencies in the specification and sets the stage for design of test cases. The number of test cases needed to satisfy the BOR strategy grows linearly with the number of Boolean operators in CEG, and BOR testing guarantees detection of certain classes of Boolean operator faults. But, what makes the approach especially attractive is that the BOR based test suites appear to be very effective in detecting other fault types. We have empirically evaluated this broader aspect of the CEG-BOR strategy on a simplified safety-related real-time control system, a set of N-version programs, and on elements of a commercial data-base system. In all cases, CEG-BOR testing required fewer test cases than those generated for the applications without the use of CEG-BOR. Furthermore, in all cases CEG-BOR testing detected all faults that the original, and independently generated, application test-suites did. In two instances CEG-BOR testing uncovered additional faults. Our results indicate that the CEG-BOR strategy is practical, scalable, and effective across diverse applications. We believe that it is a cost-effective methodology for the development of systematic specification-based software test-suites.