|
Software Testing Tutorial
November 16, 2005
Organized by IBM Research Lab in Haifa, Israel
Testing Multithreaded and Distributed Applications
Eitan Farchi and Shmuel Ur
Full-day tutorial with hands-on session
The industry's newest proven techniques and tools for reviewing and testing multithreaded applications will be presented in this tutorial, along with hands-on experience in their use. We will begin with an overview of why multithreaded code is so difficult to develop, and present, in some detail, the bug patterns commonly found in such code. We will then introduce the interleaving review technique (IRT), a very effective, scenario-based review technique for exposing scheduling problems. A review exercise in which participants will use concurrent bug patterns to find deadlocks and race conditions will conclude the first part of the tutorial.
The second part of the tutorial will discuss effective unit testing of multithreaded programs that expose concurrent bugs such as deadlocks, and race conditions. This is of utmost importance: bugs in multithreaded code are usually found during system or stress testing, or at the customer site, often with catastrophic consequences. We will further explain how function and system testing of multithreaded programs can be improved. An exercise in test execution techniques and analysis will conclude the tutorial. In the course of the session, we will also touch on aspects of debugging and coverage that are unique to multithreaded and distributed applications.
The techniques taught in the tutorial are being applied in the industry to multithreading distributive and concurrent code. They are used on all forms of software from embedded micro code all the way to huge middleware applications.
The following topics are covered:
- Typical problems in designing, coding, inspecting and testing multithreaded code
- Design-level test scenario selection and representation
- Bug-pattern based review
- The Interleaving Review Technique (IRT)
- Roles in review: program counter, devil's advocate, and stenographer
- Devil's advocate guidelines
- Bug patterns
- How to represent the system state and the system changes when multi-threading, several CPUs, and asynchronous calls are used
- Testing tools
- Unit testing of concurrent programs
- Writing unit level multithreaded tests
- Fast creation of multithreaded testing harness
- A method for stressing units and evaluating multithreaded tests
- Function and system testing
- Writing system level multithreaded tests
- Making tests more effective
- Evaluating test quality
| |
|
