High-volume performance test framework using big data

Abstract

The inherent issues with handling large files and complex scenarios cause the data-driven approach [1] to be rarely used for performance tests. Volume and scalability testing of enterprise solutions typically requires custom-made test frameworks because of the complexity and uniqueness of data flow. The generation, transformation and transmission of large sets of data pose a unique challenge for testing a highly transactional back-end system like the IBM Sterling Order Management (OMS). This paper describes a test framework built on document-oriented NoSQL database, a design that helps validate the functionality and scalability of the solution simultaneously. This paper also describes various phases of planning, development, and testing of the OMS solution that was executed for a large retailer in Europe to test an extremely high online sales scenario. An out-of-the-box configuration of the OMS with the feature support for database sharding was used to drive scalability. The exercise was a success, and it is the world's largest IBM Sterling Order Management benchmark in terms of sales order volume, to date.