About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
WHPCF 2010
Conference paper
Adding stream processing system flexibility to exploit low-overhead communication systems
Abstract
Previously, we demonstrated that we can build a real-world financial application using a stream processing system running on commodity hardware. In this paper, we propose making stream processing systems more flexible and demonstrate how this flexibility can be used to exploit low-overhead communication systems to speed up streaming applications. With our prototype, we now have an options market data processing system that can achieve less than 30 μsec average latency at 30x the February 2008 OPRA rate on a cluster of blades using InfiniBand. Across shared memory, this system can achieve less than 20 μsec average latency at 25x the February 2008 OPRA rate on a single machine. © 2010 IEEE.