The trickle-down effect: Web caching and server request distribution

Abstract

Web proxies and Content Delivery Networks (CDNs) are widely used to accelerate Web content delivery and to conserve Internet bandwidth. These caching agents are highly effective for static content, which is an important component of all Web-based services. This paper explores the effect of ubiquitous Web caching on the request patterns seen by other components of an end-to-end content delivery architecture, including Web server clusters and interior caches. In particular, object popularity distributions in the Web tend to be Zipf-like, but caches disproportionately absorb requests for the most popular objects, changing the reference properties of the filtered request stream in fundamental ways. We call this the trickle-down effect. This paper uses trace-driven simulation and synthetic traffic patterns to illustrate the trickle-down effect and to investigate its impact on other components of a content delivery architecture, focusing on the implications for request distribution strategies in server clusters. © 2002 Elsevier Science B.V. All rights reserved.