Student Contribution

SC Conference - Activity Details

Using Overlays for Efficient Data Transfer over Shared Wide-area Networks

Gaurav Khanna  (Ohio State University)
Umit Catalyurek  (Ohio State University)
Tahsin Kurc  (Ohio State University)
Rajkumar Kettimuthu  (Argonne National Laboratory)
P. Sadayappan  (Ohio State University)
Ian Foster  (Argonne National Laboratory)
Joel Saltz  (Ohio State University)
Papers Session
Grid Virtualization and Overlays
Thursday,  01:30PM - 02:00PM
Room Ballroom F
Data-intensive applications frequently transfer large amounts of data over wide-area networks. The performance achieved in such settings can often be improved by routing data via intermediate nodes chosen to increase aggregate bandwidth. We explore the benefits of overlay network approaches by designing and implementing a service-oriented architecture that incorporates two key optimizations -- multi-hop path splitting and multi-pathing -- within the GridFTP file transfer protocol. We develop a file transfer scheduling algorithm that incorporates the two optimizations in conjunction with the use of available file replicas. The algorithm makes use of information from past GridFTP transfers to estimate network bandwidths and resource availability. The effectiveness of these optimizations is evaluated using several application file transfer patterns: one-to-all broadcast, all-to-one gather, and data redistribution, on a wide-area testbed. The experimental results show that our architecture and algorithm achieve significant performance improvement.
The full paper can be found in the IEEE Xplore Digital Library and ACM Digital Library
   IEEE Computer Society  /  ACM     2 0   Y E A R S   -   U N L E A S H I N G   T H E   P O W E R   O F   H P C