SC Conference - Activity Details

Gaurav Khanna  (Ohio State University)

Umit Catalyurek  (Ohio State University)

Tahsin Kurc  (Ohio State University)

Rajkumar Kettimuthu  (Argonne National Laboratory)

P. Sadayappan  (Ohio State University)

Ian Foster  (Argonne National Laboratory)

Joel Saltz  (Ohio State University)

Grid Virtualization and Overlays

Thursday,  01:30PM - 02:00PM

Room Ballroom F

Data-intensive applications frequently transfer large amounts of data over wide-area networks. The performance achieved in such settings can often be improved by routing data via intermediate nodes chosen to increase aggregate bandwidth. We explore the benefits of overlay network approaches by designing and implementing a service-oriented architecture that incorporates two key optimizations -- multi-hop path splitting and multi-pathing -- within the GridFTP file transfer protocol. We develop a file transfer scheduling algorithm that incorporates the two optimizations in conjunction with the use of available file replicas. The algorithm makes use of information from past GridFTP transfers to estimate network bandwidths and resource availability. The effectiveness of these optimizations is evaluated using several application file transfer patterns: one-to-all broadcast, all-to-one gather, and data redistribution, on a wide-area testbed. The experimental results show that our architecture and algorithm achieve significant performance improvement.

The full paper can be found in the IEEE Xplore Digital Library and ACM Digital Library