SC Conference - Activity Details

Arifa Nisar  (Northwestern University)

Wei-keng Liao  (Northwestern University)

Alok Choudhary  (Northwestern University)

I/O and File Systems

Tuesday,  11:30AM - 12:00PM

Room Ballroom G

Increasingly complex scientific applications require massive parallelism to achieve the goals of fidelity and high computational performance. Such applications periodically offload checkpointing data to file system for post-processing and program resumption. As a side effect of high degree of parallelism, I/O contention at servers doesn't allow overall performance to scale with increasing number of processors. To bridge the gap between parallel computational and I/O performance, we propose a portable MPI-IO layer where certain tasks, such as file caching, consistency control, and collective I/O optimization are delegated to a small set of compute nodes, collectively termed as I/O delegate nodes. A collective cache design is incorporated to resolve cache coherence and hence alleviates the lock contention at I/O servers. By using popular parallel I/O benchmark and application I/O kernels, our experimental evaluation indicates considerable performance improvement with a small percentage of compute resources reserved for I/O.

The full paper can be found in the IEEE Xplore Digital Library and ACM Digital Library