Introductory: 30% Intermediate: 40% Advanced: 30%

SC Conference - Activity Details

S04: Parallel I/O in Practice

Robert Ross  (Argonne National Laboratory)
Robert Latham  (Argonne National Laboratory)
Marc Unangst  (Panasas)
Brent Welch  (Panasas)
Tutorials Session
Sunday,  08:30AM - 05:00PM
Room TBD
I/O on HPC systems is a black art. This tutorial sheds light on the state-of-the-art in parallel I/O and provides the knowledge necessary for attendees to best leverage I/O resources available to them. We cover the entire I/O software stack from parallel file systems at the lowest layer, to intermediate layers (such as MPI-IO), and finally high-level I/O libraries (such as HDF-5). We emphasize ways to use these interfaces that result in high performance, and benchmarks on real systems are used throughout to show real-world results. This tutorial first discusses parallel file systems in detail (PFSs). We cover general concepts and examine five examples: GPFS, Lustre, PanFS, PVFS, and NetApp GX. Next we examine the upper layers of the I/O stack, covering POSIX I/O, MPI-IO, Parallel netCDF, and HDF5. We discuss interface features,show code examples, and describe how application calls translate into PFS operations. Finally we discuss I/O best practice.
   IEEE Computer Society  /  ACM     2 0   Y E A R S   -   U N L E A S H I N G   T H E   P O W E R   O F   H P C