SC Conference - Activity Details

When Workflow Management Systems and Logging Systems Meet: Analyzing Large-scale Execution Traces

Dan Gunter  (Lawrence Berkeley National Laboratory)
Scott Callaghan  (University of Southern California)
Gaurang Mehta  (Information Sciences Institute)
Gideon Juve  (University of Southern California)
Keith Beattie  (Lawrence Berkeley National Laboratory)
Ewa Deelman  (Information Sciences Institute)
Posters Session
Tuesday,  05:15PM - 07:00PM
Room Rotunda Lobby
This poster shows the benefits of integrating a workflow management system with logging and log mining capabilities. By combing two existing, mature technologies: Pegasus-WMS and Netlogger, we are able to efficiently process execution logs of earthquake science workflows consisting of hundreds of thousands to one million tasks. In particular we show results of processing logs of CyberShake, a workflow application running on the TeraGrid. Client-side tools allow scientists to quickly gather statistics about a workflow run and find out which tasks executed, where they were executed, what was their runtime, etc. These statistics can be used to understand the performance characteristics of a workflow and help tune the execution parameters of the workflow management system. This poster shows the scalability of the system presenting results of uploading task execution records into the system and by showing results of querying the system for overall workflow performance information.
   IEEE Computer Society  /  ACM     2 0   Y E A R S   -   U N L E A S H I N G   T H E   P O W E R   O F   H P C