Stretch optimization for virtual screening on multi-user pilot-agent platforms on grid/cloud

Virtual screening has proven very effective on grid infrastructures where large scale deployments have led to the identification of active inhibitors for biological targets of interest against malaria, SARS or diabetes. Operating a dedicated virtual screening platform on grid resources requires optimizing the scheduling policy. The scheduling can be done at 2 levels; at site level and at platform level. Site scheduling is done at each site independently; each site is autonomous in its choice of job scheduling. Each site allocates time slots for different groups of users. Platform scheduling is done at group level: inside a time slot jobs from many users are allocated. Pilot agents are sent to sites and act as a container of actual users jobs. They pick up users jobs from a central queue where the second stage scheduling is done. In this paper, we focus on pilot-agent platform shared by many virtual screening users. They need a suitable scheduling algorithm to ensure a certain fairness between users. We have studied the scheduling of users jobs inside central queue and examined the relevance and impact of different scheduling policies (FIFO, SPT, LPT and Round Robin) on the user experience. Optimal criterion used in our research is the stretch, a measure for user experience on the platform. In a first step, we simulated the operation of virtual screening applications on the pilot-agent platform in order to compare the scheduling policies. According to simulation, SPT algorithm was shown to significantly improve scheduling performances. In a second step, the Shortest Processing Time (SPT) and Longest Processing Time (LPT) scheduling policies were implemented on a DIRAC pilot-agent platform at IFI in Hanoi and tested on EGI Biomed Virtual Organization. Experimental results are in good agreement with simulation and confirm that SPT algorithm significantly improves user experience. The relevance of our conclusions also extends to cloud computing. Indeed, cloud infrastructures are also characterized by limited machine availability.

[1]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[2]  Johan Tordsson,et al.  Modeling for Dynamic Cloud Scheduling Via Migration of Virtual Machines , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[3]  J. Moscicki.  Distributed analysis environment for HEP and interdisciplinary applications , 2003 .

[4]  T Maeno,et al.  PanDA: distributed production and distributed analysis system for ATLAS , 2008 .

[5]  N. Jacq,et al.  Grid-Enabled High-Throughput In Silico Screening Against Influenza A Neuraminidase , 2006, IEEE Transactions on NanoBioscience.

[6]  D S Goodsell,et al.  Automated docking of flexible ligands: Applications of autodock , 1996, Journal of molecular recognition : JMR.

[7]  T.C.E. Cheng,et al.  A state-of-the-art review of parallel-machine scheduling research , 1990 .

[8]  K. Perez.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment , 2014 .

[9]  Emmanuel Medernach,et al.  Workload Analysis of a Cluster in a Grid Environment , 2005, JSSPP.

[10]  Fang Wang,et al.  DIET — A Scalable, Robust and Adaptable Multi-Agent Platform for Information Management , 2003 .

[11]  Frédéric Vivien,et al.  Minimizing the stretch when scheduling flows of biological requests , 2006, SPAA '06.

[12]  Rajkumar Buyya,et al.  A Particle Swarm Optimization-Based Heuristic for Scheduling Workflow Applications in Cloud Computing Environments , 2010, 2010 24th IEEE International Conference on Advanced Information Networking and Applications.

[13]  M. Rarey,et al.  FlexX‐Scan: Fast, structure‐based virtual screening , 2004, Proteins.

[14]  Shantenu Jha,et al.  SAGA BigJob: An Extensible and Interoperable Pilot-Job Abstraction for Distributed Applications and Systems , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[15]  Mohd Shahir Shamsir,et al.  Performance comparison of priority rule scheduling algorithms using different inter arrival time jobs in grid environment , 2011 .

[16]  Johan Montagnat,et al.  Grid-enabled Virtual Screening Against Malaria , 2006, Journal of Grid Computing.

[17]  Gerhard J. Woeginger,et al.  A Review of Machine Scheduling: Complexity, Algorithms and Approximability , 1998 .

[18]  Dror G. Feitelson,et al.  Packing Schemes for Gang Scheduling , 1996, JSSPP.

[19]  N. Brook,et al.  DIRAC - Distributed Infrastructure with Remote Agent Control , 2003, ArXiv.

[20]  Eugene L. Lawler,et al.  Chapter 9 Sequencing and scheduling: Algorithms and complexity , 1993, Logistics of Production and Inventory.

[21]  Igor Sfiligoi,et al.  glideinWMS - A generic pilot-based Workload Management System , 2008 .

[22]  V. Srinivasa Rao,et al.  Modern drug discovery process: An in silico approach , 2011 .

[23]  Johan Montagnat,et al.  Multi-infrastructure workflow execution for medical simulation in the Virtual Imaging Platform , 2011 .

[24]  Cheng Wang,et al.  A Survey of Job Scheduling in Grids , 2007, APWeb/WAIM.

[25]  Ryan G. Coleman,et al.  Protein Pockets: Inventory, Shape, and Comparison , 2010, J. Chem. Inf. Model..

[26]  Tom Fifield,et al.  Integration of cloud, grid and local cluster resources with DIRAC , 2011 .

[27]  Günter Schmidt,et al.  Scheduling with limited machine availability , 2000, Eur. J. Oper. Res..

[28]  Allen B. Downey,et al.  A parallel workload model and its implications for processor allocation , 1996, Proceedings. The Sixth IEEE International Symposium on High Performance Distributed Computing (Cat. No.97TB100183).

[29]  Rajmohan Rajaraman,et al.  Online Scheduling to Minimize Average Stretch , 2004, SIAM J. Comput..

[30]  Francine Berman,et al.  Application-Level Scheduling on Distributed Heterogeneous Networks , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[31]  Martin Hofmann-Apitius,et al.  WISDOM-II: Screening against multiple targets implicated in malaria using computational grid infrastructures , 2009, Malaria Journal.

[32]  Martin Hofmann-Apitius,et al.  Large Scale In Silico Screening on Grid Infrastructures , 2006, ArXiv.

[33]  Eugene L. Lawler,et al.  Sequencing and scheduling: algorithms and complexity , 1989 .