Adaptive virtual distributed environments for shared cyberinfrastructures

A shared distributed cyberinfrastructure is formed by federating computation resources from multiple domains. Such shared infrastructures are increasing in popularity and are providing massive amounts of aggregated computation resources to a large numbers of users. Meanwhile, virtualization technologies, at machine and network levels, are maturing and enabling mutually isolated virtual computation environments for executing unmodified parallel and distributed applications on top of such a shared physical cyberinfrastructure. In this dissertation, we go one step further by supporting runtime adaptation of virtual computation environments as integrated, active entities. More specifically, driven by both dynamic availability of infrastructure resources and dynamic application resource demand, a virtual computation environment is able to automatically relocate itself across the infrastructure and adjust its share of infrastructural resources. Such runtime adaptation is transparent to both users of a virtual environment and applications running inside the environment, providing the look and feel of a private, well-provisioned computational environment to the users and the applications. This dissertation presents the design, implementation, and evaluation of a middleware system, VIOLIN, which enables adaptive virtual computational environments. Each VIOLIN virtual environment is composed of a virtual network of virtual machines and is capable of live migration across a multi-domain physical infrastructure. This research effort includes the development of the enabling mechanisms for virtual environment creation and adaptation, as well as, policies for effectively utilizing these mechanisms. Through this effort, we have found that by combining virtual environments with adaptation policies and mechanisms, we can provide significant increases in both usability and performance of shared computational infrastructures. Further, the VIOLIN middleware is currently deployed on a production cyberinfrastructure, called the nanoHUB, as a major computational facility used to execute parallel and distributed nanotechnology simulations.