Laboratoire de l'Informatique et du Parallélisme » XLcloud

Cloud computing \cite{ArmbrustCloud:2009} has become an important building block to providing IT resources and services to organisations of all sizes. Amongst the claimed benefits of clouds, the most appealing derive from economies of scale and often include resource consolidation, elasticity, good availability, and wide geographical coverage. The workload consolidation enabled by virtualising physical resources and enabling customers to share the same physical infrastructure brings several advantages such as energy efficiency and increased utilisation. The on-demand business model practiced by most cloud providers permits customers to request resources as they need them and pay only for what they use.

This model suits most of today's use cases, but certain applications such as those that demand High Performance Computing (HPC) are hardly fully portable to this new usage model. Such applications are generally resource intensive and sensitive to performance variations. The press occasionally reports on examples of using clouds to perform HPC, but most reported cases are of bag-of-task applications and require almost Herculean effort to execute \cite{ec2supercomputer:2013}. A large part of HPC applications still demand homogeneity amongst computing nodes and predictable network performance.

The means employed by cloud providers to offer customers with high and predictable performance mostly consist in deploying bare-metal resources or highly specialised and low-overhead virtual machines placed in groups where high network throughput and low latency can be guaranteed. This model may seem in contrast with traditional cloud use cases, as it is expensive and provides little flexibility in terms of workload consolidation and resource elasticity. On the other hand, attempts to co-locate HPC applications on the same physical hardware have proven difficult.

Though the on-demand nature of clouds is interesting as it allows customers to request resources whenever they need, a large part of HPC users has been tolerant to resource availability as large clusters hosted by research organisations are often shared by a large group of users. That is to say, to gain exclusive access to a subset of resources, users submit a request that waits in a queue for a time that can be longer than the request execution itself. While we do not consider that clouds should adopt a such a similar queuing model, we argue that a compromise between wait time and on-demand access could be explored via resource reservations. Reservations are attractive as they provide means for reliable allocation and allow customers to plan the execution of their applications. Although interest for more flexible reservation in the cloud has been gaining momentum, the models currently in use generally rely on reserving resources in advance for a long time period (\textit{i.e.} a year) or bidding for virtual machine instances in a spot market.

In this paper, we describe a reservation framework for reserving and provisioning specialised HPC resources using a popular open source cloud platform. The proposed solution is implemented as a configurable component of OpenStack\footnote{https://wiki.openstack.org}.

@MISC{ec2supercomputer:2013,

  title = {18 Hours, \$33K, and 156,314 Cores: Amazon Cloud HPC Hits a Petaflop}, 

  howpublished={http://arstechnica.com/information-technology/2013/11/18-hours-33k-and-156314-cores-amazon-cloud-hpc-hits-a-petaflop/},

  month = {November},

  year = {2013}

}