Distributed and Cloud Computing: Keep virtual clusters manageable

Virtualization and virtual clusters enable cloud providers to maximize efficiency, but they also beget challenges. Learn how to tackle them in this free chapter download.

Virtualization alone doesn't define cloud computing, but it plays a vital and often challenging role in how service providers build and manage their cloud environments. Virtualization and virtual server clusters enable providers to maximize the efficiency of their physical infrastructure by supporting multi-tenant environments within individual servers, storage devices and network appliances. But what makes virtualization and virtual clusters so beneficial also begets their biggest challenges, especially in large-scale cloud provider environments. 

This excerpt from the third chapter of Distributed and Cloud Computing, "Virtual Machines and Virtualization of Clusters and Data Centers," offers cloud providers a deep dive into virtualization levels, virtual machine (VM) architectures, virtual networking, virtual cluster construction and virtualized data center design and automation in cloud computing.

Adapted from Distributed and Cloud Computing: From Parallel Processing to the Internet of Things, by Kai HwangJack Dongarra and Geoffrey C. Fox (Published by Morgan Kaufmann):

3.4 - Virtual clusters and resource management in cloud computing

A physical cluster is a collection of servers (physical machines) interconnected by a physical network, such as a LAN. In Chapter 2, we studied various clustering techniques on physical machines. Here, we introduce virtual clusters and study their properties as well as explore their potential applications. In this section, we will study three critical design issues of virtual clusters: live migration of VMs, memory and file migrations, as well as the dynamic deployment of virtual clusters.

When a traditional VM is initialized, the administrator needs to manually write configuration information or specify the configuration sources. When more VMs join a network, an inefficient configuration always causes problems with overloading or underutilization. Amazon's Elastic Compute Cloud (EC2) is a good example of a Web service that provides elastic computing power in a cloud. EC2 permits customers to create VMs and to manage user accounts over the time of their use. Most virtualization platforms, including XenServer and VMware ESX Server, support a bridging mode that allows all domains to appear on the network as individual hosts. By using this mode, VMs can communicate with one another freely through the virtual network interface card and configure the network automatically.

3.4.1 - Physical versus virtual clusters

Virtual clusters are built with VMs installed at distributed servers from one or more physical clusters. The VMs in a virtual cluster are interconnected logically by a virtual network across several physical networks. Figure 3.18 illustrates the concepts of virtual clusters and physical clusters. Each virtual cluster is formed with physical machines or a VM hosted by multiple physical clusters. The virtual cluster boundaries are shown as distinct boundaries.

Provisioning VMs to a virtual cluster is done dynamically and has the following properties:

  • The virtual cluster nodes can be either physical or virtual machines. Multiple VMs running with different operating systems (OSes) can be deployed on the same physical node;
  • A VM runs with a guest OS, which is often different from the host OS that manages the resources in the physical machine where the VM is implemented;
  • The purpose of using VMs is to consolidate multiple functionalities on the same server. This will greatly enhance server utilization and application flexibility;
  • VMs can be replicated in multiple servers for the purpose of promoting distributed parallelism, fault tolerance and disaster recovery;
  • The size (number of nodes) of a virtual cluster can grow or shrink dynamically, similar to the way an overlay network varies in size in a peer-to-peer (P2P) network; and
  • The failure of any physical nodes may disable some VMs installed on the failing nodes, but the failure of VMs will not pull down the host system.

Continue reading and download this chapter excerpt on virtual machines and virtual clusters from the book Distributed and Cloud Computing.

©2012 Elsevier, Inc. All rights reserved. Printed with permission from Morgan Kaufmann, a division of Elsevier. Copyright 2012. For more information on this title and other similar books, please visit www.mkp.com.

About the book:

From the leading minds in the field, Distributed and Cloud Computing is an overview of modern distributed models. The book exposes the design principles, systems architecture and innovative applications of parallel, distributed and cloud computing systems. It will teach you how to create high-performance, scalable and reliable systems -- providing comprehensive coverage of distributed and cloud computing, including the following topics:

  • Facilitating management, debugging, migration and disaster recovery through virtualization;
  • Clustered systems for research or ecommerce applications;
  • Designing systems as Web services;
  • Social networking systems using peer-to-peer computing;
  • Principles of cloud computing using examples from open-source and commercial applications; and
  • Cloud-based systems for research, e-commerce, social networking and more, which the text describes using examples from open source and commercial vendors.

About the authors:

Kai Hwang is a professor of computer engineering at the University of Southern California and an IV-endowed visiting chair professor at Tsinghua University in China. He holds a Ph.D. in electrical engineering and computing science from UC Berkeley. An IEEE Life Fellow, Hwang has written extensively about computer architecture, digital arithmetic, parallel processing, distributed systems, Internet security and cloud computing.

Jack Dongarra is a university distinguished professor of electrical engineering and computer science at the University of Tennessee; distinguished research staff at the Oak Ridge National Laboratory in Tennessee; and a Turning Fellow at the University of Manchester in the U.K. An ACM/IEEE/ SIAM/AAAS Fellow, Dongarra has written about and pioneered the areas of supercomputer benchmarks, numerical analysis, linear algebra solvers and high-performance computing.

Geoffrey Fox is a distinguished professor of informatics, computing and physics and serves as associate dean of graduate studies and research in the School of Informatics and Computing at Indiana University. He previously taught and led many research groups at the California Institute of Technology and Syracuse University. He received his Ph.D. from Cambridge University in the U.K. Fox is well known for his work and publications on parallel architecture, distributed programming, grid computing, Web services and Internet applications.

This was last published in November 2011

Dig Deeper on Telecommunication networking