What processor instruction set is required in order to utilize virtualization software?

Hardware-Assisted Virtualization

Hardware-assisted virtualization is also called native virtualization, accelerated virtualization, or hardware VM, depending on the vendor. Hardware-assisted virtualization is a technology that allows for a CPU instruction set communication in which the VMM runs in a new root level mode below the OS kernel level. Support of this type of virtualization on Intel and AMD processors became available in 2006 but was first introduced on the IBM System/370 in 1972. In this type of virtualization, privileged and sensitive calls are set to automatically trap to the hypervisor. The binary translation used in full virtualization or the use of hypercalls in paravirtualization is no longer needed. Depending on the CPU manufacturer, the guest state is stored in either VM Control Structures (Intel) or VM Control Blocks (AMD). First-generation hardware-assisted technologies still lag behind in performance when compared to the full virtualization, but development of second-generation hardware-assisted technologies will improve virtualization performance while reducing memory overhead.

Figure 1.7 shows hardware-assisted virtualization concepts.

As a recap of the virtualization concepts discussed here, Table 1.1 is a comparison of the hardware virtualization types as listed in a white paper published by VMware titled “Understanding Full Virtualization, Paravirtualization, and Hardware Assist.”

Table 1.1. VMware's Summary Comparison of x86 Processor Virtualization Techniques

Alan Murphy of F5 published a white paper titled “Virtualization Defined – Eight Different Ways.” Several of the eight methods of virtualization outlined in the paper will be discussed in the next several sections.

Hardware virtualization techniques

Hardware-assisted virtualization. This term refers to a scenario in which the hardware provides architectural support for building a virtual machine manager able to run a guest operating system in complete isolation. This technique was originally introduced in the IBM System/370. At present, examples of hardware-assisted virtualization are the extensions to the x86-64 bit architecture introduced with Intel VT (formerly known as Vanderpool) and AMD V (formerly known as Pacifica). These extensions, which differ between the two vendors, are meant to reduce the performance penalties experienced by emulating x86 hardware with hypervisors. Before the introduction of hardware-assisted virtualization, software emulation of x86 hardware was significantly costly from the performance point of view. The reason for this is that by design the x86 architecture did not meet the formal requirements introduced by Popek and Goldberg, and early products were using binary translation to trap some sensitive instructions and provide an emulated version. Products such as VMware Virtual Platform, introduced in 1999 by VMware, which pioneered the field of x86 virtualization, were based on this technique. After 2006, Intel and AMD introduced processor extensions, and a wide range of virtualization solutions took advantage of them: Kernel-based Virtual Machine (KVM), VirtualBox, Xen, VMware, Hyper-V, Sun xVM, Parallels, and others.

Full virtualization. Full virtualization refers to the ability to run a program, most likely an operating system, directly on top of a virtual machine and without any modification, as though it were run on the raw hardware. To make this possible, virtual machine managers are required to provide a complete emulation of the entire underlying hardware. The principal advantage of full virtualization is complete isolation, which leads to enhanced security, ease of emulation of different architectures, and coexistence of different systems on the same platform. Whereas it is a desired goal for many virtualization solutions, full virtualization poses important concerns related to performance and technical implementation. A key challenge is the interception of privileged instructions such as I/O instructions: Since they change the state of the resources exposed by the host, they have to be contained within the virtual machine manager. A simple solution to achieve full virtualization is to provide a virtual environment for all the instructions, thus posing some limits on performance. A successful and efficient implementation of full virtualization is obtained with a combination of hardware and software, not allowing potentially harmful instructions to be executed directly on the host. This is what is accomplished through hardware-assisted virtualization.

Paravirtualization. This is a not-transparent virtualization solution that allows implementing thin virtual machine managers. Paravirtualization techniques expose a software interface to the virtual machine that is slightly modified from the host and, as a consequence, guests need to be modified. The aim of paravirtualization is to provide the capability to demand the execution of performance-critical operations directly on the host, thus preventing performance losses that would otherwise be experienced in managed execution. This allows a simpler implementation of virtual machine managers that have to simply transfer the execution of these operations, which were hard to virtualize, directly to the host. To take advantage of such an opportunity, guest operating systems need to be modified and explicitly ported by remapping the performance-critical operations through the virtual machine software interface. This is possible when the source code of the operating system is available, and this is the reason that paravirtualization was mostly explored in the open-source and academic environment. Whereas this technique was initially applied in the IBM VM operating system families, the term paravirtualization was introduced in literature in the Denali project [24] at the University of Washington. This technique has been successfully used by Xen for providing virtualization solutions for Linux-based operating systems specifically ported to run on Xen hypervisors. Operating systems that cannot be ported can still take advantage of paravirtualization by using ad hoc device drivers that remap the execution of critical instructions to the paravirtualization APIs exposed by the hypervisor. Xen provides this solution for running Windows-based operating systems on x86 architectures. Other solutions using paravirtualization include VMWare, Parallels, and some solutions for embedded and real-time environments such as TRANGO, Wind River, and XtratuM.

Partial virtualization. Partial virtualization provides a partial emulation of the underlying hardware, thus not allowing the complete execution of the guest operating system in complete isolation. Partial virtualization allows many applications to run transparently, but not all the features of the operating system can be supported, as happens with full virtualization. An example of partial virtualization is address space virtualization used in time-sharing systems; this allows multiple applications and users to run concurrently in a separate memory space, but they still share the same hardware resources (disk, processor, and network). Historically, partial virtualization has been an important milestone for achieving full virtualization, and it was implemented on the experimental IBM M44/44X. Address space virtualization is a common feature of contemporary operating systems.

Confirm Hardware Virtualization

You must have a computer with a CPU that has hardware virtualization capabilities to use Windows XP Mode. This should not be too difficult of a requirement to meet. Most modern processors will have this feature. If you are running a processor from Intel, you will need the Intel-VT feature. AMD processors must have the AMD-V hardware features. Microsoft has created a Hardware-Assisted Virtualization Detection Tool for your convenience. This tool can be downloaded at www.microsoft.com/downloads/details.aspx?FamilyID=0ee2a17f-8538-4619-8d1c-05d27e11adb2&displaylang=en or you can search for havdetectiontool.exe at the Microsoft Downloads Web site.

When you select the Download button, you will be asked to save or run in a window like Figure 9.4. Select Run.

Another security warning may appear as shown in Figure 9.5. Select Run to continue.

When the license agreement appears, read the license agreement and then click the I accept the license terms box as shown in Figure 9.6. Click Next to continue.

You will receive an informational window similar to Figure 9.7 that will tell you if your computer meets the virtualization requirements.

You could receive one of three possible messages:

This computer is configured with hardware-assisted virtualization. This is the message we are looking for. It tells us that we are ready to go!

Hardware-assisted virtualization is not enabled on this computer. This means your computer is capable of virtualization, but you will need to enable the hardware virtual assistance in the system BIOS. Please see the warning above before proceeding.

This computer does not have hardware-assisted virtualization. This message means that you cannot run Windows XP Mode or Virtual PC. You cannot use these features on your current computer.

You can now select if you want to share this information with Microsoft or not. Select your choice and click OK.

Hardware Assisted Virtualization

The base model for experimentation and evaluation are ARMv7 based development boards, hence the discussion focuses on the hardware assisted virtualization that ARM offers as part of ARMv7 and ARMv8 releases [109]. The RISC architecture of ARM helps achieve good balance of high performance, small code size, low power consumption, and reduced silicon area [56]. In addition, many smartphones and handheld devices have ARM based processors-with multiple cores. ARM architecture virtualization extensions and Large Physical Address Extension (LPAE) [118] enable efficient implementation of VM hypervisors for ARM architecture compliant processors.

To illustrate how ARM-based security framework works, i.e., TrustZone, Fig. 8.1 highlights the system architecture. TrustZone splits the mode into two worlds – secure and nonsecure. A special mode – monitor mode is used to switch between secure and nonsecure worlds. Although secure mode does not work in HYP mode, since trap and emulate support is not present, still we can run sensitive applications in the secure world. In [94], the authors discussed secure world and other details about CPU, memory, timer, and interrupt virtualization of ARM.

Microsoft Virtual Server

Microsoft Virtual Server is presented as a full suite of technologies for enabling comprehensive virtualized infrastructure. Microsoft Virtual Server's emphasis on virtualization technologies is rooted in creating what is called a dynamic IT environment. Microsoft's solution includes servers, desktops, and applications VM management and virtualization acceleration.

Hyper-V and Application Virtualization

Hyper-V provides software infrastructure and basic management tools in Windows Server 2008 that can be used to create and manage a virtualized server computing environment, as shown in Figure 2.2. It requires an x64-based processor, hardware-assisted virtualization, and hardware data execution protection. Microsoft offers preconfigured Virtual Hard Disks that can be downloaded and evaluated similar to the virtual appliance market.

Figure 2.2. Microsoft Virtual Server Hyper-V Manager

Installing Hyper-V

As you have seen so far in this chapter, Hyper-V is a versatile new feature for Windows Server 2008. Before explaining how to create, configure, and delegate hard drive space to virtual machines you must first be able to install the Hyper-V feature to your machine. Hyper-V has many prerequisites that must be met before installation can occur. We will now discuss the proper method for installing the virtualization features of Hyper-V.

In order to utilize virtualization you must have a clean install of Windows Server 2008 Enterprise with Hyper-V installed onto the host machine. Remember that Hyper-V requires 64-bit hardware and a 64-bit version of the operating system. You must also ensure that you have hardware-assisted virtualization enabled prior to installation. If BIOS configuration changes were made to the machine to enable these features, a full cycle power down must be completed to proceed. Once we are sure all these prerequisites are met, we can go over the steps required to install Hyper-V.

Configuring & Implementing…

Avoiding Common Bios Problems with Hyper-V

Hyper-V requires hardware-assisted virtualization. If you have an x64 Intel processor that supports hardware virtualization you should have no BIOS configuration issues. These processors come preconfigured for hardware virtualization. Hardware-assisted virtualization may need to be enabled on some processors. This may require altering the BIOS configuration of the host computer.

Intel Virtualization Technology and AMD Pacifica are both compatible with Hyper-V usage out of the box. However if you are using an x64 AMD processor there are known configuration issues that have yet to be addressed. If you are using this type of processor you should consult the online documentation located at http://techreport.com/discussions.x/13721 for your specific processor.

A BIOS update will more than likely be necessary. There is a fairly simple way of determining if you will need to update your BIOS in this case. You should check the “stepping” value for your processor using a CPU validation tool such as CPUID, which is available at http://www.cpuid.com. After running the tool consult the stepping value of your CPU. If the stepping value is 2 you will require a BIOS update. If the stepping value is 3 or higher you should have no need to update the BIOS to install Hyper-V to your machine.

This is the procedure for installing the Hyper-V role on Windows Server 2008:

Click Start and click to open Server Manager.

Locate the Roles node of the main Server Manager window.

Click Add Roles.

On the Specific Server Roles page click Add Hyper-V (as shown in Figure 10.2). Follow the prompts to finish installing the Role and click OK when finished.

Once installed, you will need to access the Virtualization Management Console to configure virtual machines. You can find the VMM within the Administrative Tools menu in the Start menu as seen in Figure 10.3. Once opened, you can use the Create Virtual Networks page to select the network adapters that you wish to make connections available to virtual machines. Also note that you should leave one network adapter free for remote access. Only Ethernet (802.3) network adapters are supported.

Click Install on the Confirm Installation Selections page.

Upon completion, click Close to end the installation.

Click Yes to restart the system.

Also note that when you open Hyper-V Manager for the first time, you must accept the end-user license agreement (EULA) using an account that is a member of the local Administrators group. If you fail to do so you will not be able to use the snap-in to perform any tasks. Be sure to log on to the computer using an appropriate account before you open the snap-in for the first time. If this problem occurs simply close Hyper-V Manager and log on to the computer using an appropriate account. Once you have done this you may open Hyper-V Manager without incident.

Server virtualization

Figure 1.2 is a simple depiction of how to get the most from your server hardware with Hyper-V. Building a Hyper-V virtual environment begins with a hardware platform designed for Windows compatibility. It must be capable of 64-bit operation and be virtual technology enabled. Installed on top of the hardware layer, and abstracting it from future virtual machines (VMs), is the hypervisor.

Warning

Not all processors are compatible with Microsoft Hyper-V. Processors must support hardware-assisted virtualization (i.e., Intel VT or AMD-V technology).

The hypervisor “decouples” hardware from the production operating systems running in the VMs. Configured and managed via the parent VM, it oversees hardware resources by

Supporting the creation and deletion of VMs

Managing memory access and security rules

Enforcing CPU usage policy

Scheduling and managing processor usage

Managing attached/installed device ownership

VMs in a Hyper-V world live in partitions. The first partition created contains the parent VM, which must run Windows Server 2008 ×64 or Windows Server Core. Once the parent partition is in production, you can create child partitions which contain your business server environments.

Note

All partitions share the following characteristics:

Each partition is configured with one or more virtual processors

Each partition participates in hardware resource sharing

Each partition hosts software known as a guest

2.1 General architectures

A hypervisor provides an efficient, isolated duplicate of the physical machines for virtual machines. Popek and Goldberg (1974) claimed that all sensitive instructions, i.e., those changing resource availability or configuration, must be privileged instructions in order to build an effective hypervisor for a certain system. In such an environment, all sensitive instructions cross the hypervisor, which is able to control the virtual machines appropriately.

This concept is today known as full virtualization and has the advantage that the host-operating system does not have to be adapted to work with the hypervisor, i.e., it is unaware of its virtualized environment. Obviously, a number of systems are far from perfect and require a number of additional actions in order to be virtualizable, leading to the technologies of paravirtualization, binary translation, and hardware-assisted virtualization (Pearce et al., 2013).

Paravirtualization encompasses changes to the system in order to redirect these nonprivileged, but sensitive, instructions over the hypervisor to regain full control on the resources. Therefore, the host-operating system has to undergo various modifications to work with the hypervisor, and the host is aware that it is virtualized. Applications running atop the altered OS do not have to be changed. Undoubtedly, these modifications require more work to implement, but on the other hand may provide better performance than full virtualization which often intervenes (Rose, 2004; Crosby and Brown, 2006). The best-known hypervisor of this type is Xen (Xen Project, n.d.).

Hardware-assisted virtualization is achieved by means of additional functionality included into the CPU, specifically an additional execution mode called guest mode, which is dedicated to the virtual instances (Drepper, 2008; Adams and Agesen, 2006). However, this type of virtualization requires certain hardware, in contrast to paravirtualization, which is in general able to run on any system. The latter also eases migration of paravirtualized machines. A popular representative for this virtualization type is the Kernel-based Virtual Machine (KVM) infrastructure (KVM, n.d.). Combinations of the two techniques are commonly referred to as hybrid virtualization.

Binary translation is a software virtualization and includes the use of an interpreter. It translates binary code to another binary, but excluding nontrapping instructions. This means that the input contains a full instruction set, but the output is a subset thereof and contains the innocuous instructions only (Adams and Agesen, 2006). This technology is also the closest to emulation, where the functionality of a device is simulated and all instructions are intercepted. The performance is dependent on the instructions to translate. VMware is an example of virtualization using binary translation (VMware, n.d.).

Hypervisors can also be distinguished by their relation to the host-operating system. In the case where the hypervisor fully replaces the operating system, it is called a bare-metal or Type I hypervisor, and where a host-operating system is still necessary, the hypervisor is hosted, or of Type II. Classifying the aforementioned hypervisors: Xen and KVM are both bare-metal—VMware ESX also, but its Workstation equivalent is hosted. Most workstation hypervisors are hosted as they are typically used for testing or training purposes.

Hyper-V system requirements and capacity planning

Prior to installing the role, you will need to ensure that your server hardware meets Hyper-V system requirements. Basic requirements are the same as the Windows Server 2008 R2 operating system; however, Hyper-V additionally requires your system to have the following features:

Hardware-assisted virtualization—Both the processor and BIOS must support virtualization. Intel refers to this feature as INTEL VT, while AMD refers to the technology as AMD-V. You will want to ensure that these features are supported and turned on in the system BIOS.

Data execution prevention (DEP)—DEP is a security technology available in most modern systems. You will want to ensure that this is also enabled in the BIOS.

After ensuring that the initial system requirements are met, you will want to properly plan for resource capacity of your Hyper-V hosts. This can be one of the most challenging aspects of planning. VMs require processor, memory, and disk space just like any other server. Your hosts will need to have enough resources to ensure that each VM can use its required resources when necessary. If you have existing physical servers that you plan on virtualizing, it may be helpful to capture performance data for a few weeks prior to moving the server to a VM. By capturing and reviewing performance data, you can better determine how much processing and memory power are required for a given server.

Notes from the field

If you happen to begin deploying VMs on a host with inadequate resources, you may find yourself in a situation where VMs experience poor performance or will not even start because of lack of capacity. Hyper-V does not support overcommitting memory, which means that if sufficient memory is not available, Hyper-V will not let you start the VM.

Notes from the field

As you plan for Hyper-V capacity, be sure to leave yourself some room for growth on your Hyper-V hosts. This will ensure that you can easily add new VMs without worrying about insufficient resources.

Kernel data

Much like the process entries, these files offer information about the kernel itself. Some of the objects may not be present on all systems, and will depend on how the kernel was configured, and whether all the required modules are loaded into memory.

/proc/cmdline – This file shows the arguments (Kernel Parameters, n.d.) passed to the Linux kernel at boot done. The information can be useful when comparing system states and different configurations between platforms with the same hardware and workloads that might be exhibiting different performance or application behavior. It is also useful to verify that certain boot time parameters have been correctly passed to the kernel.

/proc/config.gz – This file contains the kernel configuration in a compressed archive. The contents can be viewed using commands such as zcat or zgrep and dumped into a regular text file with redirection. The usefulness of the file is in that it contains all the configurations used to compile the running kernel, and they may not necessarily match the .config file under /usr/src/Linux. Again, this is very useful when comparing system behavior.

/proc/cpuinfo – This variable is a collection of CPU information, including model, CPU clock frequency, the number of physical and logical cores and threads. System administrators may want to consult the data if they need to debug issues related to CPU speed, hyperthreading, comparison between processor models, and more. The file also shows all the CPU flags and supported architecture-dependent extensions. For instance, if you need to use hardware-assisted virtualization (such as Intel VT-x), you can check whether the system supports this. Some of the flags may be masked by the BIOS/UEFI.

/proc/interrupts – The file records the number of interrupts per CPU, for each registered I/O device, as well as interrupts such as nonmaskable interrupts (NMI), TLB flush interrupts (TLB), and others. You should consult these data if you suspect one of the hardware resources may be malfunctioning, and to examine the performance of specific subsystems, such as network cards or the ATA interface, for instance. On systems with a very large number of processor cores, the output may be difficult to read, as it will almost inevitably overflow the console buffer. Sample output (shortened, for brevity):

/proc/kcore – This file represents the physical memory of the system (in ELF format). Using a debugger and unstrapped kernel binary, it is possible to read and examine the current state of kernel structures.

/proc/meminfo – Reports statistics about memory usage of the system, as we have briefly mentioned in the previous chapter. A large number of fields is reported (Documentation for /proc/sys/vm/*, n.d.), including totally usable RAM, free memory, buffers, caches, cached swap, amount of memory pages in different states, and more. The full listing is beyond the scope of this chapter, but some portions thereof can be useful when troubleshooting system issues.

Consider the following example. One of your users complains that they observe a degradation in responsiveness after a prolonged heavy I/O activity on one of their systems. You can confirm the symptoms on the affected single host, but you cannot immediately decide why it only comes to bear on that particular system and not the others, even though the engineers run the same kind of workload and I/O activity patterns.

Analyzing this kind of problem might not be easy, and a deep understanding of the system memory management facility is required, but if you know where to look, you could narrow down the investigation to a more manageable set of unknowns.

We will discuss some of the concepts later on, but let us very briefly deviate into the space of memory management. The Linux virtual memory (VM) subsystem is accessible under /proc/sys/vm. The tunables under this subtree govern the behavior of the memory when certain thresholds are reached. For example, the “dirty” parameters define when the write cache should be flushed to disk. The policies include a ratio of total memory, size, and expiration time.

An experienced system administrator may immediately see a possible source for the problem. Since some of the dirty parameters are expressed as a percentage of total memory, systems with different values – or even different physical memory size – will not react in the same way even when subject to seemingly the same workloads. The same policies apply to huge pages, caches, swap behavior, and more.

If you compare the system experiencing degradation under sustained I/O load to other systems that do not exhibit the same issue, you may discover different values under the vm tree, which could possibly explain the behavior.

Again, on their own, the values are not as useful as when compared to other systems, because they provide insights into how seemingly identical platforms are configured. This is a valuable tool for troubleshooting complex problems without a simple root cause. Moreover, if you make changes to the vm subsystem, you might want to observe and measure the results through the meminfo file.

/proc/modules – A list of all kernel modules currently loaded into memory. The same information is provided by the lsmod command.

/proc/mounts – The list of all mounted filesystems. The information can be correlated to the /etc/mtab file. On some systems, the latter may actually be a symbolic link to this pseudo-filesystem. The list is useful when troubleshooting issues related to network locations, mount and umount problems, stale mount points or hanging filesystems, and more.

/proc/slabinfo – This file provides information about kernel caches. Although the total size of slab pools is available in the meminfo variable, a higher level of details is available here. However, the contents are best parsed using a command such as slabtop or slabinfo.