Gradwell VMWare Platform

Gradwell relies heavily on server virtualisation, using the industry-leading VMWare ESX platform. All of our services run on the ESX platform, and rely on virtualised storage running on dedicated networked storage servers.

In December 2007, Gradwell made the strategic decision to migrate all products and services to run in a virtualised environment. We chose VMWare ESX (now known as VMWare vSphere) as the virtualisation platform, as it is the clear market leader and is backed by a multi-billion dollar US corporation.

Our VMWare ESX platform looks like this:

In a virtualised environment, physical servers run VMWare's ESX software, and are known as hosts. Each host in turn runs multiple virtual servers, known as guests or virtual machines.

We have standardised on Hewlett Packard DL380 servers as our hosts. Each server has:

• 8 power-efficient Intel Xeon CPU cores
• min 16GB RAM
• redundant power supplies
• redundant network links
• full integrated lights-out (iLO) remote management

We will be adding HP blade servers to provide additional processing capacity in the near future.

The virtual machines aren’t stored on the disk drives inside the host servers; instead they are stored on dedicated networked storage servers. This is best practice, and is intended to make it easier to get services back up and running when an individual host suffers a hardware failure.

We have split our enterprise storage up into two different types - clustered storage for customer data, and NFS fileservers for clustered virtual servers.

Customer data is stored on a cluster of LeftHand SAN units. We have chosen LeftHand (now part of HP) for the high-availability and performance that it offers.

• The cluster presents itself to VMWare as a single iSCSI device.
• Whenever VMWare saves data to the cluster, each piece of data is written to a minimum of two individual LeftHand SANs.
• If one LeftHand SAN is offline for any reason (e.g. planned maintenance, hardware failure), the rest of the cluster continues to serve up and save data from the other copy of the data.
• Each individual SAN has 12 disks setup in a RAID configuration, to ensure that data is not lost when individual hard drives fail.

The result of this unique approach is that customer data is safe and always available, even if individual storage servers are unavailable.

We currently have over 3TB of customer data on our clustered SANs.

Virtual machines are stored on a pool of non-redundant NFS servers.

• Each NFS server is a HP DL160 with an attached disk shelf, giving it a total of 29 hard drives (4 in the DL160, and another 25 in the disk shelf).
• The disks inside each NFS server are setup in a RAID configuration, to ensure that data is not lost when individual hard drives fail.
• Each NFS server is presented to VMWare as a separate server.
• Each disk shelf is attached to two DL160's, allowing us to keep the storage online even if one of the DL160's is offline for any reason (e.g. routine maintenance).
• We do not store unique data on any of the NFS servers. All customer data is stored on our clustered SANs.
• Each virtual machine stored on the NFS servers is part of a clustered service (e.g. the PHP 5.2 Cluster). If a single NFS server fails, the clustered service will continue to work, using the virtual machines still running on the remaining NFS servers.

We currently have over 12TB of storage in our NFS server pool.

Our VMWare hosts and our enterprise storage is currently connected to each other via the Gradwell Core Network.

• Our core network is built around two stacks of Cisco 3750G's. We have one stack in each of our datacenters.
• Each datacenter has separate switched networks for data and storage, using a mixture of Cisco and HP gear.

The vast majority of our products and services run on our VMWare platform, or are in the process of being migrated onto the platform from legacy hardware.

So far, the only service we offer that does not function well under VMWare is Voice-over-IP (VoIP) audio handling (NAT proxies, our hosted PBXs, and our conference room servers). We continue to run these services on dedicated hardware.