Virtualizing Exchange and Scaling Up

• Matt Liebowitz
• November 4, 2013November 4, 2013
• 3
• 1128

I have to admit – Exchange virtualization is one of my favorite topics.  I love talking about it with customers, colleagues, or anyone who will listen. It has a very well known workload profile which makes sizing consistent, it’s virtualization friendly in that it supports vSphere features like vMotion and HA, and I like talking about the challenges and creative solutions that are sometimes required to virtualize it. Yep, Exchange virtualization is my pal.

I’m also very familiar with its support limitations as I’ve written about them in the past (see here). One of the support policies has to do with CPU over-subscription. That is, assigning more vCPUs than there are physical pCPUs in the host. From Microsoft’s support policy page for Exchange 2013 virtualization (though this applies to Exchange 2010 also):

Many hardware virtualization products allow you to specify the number of virtual processors that should be allocated to each guest virtual machine. The virtual processors located in the guest virtual machine share a fixed number of logical processors in the physical system. Exchange supports a virtual processor-to-logical processor ratio no greater than 2:1, although we recommend a ratio of 1:1. For example, a dual processor system using quad core processors contains a total of 8 logical processors in the host system. On a system with this configuration, don’t allocate more than a total of 16 virtual processors to all guest virtual machines combined.

Note the section I bolded above – that says that you cannot go beyond a 2:1 vCPU to pCPU ratio and 1:1 is recommended. Not only are we used to much larger consolidation ratios, once you factor in DRS automatically moving VMs to balance out utilization it becomes almost impossible to enforce.

Combine this with the relative disparity between the maximum amount of RAM in a host compared to CPUs. We have customers that have deployed ESXi hosts that have 16-24 physical CPUs but 1TB of RAM. If you want to virtualize Exchange on those servers, you’re left with only two possible outcomes:

1) You ignore Microsoft’s support restriction and exceed the 2:1 vCPU/pCPU ratio.

2) You don’t use anywhere near the amount of RAM you have in the host.

The good news is there’s a third option that allows you to work around this problem: CPU reservations. CPU reservations in vSphere allow you to guarantee that important virtual machines will have access to the CPU resources they need even if CPU resources are constrained. This works great for virtualizing Exchange for a few reasons:

1) You can effectively exceed the 2:1 vCPU:pCPU ratio for workloads other than Exchange while guaranteeing Exchange VMs access to those CPU resources. Doing so does not violate Microsoft’s support policies – see this great TechEd presentation on virtualizing Exchange where Microsoft’s Jeff Mealiffe discusses this as an option.

2) Hosts that have a disproportionate amount of RAM to CPUs can still be utilized without wasting resources.

The other nice thing about CPU reservations is that when resources are not over-subscribed, CPU resources can be shared evenly with VMs that do not have a CPU reservation. In this way they work differently than memory reservations, which do not allow reserved memory resources to be shared with other VMs even if memory is plentiful. With CPU reservations, when resources are available then all VMs get equal access to the CPU. Once CPU resources become over-subscribed, the VMs with a reservation get guaranteed access to their reserved CPU resources while VMs without CPU reservations do not.

It’s worth mentioning that this post came about mostly as a result of a Twitter conversation that Chris Wahl (@ChrisWahl) brought me into recently. This is one of the reasons why I love Twitter – great conversation that led to sharing of information.

I hope this helps those that are looking to virtualize Exchange and are concerned about the CPU over-subscription limitations. Using CPU reservations in vSphere will let you stay compliant with Exchange support while allowing you to over-subscribe CPU resources for non-Exchange workloads.  Everybody wins!