There are a lot of new features in vSphere 4.1 and I covered the major ones in my "Best of vSphere 4.1 for vNerds" mini-training course (free with any Train Signal vSphere purchase). Today, I want to take the opportunity to release one of those videos to the world and use it to educate on the topic of vSphere Storage I/O Control (or SIOC).
Are you able to identify precisely which processes are sucking up resources and slowing down your servers? Can you do this equally well over VM guests that VMotion?
OpManager also allows admins to remotely shut down problem-causing processes. With over 500 built-in monitors & 70 deep VMware metrics reported on, OpManager is one of the most comprehensive fault & performance management solutions available today for entire server infrastructure - both physical and virtual.
Background on vSphere Resource Controls
To understand vSphere SIOC, you first need to understand resource controls in vSphere. To control resources between VMs, vSphere uses the concept of "shares". A share is piece of all the CPU and RAM resources that are available. Thus, whether you have a single server or a group of servers in a DRS (distributed resource scheduler) load-balanced cluster, you will still have a total amount of resources and then each VM will get a piece of those resources (the share). By default, they will all get the same share of resource but you, as a VMware Admin can determine whether some applications on some VMs are more important than others and set the share value for those VMs higher.
The problem with shares is that, until vSphere 4.1, they didn't apply to storage as the shared storage was between the ESX/ESXi hosts and those hosts weren't communicating with each other to determine who had more shares of the storage resources. vSphere 4.1 comes with a new feature called Storage I/O Control that provides a kind of Quality of Service (QoS), which essentially ensures that all virtual machines get the storage performance they need. It does this by preventing a single VM from monopolizing storage resources over other virtual machines.
(Instructional video below provides a walkthrough of the steps contained in this article.)
How Storage IO Control (SIOC) Works
Figure below illustrates how Actual Disk Resources are utilized in a previous version of vSphere vs. version 4.1, where Storage I/O Control is employed. Older versions already used what are known as “disk shares”.
In both scenarios shown in the figure, you have two ESX Servers communicating with a Storage Array. One ESX server has two VMs, while the second has only one. Also, in both scenarios, VM A is supposed to have the largest space with 1500 shares, while both VM B and VM C only have 500.
In the older version of vSphere, you can see that there is no clear communication between ESX Servers regarding the disk shares found in them. As a result, because VM C is the only VM in one ESX Server, it subsequently ends up occupying a larger space in the Storage Array Queue than VM A, even if VM A has a much larger amount of shares.
As shown in the scenario on the right, Storage IO Control is able to resolve that issue and each VM is given an appropriate allocation of disk resources.
Now that you know what benefit comes with Storage IO Control (SIOC), let us proceed to talk about what you need to prepare in order to use this new feature.
Preparing for SIOC
First of all, you need to have at least version 4.1 of vSphere to use SIOC. Once you have that, note that Datastores will have to be managed by a single vCenter Server. That means SIOC won’t work if you have multiple vCenter Servers talking to the same Datastores. Note also that NFS and RDM are not supported; only FC and iSCSI. Datastores with multiple extents are likewise not supported.
Make sure you have resolved these issues before configuring Storage IO Control.
How to enable SIOC in vSphere 4.1
Now that everything’s set, head on to your vSphere client. In the Inventory section, select Datastores.
Once you’re brought to the next window, look at the list of datastores in the left panel. You’ll want to prioritize datastores that are expected to experience the most contention. In all likelihood, that’s going to be the shared storage datastores; i.e., datastores being shared by multiple virtual machines. In the screenshot below, it’s the IomegaSAN which has two datastores (LUN1 and LUN2).
Next, select a datastore, click its corresponding Configuration tab, and then click the Properties link to bring forward the Properties window as shown below.
You can now enable the Storage I/O Control by simply checking the check box labeled Enabled. Click the Close button when you’re done.
You’ll know you were successful if, in the vShpere client window, you now see Enabled under the Storage I/O Control item of the Datastore Details section.
Do the same thing on your other shared storage datastores.
How to configure the shares and IOPs of the VMs on those datastores
The next step is to configure the shares and IOPs (this refers to the number of I/O per second) of the virtual machines under the datastores which have been enabled for SIOC.
Select a datastore. In the screenshot below, for example, it’s the IomegaSAN-LUN1. Click the corresponding Virtual Machines tab to reveal the VMs on that datastore.
If you scroll to the right, you’ll notice that the Shares Value and Limit - IOPs columns are empty. By default, the Share Value of each virtual machine is 1000, while the IOPs Limit is unlimited.
To configure the shares and IOPs, scroll back to the left and right-click on a VM name. When the context menu appears, select Edit Settings.
When the Virtual Machine Properties window appears, go to the Resources tab, and select Disk. In the corresponding panel on the right, you’ll notice that the settings for a disk are as follows: Shares - Normal, Shares Value - 1000, Limit - IOPs - Unlimited.
If we set the Shares to High (just click and select from the list as shown below), the Shares Value will subsequently change to 2000. Assign what you think is the appropriate amount of shares for that particular VM. This is where you can come back in the future if you want to change the Shares Values and IOPs limits.
Once you’re done, just click the OK button to go back to the Virtual Machines tab
When you’re back at the Virtual Machines tab, you can verify the changes by scrolling again to the right.
In case you haven’t noticed, not only is this datastore serving multiple virtual machines, it is also serving multiple ESX Servers. That’s because those VMs reside in different ESX Servers. This is exactly the kind of scenario illustrated in the first figure on this article.
So Storage IO Control is going to use those Share Values if contention occurs, to prevent a single VM from negatively affecting the performance of other VMs connected to the same datastore.
Where to monitor SIOC performance
You will want to monitor the disk latency and IOPs of your datastore to see if adjustments have to be made. To monitor, just click the Performance tab of the selected datastore. Next, select Performance from the drop-down list beside the View option.
The View should then change to something like this:
Here are samples of the kind of information you’ll be able to see on that page.
These graphs will help you determine whether changes have to be made in the shares per virtual machine or the maximum number of IOPs.
As soon as you have more than one ESX/ESXi host connecting to the same shared storage, the resource controls you have put into place go out the window as a storage I/O bottleneck can easily occur. New in vSphere 4.1, VMware's Storage IO Control (SIOC) offers a solution to that problem. I hope you'll watch the how-to video above to learn more.