SQL Server Min and Max in Virtualization

Scenario:

 

Guest VM Total Memory:  64GB

SQL Server Max Memory:  58GB (Leaving 10%, or 6.4GB roughly for Windows + Apps)

Guest VM Reservation:  48GB (75% of  Guest VM Total Memory;  in certain scenarios, 100% is preferred — if you can support it in failure situations)

SQL Server Min Memory:  43GB which gives 5GB to Windows + Apps.  48GB * 10% = 5GB to Windows + Apps, 43GB to SQL Min Memory

View Multiple esxtop Outputs Side-by-Side

The Problem

I often use esxtop to review realtime performance metrics of various aspects of my virtual infrastructure.  One annoying thing is that I cannot view multiple ESX hosts in a single esxtop session.  That would be such a nice feature to have, really.

The Solution

My solution is to use Linux screen command to accomplish side-by-side viewing of multiple esxtop output windows.

Screen does not come with ESX(I) but I have access to the ESX hosts from one of my management servers that runs a Linux distribution and has screen installed.  So I use PuTTy to access the Linux management server, and then from there launch screen and create two windows to then ssh to my ESX servers.

How I Did It

SSH to my Linux server and start screen.

  1. Created two windows (Ctrl-a c)
  2. Named each window (Ctrl-a A) ESX#
  3. Split the window vertically in screen (Ctrl-a |)
  4. On the first split, I SSH’d to the first ESX box
  5. Then Ctrl-a Tab to get to the second region of my vertical split
  6. Issue Ctrl-a 1 to access screen window #2
  7. SSH to second ESX server and run esxtop command.
 useful screen keyboard shortcuts
Ctrl-a Anew window
Ctrl-a nnext window
Ctrl-a pprevious window
Ctrl-a Ssplit terminal horizontally
Ctrl-a |split terminal vertically
Ctrl-a :resizeresize current region
Ctrl-a :fitfit screen size to new terminal size
Ctrl-a :removeremove region
Ctrl-a tabmove to next region
Ctrl-a Aset window title
Ctrl-aselect window from list
esxtop

esxtop in screen with vertical window split for side-by-side viewing of 2 ESX server’s esxtop output

Setting the Congestion Threshold Value for Storage I/O Control

This is a great KB article from VMware worth reposting.  I’m going to start analyzing storage I/O more today on all the arrays I have and LUNs.  Article Source:  http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1019687

Setting the congestion Threshold Value for Storage I/O Control

Details

The congestion threshold value for a datastore is the upper limit of latency that is allowed for a datastore before Storage I/O Control begins to assign importance to the virtual machine workloads according to their shares. You do not need to adjust the congestion threshold setting in most environments.
CAUTION Storage I/O Control will not function correctly unless all datatores that share the same spindles on the array have the same congestion threshold.

Solution

 If you do change the threshold setting, set the value based on the following considerations.
  • A higher value typically results in higher aggregate throughput and weaker isolation. Throttling will not occur unless the overall average latency is higher than the threshold
  • If throughput is more critical than latency, do not set the value too low. For example, for Fibre Channel disks, a value below 20 ms could lower peak disk throughput. On the other hand, a very high value (above 50 ms) might allow very high latency without any significant gain in overall throughput.
  • A lower value will result in lower device latency and stronger virtual machine I/O performance isolation. Stronger isolation means that the shares controls are enforced more often. Lower device latency translates into lower I/O latency for the virtual machines with the highest shares, at the cost of higher I/O latency experienced by the virtual machines with fewer shares.
  • If latency is more important, a very low value (lower than 20 ms) will result in lower device latency and better isolation among IOs at the cost of a decrease in aggregate datastore throughput.

Procedure

  1. Select a datastore in the vSphere Client inventory and click the Configuration tab.
  2. Click Properties.
  3. Under Storage I/O Control, select the Enabled check box.
  4. Click Advanced to edit the congestion threshold value for the datastore. The value must be between 10 and 100. You can click Reset to restore the congestion threshold setting to the default value (30 ms).
  5. Click Close.
For more information see the section “Managing Storage I/O Resources” on the “vSphere Resource Management Guide” (pdf) http://www.vmware.com/support/pubs/.

End of Availability of VMware ESX 4.x

 Dear Valued Customer,

VMware is announcing an End of Availability (“EoA”) date for VMware vSphere®   ESX hypervisor 4.x and for VMware Management Assistant (“vMA”) versions 1 and   4. The end of   availability date is August 15, 2013. This is a follow-on   communication to the general announcement made in July 2011 in connection   with the launch of vSphere 5.0.

This notification has NO IMPACT on existing vSphere ESXi 4.x   environments, and customers are NOT required to take any action.   However, it is recommended that customers make a backup or keep an archived   copy of these binaries and generate any necessary license keys in order to   maintain or expand a vSphere ESX hypervisor version 4.x or vMA versions 1 and   4 environment. These steps should be completed prior to August 15, 2013.   VMware will not provide any binaries or license keys for vSphere ESX   hypervisor 4.x or vMA versions 1 and 4 after August 15, 2013.

Additional information can be found at:
www.vmware.com/go/esx-end-of-availability

Please note:

  • vSphere ESX hypervisor 4.X and vMA support lifecycle
    The end of support life (“EOSL”) date remains May 21, 2014. VMware’s        support lifecycle page can be found at: www.vmware.com/support/policies/lifecycle/enterprise-infrastructure/eos.html
  • Customer’s ability to use the binaries of vSphere ESX        hypervisor 4.x or vMA versions 1 and 4 past August 15, 2013
    Customers retain the ability to use licensed binaries past the EoA or        EOSL dates. However, they will not be able to download binaries or        generate new license keys after the EoA date or obtain technical support        and subscription after the EOSL date.
  • vSphere ESXi 4.X availability and support – There is NO impact
  • vMA 4.1, 5, or 5.1 availability and support for all        versions – There is        NO impact

Can’t Bind Windows 2008 R2 VM to Interface for DHCP

UPDATE: I had originally thought I had resolved this by removing/re-adding the adapter. Turns out the issue reappeared this morning. After more researching, I found that it was caused by WDS! Since I no longer need WDS for testing, I have disabled the service. Went into bindings for DHCP and checked the interface with the static IP and it kept the setting. NOW DHCP should be working permanently. =)

Ran into a weird issue this morning when moving DHCP from an ASA to a 2008 R2 virtual machine on VMware ESX 4.1. The 2008 R2 machine had a standard network, static, adapter. The DHCP server role installed fine. I could not bind the DHCP server to the interface though! I restarted the VM a few times while troubleshooting then decided to yank currently installed adapter out (uninstalled from 2008 R2 first, then removed from VM via vSphere client).

I added a VXNET3 adapter, re-assigned the IP address it previously had then restarted DHCP server. I immediately saw the DHCP server bind to the interface.

Just a strange issue.

Bind to interface DHCP server 2008 R2

What is VDI?

Virtual desktop infrastructure or VDI is a computing model that adds a layer of virtualization between the server and the desktop PCs.

VMware describes Virtual Desktop Infrastructure (VDI) as delivering desktops from the data center. In other words, VDI is where enterprise desktop computers are virtualized, moved to the data center, then presented over the LAN or WAN to the end users. Once VDI is used, typically the end user devices are replaced with thin-client devices.  I’ve worked with some thinclient devices such as the HP T5740

While VMware has a VDI product called VDM (Virtual Desktop Manager), VDI is not a product exclusive to VMware. Other VDI vendors include Citrix XenDesktop & Kidaro (now owned by Microsoft).

With VDI, virtual desktops are served by enterprise virtualization servers running products like VMware ESX, Microsoft Hyper-V, and Xen Server. With the addition of the VDI products, these desktops can be dynamically created, pooled & shared, or even accessed from a GUI menu, over a web page.

Here is a nice infographic regarding what it will take for VDI to take off.

width=960

References:

http://en.wikipedia.org/wiki/Desktop_virtualization

A few notes on Guest Virtual Machine Memory Ballooning

Looking over some of my Windows 2008 R2 servers, I notice via Task Manager that their memory is 90%+ quite often.

Using Sysinternal’s RAMMap tool, I can quickly identify if this is due to ballooning.

Here’s a screenshot showing that 2.2GB of RAM is driver-locked (e.g., ballooned using the VMware Tools driver vmmemctl).

width=550

Server 2008 R2 showing driver locked memory > 2GB

Using vSphere Client, I check to see the performance of this VM to confirm if this is ballooned memory and what the actual memory usage is currently.

width=382

vSphere Client Resource Allocation for this 2008 R2 Guest VM

You can see that it is indeed ballooning nearly 2GB of RAM on this VM and the Guest VM is actively using only a little over 1GB of RAM.

To control how much memory the VM will balloon, you can set sched.mem.maxmemctl parameter in the virtual machine’s .vmx configuration file. (Reference: VMware KB1003586, MSDN Lock Pages in Memory VirtualLock)

The above KB mentioned is a good starting point for assisting troubleshooting Guest VMs that pin pages in memory (Lock Pages – using SQL with LPIM enabled?).

Enable Hyperthreading or 2nd CPU in Windows XP

I came across this on a VM the other morning. It seemed that assigned the VM 2vCPU and then installing XP Professional didn’t install the correct HAL/Kernel. I’m not sure why, but here’s how I noticed the problem and how I fixed it.

I first noticed the issue when it was reported that in Task Manager the CPU graph only had one CPU showing. In the View -> CPU History menu, there was only one option to show the single CPU. I checked device manager next and noted that there were in fact 2 CPUs recognized by Windows under Processors category. I then checked My Computer -> Properties and it only reported 1 E7540 @ 2.00GHz CPU. Hmmmm. I next checked the HAL by looking at Device Manager -> Computer and seeing that ACPI was installed but not ACPI Multiprocessor!

To change to ACPI Multiprocessor, you can right-click on the ACPI and choose Update Driver and it would list whichever drivers are installed. Here are some of the options that would be available (if installed):

Standard PC, Non-ACPI PIC HAL (Hal.dll)

  • Standard PC

MPS Uniprocessor PC, Non-ACPI APIC UP HAL (Halapic.dll)

  • Standard PC
  • MPS Uniprocessor
  • MPS Multiprocessor

MPS Multiprocessor PC, Non-ACPI APIC MP HAL (Halmps.dll)

  • Standard PC
  • MPS Multiprocessor

Advanced Configuration and Power Interface (ACPI) PC, ACPI PIC HAL (Halacpi.dll)

  • Standard PC
  • Advanced Configuration and Power Interface (ACPI) PC

ACPI Uniprocessor PC, ACPI APIC UP HAL (Halaacpi.dll)

  • Standard PC
  • Advanced Configuration and Power Interface (ACPI) PC
  • ACPI Uniprocessor
  • ACPI Multiprocessor

ACPI Multiprocessor PC, ACPI APIC MP HAL (Halmacpi.dll)

  • Standard PC
  • Advanced Configuration and Power Interface (ACPI) PC
  • ACPI Multiprocessor
  • MPS Multiprocessor

Well, I didn’t have any of those. So I knew Windows wasn’t installed, seemingly, with the support for Multiprocessor, etc.

Here’s how I corrected the problem.

I attached an XP Professional ISO to the VM and then from within the VM, I launched a command prompt and changed to the CD-ROM directory (in my case, D:):

cd /d d:

I expanded ntkrnlmp.ex_ and halmacpi.dl_ to c:windowssystem32:

d:>expand i386ntkrnlmp.exe_ c:windowssystem32ntkrnlmp.exe
d:>expand i386halmacpi.dl_ c:windowssystem32halmacpi.dll

Next, I modified boot.ini to specify the kernel and hal to load:

d:>attrib -r c:boot.ini
d:>edit c:boot.ini

I appended the following to the boot line:

/kernel=ntkrnlmp.exe /hal=halmacpi.dll

So the entire line looks like this now:

multi(0)disk(0)rdisk(0)partition(1)WINDOWS=Microsoft Windows XP Professional /fastdetect /noexecute=optin /kernel=ntkrnlmp.exe /hal=halmacpi.dll

Make sure you don’t make any typos or you will not be booting and you’ll have to fix your boot.ini through other methods.

After saving boot.ini, I made it read-only again:

d:>attrib +r c:boot.ini

Then I rebooted.

Voila. Now I see both vCPUs in Task Manager and My Computer Properties and today is a good day.

This is the first VM I came across with this problem and I have to believe it was because it was installed when there was only 1vCPU configured for it.

Get VM Extra Configuration Details

Add-PSSnapin VMware.vimautomation.core
Connect-VIServer -server 10.10.10.1 -user "user" -password "luser"
(get-vm -name "ZEUS").extensiondata.config.extraconfig | sort key | format-table

Example Output

Key                            Value                          DynamicType                    DynamicProperty
---                            -----                          -----------                    ---------------
ethernet0.pciSlotNumber        32
evcCompatibilityMode           true
guestCPUID.0                   0000000b756e65476c65746e496...
guestCPUID.1                   000106a400010800809822010fe...
guestCPUID.80000001            000000000000000000000001281...
hostCPUID.0                    0000000b756e65476c65746e496...
hostCPUID.1                    000206e60020080000bce3bdbfe...
hostCPUID.80000001             000000000000000000000001281...
nvram                          ZEUS.nvram
pciBridge0.pciSlotNumber       17
pciBridge0.present             true
pciBridge4.functions           8
pciBridge4.pciSlotNumber       21
pciBridge4.present             true
pciBridge4.virtualDev          pcieRootPort
pciBridge5.functions           8
pciBridge5.pciSlotNumber       22
pciBridge5.present             true
pciBridge5.virtualDev          pcieRootPort
pciBridge6.functions           8
pciBridge6.pciSlotNumber       23
pciBridge6.present             true
pciBridge6.virtualDev          pcieRootPort
pciBridge7.functions           8
pciBridge7.pciSlotNumber       24
pciBridge7.present             true
pciBridge7.virtualDev          pcieRootPort
replay.supported               false
sched.swap.derivedName         /vmfs/volumes/500d986f-e408...
scsi0.pciSlotNumber            160
scsi0.sasWWID                  50 05 05 65 7f 15 c3 70
scsi0:0.redo
scsi0:1.ctkEnabled             false
scsi0:1.redo
snapshot.action                keep
userCPUID.0                    0000000b756e65476c65746e496...
userCPUID.1                    000206e600200800009822010fe...
userCPUID.80000001             000000000000000000000001281...
virtualHW.productCompatibility hosted
vmci0.pciSlotNumber            33
vmotion.checkpointFBSize       4194304
vmware.tools.installstate      none
vmware.tools.internalversion   0
vmware.tools.lastInstallStatus unknown
vmware.tools.requiredversion   8300