VMware Tools: VMwareService.exe or vmware-guestd High Memory Usage

Ever seen the VMwareService.exe or vmware-guestd process taking all your memory? Well I did see this issue on a few VM’s. Like this VM with Windows Server 2003 SP2 R2:

The VMwareService,exe took 1,3GB of RAM. Ok this is a problem but what started this memory “leaking”.

In my case it was the usage of ~100 ip addresses on a single network adapter:

So I opened a SR at VMware and in the mean time a added the following workaround:

On a Windows VM you can create a simple batch file like this:

@echo off
echo Restart VMware Tools
net stop "VMware Tools Service"
net start "VMware Tools Service"

and schedule it on a daily basis.

On a Linux VM you can use the following cron entry to automatically restart the vmware-tools. Use crontab –e to edit the file:

# Workaround Memory Leak problem
30 6 * * * /etc/init.d/vmware-tools restart >/dev/null 2>&1

In the mean time VMware has released a patch to address this issue: ESX400-201009401-SG. You will find the following quote in the release notes:

Virtual machines display increased memory usage in vmware-guestd and vmwareservice.exe. The memory footprint of the process continues to increase until the available memory is drained and the process cannot allocate any memory. This issue is more prominent when the guest operating system has a large number of IP addresses associated with it.

I have installed the patch on one of my labs and I am running a test to see if this patch resolved this issue. I installed a Windows 2003 SP R2 VM and added 100 IP addresses to the Local Area Connection 2 adapter. You can do this with the following for loop and the netsh command line utility:

for /L %a in (1,1,254) do netsh in ip add address "Local Area Connection 2" 10.0.0.%a 255.255.255.0

or you can use Powershell with the following script:

# http://www.powergui.org/thread.jspa?threadID=11829
# configure the first ip address 10.0.0.1 on the adapter where you want to add the 100 IP addresses.

$ips = 1..100 | %{ "10.0.0.$_" }
$masks = 1..100 | %{"255.255.255.0"}
$nic = Get-WMIObject win32_networkadapterconfiguration -ComputerName localhost | where { $_.IPAddress -contains "10.0.0.1" }
$nic.EnableStatic($ips,$masks)

After this step you just have to wait en monitor the VMwareService.exe process to see if this patch will solve this issue. I will post my results later.

Update test results:

Before I installed the ESX400-201009401-SG pach:

Start test: 11:57 16-10-2010

VMwareService.exe         6792 K    4712 K

End test: 12:06 17-10-2010

VMwareService.exe     105336 K    103332 K

After installing the ESX400-201009401-SG patch and upgrading the VMware Tools on the VM I got the following results:

 

Start test: 10:30 17-10-2010

VMwareService.exe    6792 K    4712 K

Einde test: 10:30 18-10-2010

VMwareService.exe    6792 K    4712 K

So to resolve this issue on vSphere 4.0 install the ESX400-201009401-SG patch. You also need to upgrade the VMware Tools and restart the VM to fix this issue.

If you’re running vSphere 4.1 you should not have this issue. Thanks @Andrea_Mauro for recreating and testing this issue on a Linux VM on vSphere 4.1:

There is also another “bug” fixed with this patch. After installing this patch you should be able to see all the guest IP addresses within the vSphere client: