05 July, 2013

Shutdown unresponsive VM on ESX(i) 4.x / 5.x

Sometimes you will run into a VM that is unresponsive and or unreachable by RDP, even the vSphere client console will not work. The only way to solve this is to shutdown the VM, in some occasions even this will not work nor will the power-off or reset commands work.
When this is the case you will be getting error messages like: The operation is not allowed in current state or The attempted operation cannot be performed in the current state (Powered Off) or anything else along this line.
Recently a was working at a customer and the had a similar situation with 2 VMs, the admin's working on it where not able to successfully shutdown the either of the VMs. They asked if I knew a way to do it, maybe from the CLI.
I knew I could kill the process of these VMs thru CLI with the command  esxcli vm process list to get the world-id followed by esxcli vm process kill -t [soft,hard,force] -w WorldNumber to kill the process running the VM. But for some reason I was not able to find the VMs concerning in the output presented in the CLI (perhaps to many VMs on the host and over-looked it).
As alternative I also knew there was a way to kill processes from esxtop (only available on ESXi 4.x and 5.x), but I did not have all the details on the steps to perform but this was quickly solved by a quick search thru the VMware KB.
As it turned out, I think I prefer the "esxtop way" above the other ways of doing it, for 2 reasons; first esxtop is something you use regularly (I assume) and second it is a very clear and "visible" way.

How to use esxtop to kill processes of unresponsive VMs:

  1. On the ESXi console, enter Tech Support mode or connect thru SSH and log in as root.
  2. Run esxtop
  3. Press c to switch to the CPU resource utilization screen.
  4. Press Shift+v to limit / filter the view to virtual machines. 
  5. Press f to display the list of fields.
  6. Press c to add the column for the Leader World ID.
  7. Identify the target virtual machine by its Name and Leader World ID (LWID).
  8. Press k.
  9. At the World to kill prompt, type in the Leader World ID from step 7 and press Enter.
  10. Wait 30 seconds and validate that the process is not longer listed.
For more information on ways to shutdown unresponsive VMs please read VMware KB1014165

*Note; for the one's running ESX instead of ESXi please refer to VMware KB1004340 as there a different ways to do this on these systems.

1 comment:

  1. Thanks it works for me.
    However it takes 1min instead of 30 sec.
    Rest all perfect.