Friday, March 2, 2012

Slow Read Performance from Storage Arrays - Delayed Ack

I recently stood up an ESX host on 10GbE.   The ESX host is using the Cisco Nexus 1000v connected to a Nexus 5K.  Storage is an Equallogic PS6010, also using 10GbE.

I ran IOMeter tests against this new setup and was disappointed with the poor read performance.  For consistency, the OpenPerformanceTest.icf config file from the following post was used:

http://communities.vmware.com/thread/73745

The IOPs for the Max Throughput -100%Read test was showing 584.   That's considerably slower than our existing production environment using a 1GbE infrastructure.

To resolve this issue, DelayedAck was disabled on the ESX host. TCP Delayed Ack groups together several ACK responses into a single reply. Delayed Ack can be disabled at the Discovery Address level (preferred), OR globally for the vmhba used for iSCSI.

Discovery Address Level - Select the iSCSI server under the Dynamic Discovery tab for your iSCSI vmhba.  Then, uncheck Inherit from Parent, and DelayedAck.  Reboot Host.
Globally - Uncheck DelayedAck in Advanced Setting for the vmhba used for iscsi. Reboot Host.


After making this change, I ran the same test and and was pleasantly surprised by an IOPs score of 22,354!!

Slow SSH Login - Fix

I was having slow login times on one of my newly built ESX hosts.  Other hosts on the same network were logging in without delay.  However, this particular host was taking 10-15 seconds.

In my particular case, it was a DNS configuration issue on the host.  Confirm that both the Preferred and Alternate DNS servers are still responding to requests.

After entering valid DNS Server IPs, I was getting lighting fast logins.