Performance issues with a New 2 node Install

[bkdilse]

Hi,

I decided to try out Starwind VSAN Free, as a dedicated Storage Server (no VM Workload), purely for storage via iSCSI. Here are the details, it's a bit long, but I wanted to give as much details as I could.

Build details 2 x identical Servers with the following specs:

Quad Core XEON 1225 v3
32 GB ECC RAM
2 x Quad Port 1GB NICs (Total 8 Ports)
1 x 250GB HDD (for OS)
2 x 500GB SDD (for Data)
2 x 6TB SSD (for Data)

I installed Hyper-V Server 2016 on each node, updated fully, and joined an existing domain.

On the 6TB HDD, i've created devices (2TB, 2TB, 1TB), and replicated to the 2nd node (I used the 1GB, replicate, then expand trick to save time). This will be used for VMs Data Disks.

On the 500GB SSD, i've created a device (400GB), and replicated to the 2nd node. This will be used for VMs OS Drive, for faster access.

L1 Cache, all using write-through mode, with the reccomended L1 cache setting, so 2GB, 2GB,1GB for HDDs and also 1GB for SSD.

I've configured 2 ports on each server for Sync, and 4 ports on each server for iSCSI.

I then connected my ESXi Hosts and Hyper-V Hosts to the Storage serve via iSCSI, and started migrating my VMs (from the FreeNAS box).

So far, everything looked great, and perfomance was as expected.

PROBLEM1: I then added 2 more ports for Sync, which made the CPUs shoot up to 100%, so I removed the 2 ports again, and left it as 2 sync channels only. This is a slight concern, but not my main problem.

PROBLEM2: I removed the replica from 1 of the SSD drives, as I had to swap a drive in the 2nd node. When I re-created replica, a fully sync started, which has now been running for overs 12 hours. There is about 300GB of data on the 400GB device, is this replication time normal?

PROBLEM3: Performance seems to have de-graded since what it was when I initially started. CPU usage on each host is on avergage 60%.

QUESTION1: Am I better off using the SSDs as L2 cache instead of using them for VM disks. I've set it up this way, as I had great performance on FreeNAS for OS Boot drives on SSD, compared to HDD.

QUESTION2: To attempt to resolve PROBLEM3, I have set the sync priority to 25%, but have not yet rebooted the hosts, due to PROBLEM2 (still syncing). Should this setting help with performance of VMs?


Any help would be appreciated.

[Oleg(staff)]

Hi bkdilse,
Could you please log a support case using this form?

[bkdilse]

Thanks, I've raised it as requested.

[Boris (staff)]

Thank you. We have received your ticket and will process it accordingly.

[zenny7100]

Was this ever resolved? If so what was the cause?

[bkdilse]

Nope. Support were great at trying to help diagnose this, but in the end, we could not pinpoint where the issue lied. I ended up accepting the performance as it is.

I have non-enterprise disks, and old network cards, so that may the a factor.

[Boris (staff)]

Just in case, after this thread had been initiated there were 3 builds of StarWind VSAN released, and at least one of them - 12658 - contains strong recommendation on updating the software due to an important fix the new builds introduce.

[bkdilse]

Just in case, after this thread had been initiated there were 3 builds of StarWind VSAN released, and at least one of them - 12658 - contains strong recommendation on updating the software due to an important fix the new builds introduce.

I installed the latest release this weekend, but have not noticed any changes in performance. Thanks for pointing It out.

[Boris (staff)]

In that case it may still be related to networking.

[bkdilse]

In that case it may still be related to networking.

I will re-visit this, when upgrade my Hyper-V hosts to 2019. For now I can live with the performance as is

[Boris (staff)]

Keep us posted if any further investigation is done.