Massive speed difference between 2 images

hongman · Wed May 18, 2011 11:16 am

Hi

I have 2 starwind images (VMWare vmdk) on the same array of 15k SAS drives.

Users are experiencing slow access to one of the drives, so I ran a disk speed utility:

Problem disk:

Ok disk:

Both disks have WB Caching enabled at 2048MB, and are identical really.

Can you shed some light?

hongman · Wed May 18, 2011 11:20 am

The disk speed utility was run within the Windows VM.

Here is the same results, run directly on the SAN:

Please help me understand the results and any problems...

Wed May 18, 2011 3:55 pm

Are the Disks connected via the same NIC interface, any Vlans configured?
I think you need to benchmark the physical array itself, the problem may appear to be in the hardware.

hongman · Wed May 18, 2011 4:08 pm

The comparisons below were run from 2 different ESXi hosts, but they both have the same capacity links back to the SAN and are both identical hardware.

What benchmarking software would you recommend?

Wed May 18, 2011 5:53 pm

ATTO Disk Benchmark (quick run), Intel I/O Meter (long run), SQLIO and Intel NAS Performance Toolkit for production-level tests (together with Intel I/O Meter with special access patterns). Please re-run the tests with at least ATTO, check network performance between both ESX nodes and storage and let us know your numbers. Thanks!

hongman · Thu May 19, 2011 8:52 am

Thanks.

Overnight, one of the (brand new) 15K SAS drives failed so now the array is running degraded, so a bit pointless doing tests until the drive is replaced!

Shall download the programs you mentioned to get ready though, will be posting results asap.

Thu May 19, 2011 9:20 am

Disk being dead on arrival could be a good reason why you get bogus data during your tests.

Sure! Please take your time on fixing hardware related issue and we'd be happy if you'd share your test results with us

hongman wrote:Thanks.

Overnight, one of the (brand new) 15K SAS drives failed so now the array is running degraded, so a bit pointless doing tests until the drive is replaced!

Shall download the programs you mentioned to get ready though, will be posting results asap.

hongman · Thu May 19, 2011 11:17 am

The disk only died last night, not doa.

But I'm thinking if it was on it's way out, could
Also be the reason for funny results!

Thu May 19, 2011 11:24 am

Absolutely!

hongman wrote:The disk only died last night, not doa.

But I'm thinking if it was on it's way out, could
Also be the reason for funny results!

hongman · Fri Jun 03, 2011 12:07 pm

Ok, sorry about the delay, got caught up.

ATTO Benchmarks:

Looks "normal" to you?

Fri Jun 03, 2011 7:17 pm

From what I understand you're providing RAW disk numbers (no StarWind involved). If yes then they are probably OK as RAID5/6 has issues with writes being much slower then reads (see link below). I would try to enable write back cache on RAID controller and experiment with RAID10 configuration.

If not then please provide details about network configuration, StarWind version, amount of cache allocated etc etc etc

http://daverdave.com/node/165 <-- Nice place to start reading about RAID5/6 Vs. anything else on Earth.