Supermicro recommendation

caleb72 · Fri Mar 19, 2010 1:40 pm

I am getting a parts list together for a Starwind HA configuration. I am looking at using a Supermicro SC826 series chassis with 12 2TB WD SATA drives. Drives could change though in final configuration, I may do a mix of SATA and SAS drives. I am interested to hear from those that are using Supermicro what motherboard and RAID controller combonations you have or would reccommend for Starwind.

Thanks

Fri Mar 19, 2010 7:00 pm

Waiting for a user feedback here (of course) but as a developer and main system architect I can say StarWind was designed to be 100% vendor agnostic. We really don't care about hardware. If OS is happy - we're happy as well

There are some "beasts" we do recommend to avoid but neither is produced by SM so far.

caleb72 wrote:I am getting a parts list together for a Starwind HA configuration. I am looking at using a Supermicro SC826 series chassis with 12 2TB WD SATA drives. Drives could change though in final configuration, I may do a mix of SATA and SAS drives. I am interested to hear from those that are using Supermicro what motherboard and RAID controller combonations you have or would reccommend for Starwind.

Thanks

peekay · Tue Mar 23, 2010 3:46 pm

We have Starwind 5.2 running on the following config:

Supermicro SC216E2-R900U case with X8DTL-iF motherboard (the case has two SAS/SATA expanders)
2 x Intel Xeon E5530 (dual core) processor, 8 GB total RAM
Adaptec 5805 PCIe SAS/SATA controller using two SATA multi-lane cables (one per expander in the Supermicro case)
13 x Seagate 2.5" Savvio SAS drives (RAID 1 and large RAID 5 + spare)
Intel PCIe Quad Server 1Gpbs NIC
Win 2008 x64 Standard

The system is very reliable, rock steady and fast. With the Adaptec controller, we can mix SAS and SATA but have not done so yet.

Constantin (staff) · Wed Mar 24, 2010 4:15 pm

Hope to find out what system you final choose as final option

umarmuha · Thu Mar 25, 2010 3:55 am

@peekay those are very nice specs. I am glad to know the system is working out for you.
I was really glad to find this post since Supermicro is exactly what i configured for my iSCSI SAN yesturday. It has always been a problem in the past that i get budget and i end up buying a SAN from a vendor (expensive) and then i dont have enough money left to buy a redundant unit. So i learned from my mistakes and decided that this year i am going to use my budget wisely and for the price of one expensive SAN build my self TWO efficient and cost effective iSCSI SAN with StarWind for my VMware env. I tested StarWind with some OK hardware and i loved what i saw. I was able to get 95-110MB/sec while cloning and backing up virtual machine, which is awesome. I know that StarWind doesn't care of the underlying hardware but it would be great to get some professional feedback. So what i ask of all you amazing pros out there is take a glimpse at my configuration and give me a confirmation that i have not wasted my time. So here it goes.

(I will be buying two of these with the same exact configuration)
1. Server Chassis
1xSupermicro High Density 4U Stor Chass,1200W High-Efficiency (1+1) Redundant Pwr Supply
2. MotherBoard
1xSUPERMICRO MBD-X8DTH-iF-O Dual LGA 1366 Intel 5520 Extended ATX Dual Intel Xeon Server Motherboard
3. Memory
2xKingston ValueRAM 6GB (3 x 2GB) 240-Pin DDR3 SDRAM DDR3 1333 ECC Registered Server Memory
4. Processor
2xIntel Xeon E5520 Nehalem 2.26GHz LGA 1366 80W Quad-Core Server Processor
5. Raid Card
1xAreca Technology Corporation PCI-E to SAS RAID Adapter, 24 Internal Ports and 4 External Ports
with 2GB DDR2 PC2-4200 533MHz 240PIN DIMM ECC Unbuff CL4 1.8V No RTRN
and Battery Backup Module RoHS
6. Hard Drives
For OS: 2xIntel X25-M Mainstream SSDSA2MH080G2R5 2.5" 80GB SATA II MLC Internal Solid State Drive (SSD)
For RAID 10: 24xWestern Digital RE3 WD1002FBYS - hard drive - 1 TB - SATA-300
7. Network Card
For Management: 2xInternal Intel 1Gb nics
For iSCSI: 1xQLogic 1GB PCI-E iSCSI Copper Single Port QLE4060C-CK

Your feedback and suggestions will be much appreciated. Thanks in advance.

peekay · Thu Mar 25, 2010 6:40 pm

Nice configuration!!! However, it is somewhat overkill.. Here are my thoughts:

- Only one instance of the StarWind service can run at a time. You have a total of 8 cores which is totally overkill. You can go with a single quad
- Your RAID card has 2GB of cache. StarWind supports RAM caching also but it may not be a good idea to have both StarWind and your RAID card caching so you have way too much system RAM. However, given it is tripple channel, your only choice is to drop to 6GB which I would not do.
- The SSD for the system is good for boot but not so good if you have a swap file. Running without a swap file can be problematic. You should consider two 10K RPM Velociraptor drives instead. Once the system is up and running, there is very disk activity on the system drive. Which operating system are you running?
- You spec'd a 24 drive RAID 10. A RAID 5 with 23 drives and one hot spare will give you as good performance (large number of spindles) and won't be as hard on the drives (RAID 10 requires all the drives to be spinning since it is a mirror of spanned disks). It will also give you way more storage. Why RAID 10?
- The QLogic card you spec'd is an iSCSI HBA card that acts as an INITIATOR and will not work as a TARGET. StarWind recommends a TOE NIC and in your case, if you can afford it, go with a 10Gbps TOE NIC. How many servers will be connected?

umarmuha · Fri Mar 26, 2010 1:10 am

Hey Thanks so much for the detailed info. I totally agree with you. When i was configuring the system i had the same feeling that this all seems a little over kill. The recommendations are great and i will bring the configuration down a notch. It makes sense. The reason i added ram on the raid card was because i recently tried StarWind with cache (Dell PE 6800 4GB RAM with PERC 5e RAID card) and ran into all sort of issues with VMware. I have a open case with the support to figure out whats going on. I just figured that if i leave all the caching for the Card, might not be a bad idea. I will be running Windows 2008 Enterprise Edition x64 bit and planning to install that on the 2 SSD's with RAID 1. So in windows will the swap file translate into the page file? I was not sure what you meant that SSD are not recommend when there is a swap file. In the past i had two MD1000 units with 750GB 7200RPM SATA drives connected to two different servers. One had raid 5 configuration and the other had RAID 10. Both server were connected to the MD units using the PERC 5e RAID card (eeww) and i had way better performance from the unit connect to MD1000 with RAID 10. Thats the only reason i decided to go with RAID 10 on the 24 hard drives. But like you mentioned with all the killer specs i doubt that there will be any performance hit if i go with RAID 5 right? For the network card i was going to buy Alacritech iSCSI network interface controller because that one was recommended by StarWind and Best Practicies. That pdf is from 2006 and seems like they dont make the specific card i was looking for anymore. So the next best thing i found on internet was the Qlogic. It does mention under the tech specs on the Qlogic website that it has TOE . I would totally go with the 10Gb option but i don't have 10Gb switches in my network. Any other 1Gb TOE NIC that you would recommend? intel? Thanks again.

Constantin (staff) · Fri Mar 26, 2010 11:30 am

Intel would be great option! In this PDF you can find list of all NICs that sells Intel , but unfortunately that PDF doesn`t provide any information about TOE. Well, at least this paper will allow to find what exactly NICs fit you as best option, and then find out if they support TOE.
Second great option is Broadcom NICs - most of all their chips fully support TOE .
Based on my experience I can say that quality of chips of both NICs is absolutely equivalent

umarmuha · Sun Mar 28, 2010 1:07 am

Excellent! thats very helpful. Thank you.

Mon Mar 29, 2010 1:30 pm

Please keep us updated as soon as you'll have something built-up configured & running

Thank you!

peekay · Mon Mar 29, 2010 9:09 pm

The thing that makes the biggest difference in RAID performance is the number of spindles (drives). RAID 5 with 23 drives will fly. In fact for maximum redundancy, RAID 50 is often selected (mirror of RAID 5) but the hardware cost is a big hit. The SSDs in this configuration won't really give you much performance boost except for boot speed which should not be an issue on a server. Putting SSDs for the actual RAID drives is another thing altogether

Broadcom does sell TOE cards that supposedly accelerate iSCSI but they are not compatible with ESX. However, on the StarWind side, since the drivers are WIndows drivers, they should work well. The Intel server grade NICs are also very good. More expensive TOEs may not give you that much offloading and since you have lots of horsepower, using the Broadcom or Intel is fine.

I understand the 10Gbps dilema (I was doing a bit of vicarious thinking

Mon Mar 29, 2010 9:30 pm

1) You could read a bit about RAID sub-six here: http://www.baarf.com/

2) Our engineers here managed to have Broadcom adapters running together with ESX. So it could be done. Another question is: should it be done?

) I'd go Intel...

peekay wrote:The thing that makes the biggest difference in RAID performance is the number of spindles (drives). RAID 5 with 23 drives will fly. In fact for maximum redundancy, RAID 50 is often selected (mirror of RAID 5) but the hardware cost is a big hit. The SSDs in this configuration won't really give you much performance boost except for boot speed which should not be an issue on a server. Putting SSDs for the actual RAID drives is another thing altogether

Broadcom does sell TOE cards that supposedly accelerate iSCSI but they are not compatible with ESX. However, on the StarWind side, since the drivers are WIndows drivers, they should work well. The Intel server grade NICs are also very good. More expensive TOEs may not give you that much offloading and since you have lots of horsepower, using the Broadcom or Intel is fine.

I understand the 10Gbps dilema (I was doing a bit of vicarious thinking

caleb72 · Tue Mar 30, 2010 2:05 pm

First off thanks for all the information and help. Sorry I haven't posted back earlier, but it is tax season and I work for an accounting firm.
Anyway, the following are the configs I now have to consider. With the information I am receiving I may revisit the Supermicro config, but it was put together with help from Supermicro.

Supermicro
X8SIA MOTHERBOARD
SC826E1-R800LPB
SUPER MICRO AOC-SASLP-H8IR RAID Controller
7x 2TB RE4-GP
XEON X3440 2.53GHZ
Intel PRO/1000PT 4PORT NIC
4 Kingston 4GB 1066MHZ DDR3

I am also considering an Intel solution.

INTEL
INTEL SR2612UR
1x XEON E5506-2.13GHZ
2x 150GB VELOCIRAPTOR *for OS
7x 2TB RE4-GP
Adaptec 5405Z SAS/SATA RAID CONTROLLER or 3WARE 9690SA w/BBU module
Intel PRO/1000PT 4PORT NIC
4 Kingston 4GB 1066MHZ DDR3

I do like the Intel for the simple fact that the chassis has internal drive bays for my OS drives and that leaves all 12 hot-swap bays open for storage. I have had to scale down the number of drives for now; to keep cost down, but that leaves me open for future growth or a SAS array if needed.
I'll let you know what our final config will be though.

Also wrestling with going with a 6 drive RAID10 or RAID5 the 7th drive being a spare. This will be an HA pair mainly for Xen server, but we will have a physical Exchange 2010 and a physical MS SQL 2008 with about 200 users, any thoughts?

peekay · Wed Mar 31, 2010 3:34 pm

Ok... in short:
- Intel server config GOOD
- Adaptec RAID controller w/BBU GOOD (server seems to have SAS/SATA expander midplane for 12 drives for you will need a 4-lane cable to connect)
- 7 x 2TB OK (single drive failure requires long rebuild time) - HOT SPARE IS A MUST!
- INTEL NIC GOOD (I use this NIC on our StarWInd server and it works well) - MUST BE PCIe version
- 16GB RAM GOOD, only if you use StarWind RAM CACHING - Intel has redundant PSU so that is good (MAKE SURE TO USE 2 DIFFERENT POWER SOURCES - eg. 2 UPS units)

Based on your config, I would go RAID10 with 6 drives + 1 hot spare. Not as much storage as RAID5/6 but the write performance will be better.

Wed Mar 31, 2010 4:38 pm

PCI-X is possible... And check for NICs being native PCIe and not using bridge (as it's the case for example for Neterion cards).

peekay wrote:Ok... in short:
- Intel server config GOOD
- Adaptec RAID controller w/BBU GOOD (server seems to have SAS/SATA expander midplane for 12 drives for you will need a 4-lane cable to connect)
- 7 x 2TB OK (single drive failure requires long rebuild time) - HOT SPARE IS A MUST!
- INTEL NIC GOOD (I use this NIC on our StarWInd server and it works well) - MUST BE PCIe version
- 16GB RAM GOOD, only if you use StarWind RAM CACHING - Intel has redundant PSU so that is good (MAKE SURE TO USE 2 DIFFERENT POWER SOURCES - eg. 2 UPS units)

Based on your config, I would go RAID10 with 6 drives + 1 hot spare. Not as much storage as RAID5/6 but the write performance will be better.