2 Node L2 Cache RAID 0 on SSDs? Also 3 node RAID0 everywhere

[JohnTrent]

Hello,

I am beginning my journey at looking at storage and frankly I think I have found the answer at the first stop. I do have a couple of questions though.

For a 2 node StarWind cluster can I use RAID 0 on the SSD L2 cache or is this a bad idea? I ask as this could significantly reduce the cost of the solution however I don't want to sacrifice protection and do something stupid.

For a 3 node cluster I totally understand the mathematics of why RAID 0 is safe and cost effective but it is one of those thoughts which hits you in the head and makes you doubt your entire working knowledge. Do people actually use this in the production environments?

For a rebuild in a 3 node cluster which has to happen on a failed disk how much overhead is there. Say I have two 10GB dual port Nics dedicated for the sync channels will we see 5GBs, 4.5GBs, 2.5GBs, 1.25GBs throughput. Obviously this is assuming that the underlying disks can provide that amount of throughput.

Is there a limit on how long the write backs can take? For example say I have a set of fast SSDs upfront but then have a 6 disk RAID6 10K drives at the backend where their write capabilities is significantly lower what happens if the write cache is full and can't purge to the disks quickly enough? i.e. getting 20k IOPS in with a full cache and can only push out 200 IOPS at the backend to disk.

On a modern server how many IOPS do you get with the RAM cache? I am assuming that it should be a level well above what the likes of a FusionIO or Intel P3700 or alternatives can give but just curious because if for example it can't do 180k IOPS then it might be better to disable the RAM cache and go for an Intel P3700 which also has the benefit of freeing up some RAM. However when you do that you then can't specify an secondary L2 cache which is a shame.

Hopefully one last question in regards to the Tape redirector and this might be one for Veeam but say I have an offsite DR server which has the tape drive connected to it can I command it from the server in my office to take a copy of the replicated VMs on that DR server. i.e. Onsite servers replicate to DR site through Veeam. I also want to take a copy of the replicated VMs to tape but ideally want to use Veeam in my office to handle that (due to licensing costs). In this case would the data have to traverse the WAN link twice? Once to read the data from the DR site and then one to write the data back to the tape at the DR site or is it able to something like ODX?

[anton (staff)]

Cache can be RAID0 no problem. If we'll detect block is damaged (hash is incorrect) we'll just mark L2 as "failed" and continue with L1 (RAM) and back end storage and no L2 cache. However cache is typically written and read with a small blocks (< RAID stripe size) so doing RAID0 is not going to improve overall performance of it.

We don't recommend using flash in write-back mode (burns flash cells and increases number of pipeline stages so overall processing latency) so use L1 as write-back (accelerate and adsorb writes) and L2 as write-thru (accelerate reads).

RAM can do millions of IOPS. System bus is much wider and has much lower latency compared to PCIe-attached flash. I don't understand what issue you have with setting L2 cache size. Can you elaborate on that?

Also I don't understand your scenario with VEEAM and StarWind. Please spawn a separated thread for that telling more details. However if you're trying to use StarWind to somehow compromise VEEAM licensing I have bad news: we're not going to help with that. VEEAM is a friendly company for us and we're willing to be a complimentary rather then competitive solution for them.

[JohnTrent]

Hello anton,

Apologies if I wasn't clear with the Veeam question and I wasn't trying to be non compliant with their license just looking to be license efficient with looking to push backups to tape at our DR center but I will pick that up with them directly.

I have been reading some documentation and I think I have been over specifiying the NICs in the server if I am reading this document correctly (http://www.starwindsoftware.com/styles- ... manual.pdf)

Although I would like to check with you. In this document the following section says:

Picture1.png (252.03 KiB) Viewed 6166 times

In this situation is it using just 2 Dual Port 10GB NICs? If that is the case could we say that this would give enough throughput for 282,000-564,000 IOPS (4k) as noted in the High Availability Best Practises document?

I suppose my question is to get 282,000-564,000 IOPS is this 4 NIC solution enough with Sync and iSCSI running over the 4 NICS?

Picture4.png (21.28 KiB) Viewed 6166 times

Or would you have to go for 8 NICS with 4 for Sync (NICs 1-4) and 4 for iSCSI (NICs 5-8)

Picture5.png (37.57 KiB) Viewed 6166 times

If it is that with 4 dual port nics in the server we can push that many IOPS across the SAN we can have some more PCIe slots available for RAID cards and tape drive connectors etc. So it is an attractive solution.

Also in regards to both of these solutions can we push the iSCSI via direct connections and eliminate the need for any 10GB switch as we can have the general connections from the desktop machines via standard 1GB connections.

i.e. in the following post there is an attachment which shows the potential layout for a 2-node solution. Assuming that the data storage layer is capable, is the network storage layer configured in such a way to deliver 282,000-564,000 IOPS and present LUNs to both servers while allowing general access to the VMs from the LAN. Assumptions NICs 1-4 are 10GB, NICs 5-8 are 1GB and the switches are 1GB.

[JohnTrent]

Picture6.png (51.79 KiB) Viewed 6166 times

Hopefully that makes sense although if it doesn't please let me know.

[JohnTrent]

Thinking about this some more if it is able to be done over the same link then we could switch from Dual Port 10GB NIC to Single Port 40GB NIC and have 2x the throughput on the cluster.

Would it be possible to have 2 single port 40GB NICs in each server and have them both directly connected and provide the sync/iscsi vlan channels over them both to give NIC failure protection and fast throughput. Or do the sync and iSCSI channels have to be on physically separate cables.

I suppose the question is also not only if it is possible but also if it is best practise or not.

[Anatoly (staff)]

Hi!
OK let`s do through your questions one-by-one.

In this situation is it using just 2 Dual Port 10GB NICs? If that is the case could we say that this would give enough throughput for 282,000-564,000 IOPS (4k) as noted in the High Availability Best Practises document?

If those NICs will be used for one purpose, then you should. For example, if they will be used to handle the workload of some app, that gives 500000 IOPS 4k, then the NICs should do the job.

Also in regards to both of these solutions can we push the iSCSI via direct connections and eliminate the need for any 10GB switch as we can have the general connections from the desktop machines via standard 1GB connections.

You can, I would even recommend doing that.

Thinking about this some more if it is able to be done over the same link then we could switch from Dual Port 10GB NIC to Single Port 40GB NIC and have 2x the throughput on the cluster.

Sounds great, If that fit your budget then why not.

Would it be possible to have 2 single port 40GB NICs in each server and have them both directly connected and provide the sync/iscsi vlan channels over them both to give NIC failure protection and fast throughput. Or do the sync and iSCSI channels have to be on physically separate cables.

You can but you must know that synchannel should be used for synchronization process only.

I hope that makes sense.