I currently have a system with 25 hard drives – 24 for data and one for OS.
This machine is built on a budget, featuring a cheap Gigabyte motherboard (GA-990FXA-D3), a four port LSI SAS controller, a two port LSI SAS controller, a dual port gigabit Intel network card and an nVidia 520 graphics card.
The current problem that I face is that if I have more than eight of the SAS connected hard drives connected at once, the motherboard will freeze after POST, and will not proceed to the OS (I have tried leaving this overnight).
This issue is controller independent, with the problem manifesting with hard drives connected to either controller.
The current case uses SAS backplanes with SFF-8087 connectors for hard drives, so connection of the 24 data drives to the onboard ports is not possible (not that there are enough onboard ports anyway).
In the past, I have worked around this problem by creating circuitry to turn on 16 of the hard drives after the OS has started booting. Unfortunately, this creates problems for airflow and load distribution, most prevalent during RAID resyncs, where drives may drop off due to insufficient power.
My question then, is does anybody know of a consumer level motherboard that will play nicely with 24 hard drives connected via SAS controllers? I’d like to know before I start spending money.
I don’t care if it is Intel or AMD based, but it needs to support the following:
- Support for 32GB of DDR3 RAM.
- x16 PCI-E slot for graphics card.
- x16 PCI-E slot for 4 port SAS controller.
- x8 PCI-E slot for 2 port sas controller.
- x4 PCI-E slot for network card.
- Maximum ATX form-factor.
Alternatively, has anybody had any luck getting AsRock motherboards to play nicely in similar situations?
For clarification: The system has a 650W Corsair supply, and I’ve tried swapping in a 1kW Corsair PSU, which has made no difference. The only direct power related issues appear to have been caused by my switching circuitry (workaround), which I have since removed.
Additionally, I believe a commercial NAS is unsuitable for this system, as storage is not its only purpose. The large amount of RAM makes it useful for large data processing tasks.
Answer
I’d love to know why some people downvoted this question, and what people think a “proper NAS server” is – ie why it should not be a PC. I think its a valid question.
In researching my belief that this is what some of the “the big boys do”, I discovered that the Backblaze Storage Pod is essentially a 4u server with heaps of cheap drives, and probably the answer to your question – as they have opensourced the hardware details.
They use SuperMicro MBD-X8SIL-F-B motherboards with 45 3 TB drives in a custom 3U case. Have a look here for more details. Similarly their third generation POD uses a MBD-X9SCL-F – both these boards are substantially less then US$200.
Its probably worth closely investigating exactly what they do and how they do it to iron out the gotchas – but certainly they have a enough knowledge and stats to have a pretty good idea of what works cheaply and reliably.
Attribution
Source : Link , Question Author : nibbler , Answer Author : davidgo
