
The ASRock Rack GENOAD8X-2T/BCM is likely to be an uncomfortably good motherboard. It has an enormous variety of options that these seeking to construct an AMD EPYC-based system will need, exposing the platform’s large I/O capabilities to builders. On the identical time, it is a platform with some quirks to it, so it’s a enjoyable one to evaluate. Allow us to get to it.
ASRock Rack GENOAD8X-2T/BCM
Allow us to begin of with crucial spec of the motherboard, it’s EEB (12.63″ x 13″) in dimension. Meaning it should match many bigger instances in the marketplace at this time, however it should require on the lookout for EEB help. Even with a dimension that’s a lot bigger than a typical ATX motherboard, this technique is totally full of performance to the purpose it virtually seems like ASRock Rack ran out of area at some factors. Many server motherboards at this time use proprietary kind components, however boards like this assist those that want to construct their very own servers in particular chassis.

Allow us to begin with a major instance of what makes this platform cool, but in addition the constraints it’s below. We get an AMD socket SP5. Only for some sense, that enables for as much as a 96-core AMD EPYC 9004 “Genoa”, a 96-core with 1.1GB of L3 cache Genoa-X, or a 128-core Bergamo. Placing it succinctly, one can get roughly the efficiency of a dual-socket previous-generation server on this single socket.

There are trade-offs to suit these large CPUs. One of the distinguished is reminiscence. This motherboard solely helps 8-channel DDR5 reminiscence in a single DIMM per channel (1DPC) mode. We not too long ago defined Why 2 DIMMs Per Channel Will Matter Much less in Servers however we’re dropping 50% of the reminiscence channels and 66% of the utmost capability with this design. Then again, with DDR5 we get roughly 50% extra reminiscence bandwidth than the earlier era’s 8-channel DDR5 designs. Nonetheless, it’s lower than the CPUs are able to when it comes to efficiency and capability. Placing 24 DDR5 DIMM slots subsequent to a SP5 socket would have taken up an infinite quantity of motherboard area so the trade-off was made.
One thing else that could be a bit totally different with this platform is the ability enter. There are three ATX12V CPU energy inputs together with a typical ATX energy enter. Having CPUs that may use 350W or extra plus an enormous set of parts means these techniques want extra energy. We began STH within the period when it was frequent to have a single ATX12V energy connector onboard.

Then comes the enlargement slot arry that many will likely be excited to see. There are seven PCIe Gen5 x16 slots and one x8 slot. 4 of these x16 slots can run in CXL 1.1 mode.

Seven PCIe Gen5 x16 and one x8 slot could be lots of I/O alone, however there may be extra, much more. An instance is the twin M.2 slots onboard.

Earlier than we get to the remainder of the I/O, we should always most likely deal with the chips discovered throughout this motherboard. There are Phison PS7101 chips throughout that are PCIe Gen5 sign redrivers with mux/ demux. With PCIe Gen5, even reaching indicators from one facet of the motherboard to the opposite might be difficult from a sign integrity standpoint, and even moreso if, for instance, a motherboard is designed like this one the place the 2 PCIe x8 segments on the AMD Green2 facet can drive one PCIe Gen5 x16 slot or ship x8 to the slot and x4 to a M.2 and x4 to a MCIO connector. These totally different distances weren’t as huge of a problem within the PCIe Gen3 period, however with PCIe Gen5, it’s a sign integrity problem, therefore why chips just like the Phison PS7101 are getting used.

As one can see, there are two MCIO connectors on the alternative facet of the motherboard from the PCIe slots with the M.2 slots within the center.

That white 6-pin connector is a graphics 12v energy connector.
There are two extra MCIO x4 connectors above the PS7101 array and between the DIMM slots and the entrance panel USB 3 header and fan header array.

These MCIO connectors are the choices for connecting not simply slotted PCIe playing cards but in addition NVMe SSDs and with a number of the connectors issues like SATA SSDs. We’re going to put the block diagram on this evaluate as a result of it’s a very complicated setup.
With regards to the rear I/O one thing we needed to rapidly point out is that the airflow of this motherboard is commonplace server front-to-back. Some will make pedestal servers out of this platform, however we strongly advise that if you’re wanting to make use of the highest vents for CPU exhaust, that you just accomplish that with liquid cooling to not intervene with the system airflow.

At this level, we most likely have greater than a handful of STH readers pondering that this could make a tremendous workstation platform. These are wonderful ideas, nonetheless, one of many huge challenges is the large hole between the server and workstation rear I/O. This platform has a decidedly server rear I/O with legacy serial, VGA, and two USB ports for native KVM cart entry within the information middle.

The only community port stacked atop the USB ports is for out-of-band administration. The motherboard has an ASPEED AST2600 Baseboard Administration Controller or BMC onboard. This runs an industry-standard administration stack for IPMI and performance like iKVM entry.

The 2 community ports are 10Gbase-T ports powered by the Broadcom BCM57416 controller that sits below a heatsink behind the VGA and serial ports. That is additionally the rationale for “BCM” within the GENOA8X-2T/BCM product identify.

This can be a complicated motherboard, so subsequent, allow us to get to the topology.