InfiniBand vs. RoCE: Understanding the Differences For Artificial Intelligence Data Center

RoCE (RDMA over Converged Ethernet) and InfiniBand are both advanced network protocol stacks developed by the InfiniBand Trade Association (IBTA). RoCE, as an earlier protocol stack, has been used for many years, while the introduction and growth of InfiniBand raises the question of what it means and what advantages or possible drawbacks it has when comparing InfiniBand vs. RoCE in the field of artificial intelligence. This article will provide a detailed explanation. The evaluation results help in choosing a model suitable for specific application scenarios and in optimizing system performance to achieve higher efficiency and effectiveness.

What is RoCE v2

Compared with standard Ethernet environments, RoCE introduces RDMA capability, allowing data to bypass the CPU and operating system kernel and be transmitted directly between memory. This improvement significantly reduces latency and increases throughput efficiency. Early RoCE v1 works at Layer 2. This design depends on “lossless Ethernet,” meaning packet loss cannot occur in the network; RDMA performance will drop sharply. It also prevents it from scaling across Layer 3 networks.

RoCE v2, introduced around 2010, builds on RoCE by encapsulating RDMA data over UDP/IP, allowing it to be routed across standard Layer 3 networks, thus providing better scalability and flexibility. This change allows RoCE to be used in modern data center network architectures such as Spine-Leaf topology. At the same time, RoCE v2 still keeps the core RDMA capabilities, including zero-copy, kernel bypass, and low CPU usage. However, it should be noted that although RoCE v2 runs over IP networks, its performance still depends on “lossless” network behavior, which usually requires fine-tuning through mechanisms such as PFC, ECN, and DCQCN; performance may fluctuate under congestion or packet-loss conditions.

What is InfiniBand

The first release of InfiniBand occurred near the turn of the millennium; the goal of designing InfiniBand was to develop a network specifically tailored for high-performance and parallel computing applications. InfiniBand vs. RoCE: Unlike RoCE, InfiniBand is not an evolution of the currently existing Ethernet technology. Rather, it represents a totally new approach based on protocol stacks all the way down to hardware design.

InfiniBand implements the message-passing paradigm rather than TCP/IP streams, and offloads the vast majority of network operations into hardware, which leads to a shorter communication path between nodes and thus ensures more reliable performance and lower latency at high concurrent workloads.

An InfiniBand network usually consists of switches, network cards (HCA, Host Channel Adapter), cables (DAC, AOC, or optical modules), and a Subnet Manager. These components together form a highly coordinated system. For example, NVIDIA’s Quantum and Quantum-2 switches and ConnectX series network cards are all deeply optimized around the InfiniBand protocol. This kind of hardware and software co-design makes InfiniBand not only a protocol but also a complete network solution.

InfiniBand key features apart from RoCE v2

#Credit-based flow control: before transmitting, the sender needs to make sure the receiver has sufficient buffer space. This protocol eliminates any chance of buffer overflow or dropped packets. It should be noted that, unlike Ethernet, which is built to reduce packet loss by using PFC, InfiniBand is built to prevent packet loss at all costs, making its performance more reliable.

#Parallel communications across multiple links: InfiniBand vs. RoCE. InfiniBand enables connecting nodes using multiple high-speed links. Thanks to this, it is possible to build networks with huge bandwidths and low blocking rates. Some examples include Fat-tree and Dragonfly topologies, which allow connections to up to a few hundred or thousands of nodes in a machine learning cluster.

#Ultra-low latency: with a lightweight protocol stack and hardware-level RDMA, data transmission does not need to traverse complex software protocol paths. Test results show that after bypassing the kernel protocol stack, application-level end-to-end latency can drop from about 50 microseconds (TCP/IP) to 5 microseconds (RoCE) or 2 microseconds (InfiniBand). This reduces latency and CPU load, freeing up more resources for actual workloads.

#Enhanced reliability: InfiniBand includes link-level CRC checks, retransmission mechanisms, and path redundancy, allowing fast recovery when link issues occur.

Their difference from the series parts

Principle and Architecture

With respect to networking architecture, InfiniBand vs. RoCE, InfiniBand has introduced a new component called Subnet Manager (SM). This is the reason why the InfiniBand network architecture is referred to as “centralized management”. The InfiniBand network architecture has skipped the Ethernet protocol architecture, and now the data transfer pathway looks like “Application -> RDMA -> InfiniBand Transport -> Switch”, without an IP layer and without the Ethernet buffering mechanism. It is the responsibility of the Subnet Manager to calculate the paths, allocate addresses, and distribute routing policies. In the InfiniBand network architecture, the entire network is centrally managed. Flow control is managed during data sending time. This centralized management brings deterministic paths and topologies, which make the links’ behaviors deterministic as well.

RoCE v2 is fully based on the Ethernet system and is essentially a “distributed network.” The data path follows “Application → RDMA → UDP/IP → Ethernet → Switch.” Each switch and network card works according to standard Ethernet behavior without a central controller. This brings flexibility, general usage, and easy expansion, but also makes network behavior affected by many factors, such as uneven hash distribution and queue congestion, which are harder to control.

Further, InfiniBand uses a credit-based mechanism to control flow before sending, avoiding congestion and packet loss at the source. InfiniBand vs. RoCE, RoCE v2 relies on PFC and ECN to correct issues during operation. Therefore, InfiniBand is closer to a “strongly constrained system,” while RoCE v2 is closer to a “self-adaptive system.

Performance and Scenario

• In real business performance, InfiniBand vs. RoCE, InfiniBand usually provides lower end-to-end latency. Operations like AllReduce and gradient synchronization in AI training depend heavily on network latency. When communication frequency is very high, the lower latency of InfiniBand directly improves training throughput. RoCE v2 has slightly higher latency, but in most AI scenarios, its performance is sufficient.
• From a scaling perspective, the answer is apparent. InfiniBand allows scaling out clusters up to tens of thousands of GPUs without affecting speed and minimal jittering. On the other hand, RoCE v2 is appropriate for clusters with hundreds to thousands of GPUs, where stable performance is attainable depending on the network configuration.
• In operation and management, InfiniBand benefits from centralized management and provides better visibility and diagnostic capability. Through the SM, global topology, link status, and path distribution can be monitored, making troubleshooting easier. InfiniBand vs. RoCE, RoCE v2 depends on existing Ethernet tools such as SNMP and telemetry. For experienced operators, RoCE ecosystems are mature but often require more manual effort.
• Regarding cost considerations, InfiniBand systems tend to be more expensive. InfiniBand demands a specific set-up with particular switches and network interface cards, while RoCE v2 uses already existing Ethernet networks with NICs and switch changes.

Considering the vendor side, InfiniBand products have a few vendors, dominated by NVIDIA. It leads to better integration and optimization between hardware and software solutions, but fewer options are available. RoCE v2, based on Ethernet, has broader vendor support and more flexible options.

How to choose in an Artificial Intelligence Data Center

In practice, an effective strategy is to use InfiniBand for compute and storage networks. 1-10 GbE Ethernet for the management network. This has been done in numerous HPC clusters where high-speed data traffic is segregated from normal network traffic by balancing cost and efficiency. When hundreds of GPUs are deployed, InfiniBand offers clear advantages. As for the topology design, the blocking ratio may be adjusted to balance efficiency and economy. For instance, when the bandwidth reaches 200G, it can connect over 100 nodes with a single switch and up to 200 nodes with two switches.

From an overall architecture perspective, InfiniBand vs. RoCE, although InfiniBand switches are more expensive, under the same bandwidth and blocking ratio, the network structure is often simpler, so the total cost is not always higher than Ethernet solutions. Cost is also affected by the blocking ratio, and different designs can lead to noticeable differences. In cabling, compared with earlier practices such as placing a switch in each rack to reduce optical module cost, current designs tend to use centralized or end-of-row deployment, which is more reasonable overall.

After determining whether to use an InfiniBand vs RoCE v2 architecture, network performance depends not only on the protocol and switches themselves, but also closely on the selection of underlying link components. Therefore, during the selection stage, components that have been verified across multiple platforms are usually preferred.

Taking our products as an example, OPTCORE currently provides a variety of interconnect solutions, including optical modules, AOC, and DAC, covering both InfiniBand and Ethernet (RoCE) scenarios.
It supports different data rates ranging from 25G to 400G/800G. These products have completed compatibility testing across multiple mainstream switch and NIC environments and have been validated by many parties, making them reliable.

FAQ

#1 Is intelligent computing networking built on existing TCP/IP infrastructure or a dedicated high-performance network?

RoCE v2 extends RDMA capabilities over existing TCP/IP networks, while InfiniBand builds an independent high-performance network.

#2 Do they support multi-tenant isolation?

Yes, you can achieve multi-tenant isolation through virtualization, queue isolation, and network partitioning.

Conclusion

The performance and reliability of InfiniBand are impressive in high-performance computing systems. InfiniBand vs. RoCE: With InfiniBand, we can expect high bandwidth and low latency for transmitting and processing data in an AI data center. As intelligent computing continues to develop, more innovative network solutions will emerge, giving engineers more choices.

Read more

• What is an IDF (Intermediate Distribution Frame)? A Beginner’s Guide
• Understanding Terabit Ethernet: 100G, 200G, 400G, and 800G
• What Is Optical Return Loss: A Beginner’s Guide