Select the fabric in the URI.
aeron:ib accepts the network-channel parameters Aeron users already know. IPoIB addresses keep endpoint tracking, flow control, NAK routing, and multi-destination behavior in Aeron's existing model.
InfiniBand extension for the Aeron C media driver
InfiniBand,
as an Aeron media.
aeron:ib adds a verbs-backed network path beside
aeron:udp and aeron:ipc. Keep Aeron's publications,
subscriptions, flow control, and recovery. Choose Unreliable Datagram for multicast or
Reliable Connected for hardware-segmented unicast.
first-class channel URI
UD and RC queue pairs
unicast and multicast
hardware tested
One media driverper-channel selection
aeron:ipc
Shared memoryExisting local transport, unchanged.
aeron:udp
UDP socketsExisting network transport, unchanged.
aeron:ib
InfiniBand verbs NEWUD by default; RC when requested.
C, C++, and Java clients → Aeron C media driver → selected medium
5.140Mmessages/s over clean RC at MTU 4096
4.413Mmessages/s over default UD
3.40×UD versus kernel UDP at standard Ethernet MTU
+16.5%RC over UD on the same fabric and MTU
The extension
A third transport. The same Aeron contract.
This is not an InfiniBand fork of Aeron. It is a media binding inside the C driver. Existing UDP and IPC channels continue to work, and one driver can carry all three media at once.
Keep multicast and Aeron's recovery.
Unreliable Datagram is the default. It matches Aeron's datagram delivery contract and is the InfiniBand queue-pair type that can join multicast groups.
Let the HCA segment larger messages.
Reliable Connected is opt-in per channel. It adds a queue pair per peer and a shared receive queue, trades away multicast, and can carry Aeron frames larger than one 4096-byte fabric packet.
Post from the term buffer.
Publication logs are registered with the HCA. A matching frame is posted directly from the Aeron term buffer; small frames go inline, and unregistered buffers use a registered fallback.
| Channel URI | Queue pair | Multicast | Purpose |
|---|---|---|---|
aeron:ib?endpoint=10.88.88.2:20121 | UD | yes | default InfiniBand channel |
aeron:ib?endpoint=224.0.1.101:40456|interface=... | UD | yes | InfiniBand multicast |
aeron:ib?endpoint=10.88.88.2:20121|ib-qp=rc | RC | no | connected unicast |
aeron:ib?endpoint=10.88.88.2:20121|ib-qp=rc|mtu=16384 | RC | no | hardware-segmented messages |
Client boundary: C, C++, and Java clients accept aeron:ib URIs when they use the C media driver. The Java media driver has no InfiniBand binding and rejects the URI explicitly.
Inside the driver
Verbs below. Aeron above.
The extension changes how frames reach the wire, not what an Aeron publication means. Aeron still owns positions, flow control, NAKs, retransmission, and delivery to the subscription.
01 / PUBLICATION
Registered term buffer
The HCA can read an eligible frame without a send-side copy.
02 / FABRIC
UD or RC queue pair
RDMA CM resolves peers and multicast membership from IPoIB addresses.
03 / SUBSCRIPTION
Posted receive ring
Completed buffers enter Aeron's normal receive and recovery path.
Media coexistence
Each transport records the binding that created it. Shared sender, receiver, and poller paths dispatch per transport, so identical UDP and IB endpoints cannot collide or silently share media.
Peer discovery
UD unicast uses RDMA CM Service ID Resolution; multicast uses rdma_join_multicast. RC uses connected identifiers and a shared receive queue.
Receive capacity
The receive ring is derived from configured receive capacity and MTU, then clamped to the HCA's work-request and completion-queue limits.
Media timestamps
When the device supports completion timestamps, the binding feeds hardware wall-clock receive timestamps into Aeron's media timestamp field.
Measured result
The clean InfiniBand result is 5.14 million messages a second.
RC and UD were measured on the same 40 Gb/s InfiniBand fabric with the same 4096-byte
Aeron MTU. RC reached 5.140 Mmsg/s; default UD reached 4.413 Mmsg/s. Both delivered every
message and recorded ErrorStat at 0/0.
5.140MRC/4096 median,
n=44.413MUD/4096 median,
n=91.316GB/s of RC application payload
1.6–2.0%observed spread for clean IB modes
Standard interface MTUs
5,000,000 × 256 B unicast · 8 MiB receiver window · median of comparable runs
InfiniBand
VMA UDP
kernel UDP
The defensible IB comparison
RC is 16.5% faster than UD on the same fabric at the same MTU. This isolates queue-pair mode without changing the hosts, payload, receiver window, or packet size.
The 3.40× number needs context
UD is 3.40× kernel UDP on 100 GbE at MTU 1500, but it also has a larger channel MTU and bypasses the kernel. VMA UDP reaches 90.8% of UD, showing how much of the apparent lead belongs to the network stack.
| Transport | Aeron MTU | n | Median Mmsg/s | Payload GB/s | Observed range | Validation |
|---|---|---|---|---|---|---|
| InfiniBand RC | 4096 | 4 | 5.140 | 1.316 | 5.065–5.166 | full, 0/0 |
| InfiniBand UD | 4096 | 9 | 4.413 | 1.130 | 4.390–4.462 | full, 0/0 |
| 100 GbE VMA UDP | 1408 | 3 | 4.005 | 1.025 | 3.980–4.050 | errors not sampled |
| 100 GbE kernel UDP | 1408 | 10 | 1.298 | 0.332 | 0.531–1.364 | errors partly sampled |
| 10 GbE kernel UDP | 1408 | 1 | 1.285 | 0.329 | — | full, 0/0 |
Measurement discipline: rates are publisher totals; GB/s is decimal application payload, not wire throughput. No outlier was removed. Every summarized run delivered all five million messages. The VMA harness verified delivery and absence of its known fallback message, but did not invoke ErrorStat.
Frame size
Jumbo UDP tops the throughput table.
Raise the Ethernet interfaces from MTU 1500 to 9000 and the ranking changes. VMA UDP over 100 GbE reaches 6.218 Mmsg/s. Kernel UDP reaches 4.666 Mmsg/s and passes default InfiniBand UD. The small-frame runs were limited by packet processing before they filled the links.
Ethernet MTU 1500 → 9000
Aeron UDP MTU 1408 → 8960 · InfiniBand UD/4096 shown for reference
standard Ethernet MTU
jumbo Ethernet MTU
InfiniBand UD reference
| Transport | Aeron MTU | n | Median Mmsg/s | Payload GB/s | Validation |
|---|---|---|---|---|---|
| 100 GbE VMA UDP | 8960 | 3 | 6.218 | 1.592 | errors not sampled |
| InfiniBand RC | 16384 | 3 | 5.613 | 1.437 | full, 0/1 each run |
| InfiniBand RC | 8960 | 4 | 5.577 | 1.428 | full, 0/1 each run |
| InfiniBand RC | 4096 | 4 | 5.140 | 1.316 | full, 0/0 |
| 100 GbE kernel UDP | 8960 | 3 | 4.666 | 1.194 | full, 0/0 |
| InfiniBand UD | 4096 | 9 | 4.413 | 1.130 | full, 0/0 |
| 10 GbE kernel UDP | 8960 | 3 | 4.216 | 1.079 | full, 0/0 |
The larger RC numbers are recorded, not promoted as clean results. Every successful RC run above 4096 bytes delivered all messages but preserved err=0/1; the raw logs kept the count, not the error text. RC/32768 was also unstable and is excluded from the ranking.
Run it
Build it. Measure it.
The transport compiles when the rdma-core headers and libraries are present. Unit tests cover parsing and dispatch without a fabric; the two-host script validates UD unicast, UD multicast, and UDP coexistence on hardware.
aeron / ib-rdma-transport
# URI parsing, media selection, and dispatch $ ctest -R ib_channel_transport_test # Hardware path: IB unicast, multicast, then UDP $ aeron-driver/src/test/c/ib/run_ib_two_node_test.sh \ --pub-host <host> --sub-host <host> --build-dir <dir> # Omit the extension explicitly $ cmake -DAERON_ENABLE_IB=OFF ...