Hardware TCP offloading focused on the MQTT protocol

Distributing MQTT Publish packets loads the broker host's CPU, and moving that work to a SmartNIC means the card has to handle TCP as an active endpoint.

Two CodiLime engineers set out what a partial TCP offload must handle, from SEQ/ACK desynchronization to retransmissions, then follow three proofs of concept on NVIDIA BlueField 2 DPUs that significantly reduced distribution time and CPU load.

The main topics covered in this e-book:

Which Ethernet, IP and TCP guarantees a partial protocol offload must satisfy
SEQ/ACK desynchronization, ACK matching, delayed ACK and Nagle's algorithm in practice
Three designs on NVIDIA BlueField 2: TCP monitoring, lwIP stack, socket interception
JMeter measurements of MQTT Publish distribution time against direct broker-client traffic
A Rust libc shim forwarding socket calls over DOCA Comm Channel with Cap'n Proto

Hardware TCP offloading focused on the MQTT protocol — document render

Why download this e-book?

What does a TCP-based protocol offload have to handle?
The ebook lists the Ethernet, IP and TCP guarantees a partial offload must still satisfy, and shows how much work happens under the hood of a contemporary TCP/IP network stack.
Which offload designs were tried on a SmartNIC?
Three proofs of concept on NVIDIA BlueField 2 DPUs, in the order they were built: TCP monitoring, network stack integration with lwIP, and socket interception. Each comes with its design, test results and lessons learned.
What did the tests show?
Even the simplest design, TCP monitoring, significantly reduced MQTT Publish packet distribution time. The proofs of concept also lowered the CPU load on the broker host.
How does the offload fit the existing broker?
The publication explains how the offload integrates with the host application completely transparently, so the broker source code stays as it is.